Feedback system for optimizing display operations

Abstract

A system includes a projector module configured to project visual media to a user, a user tracking sensor configured to sense whether the user is looking at the visual media, and a control module configured to detect whether the user is looking at the projector module based on the user tracking sensor, in response to detecting that the user is looking at the projector module, the projector module is controlled to initiate a projection of visual media to the user, detect an interruption of viewing the visual media by the user based on the user tracking sensor, and in response to detecting the interruption of viewing the visual media by the user, the projector module is controlled to suspend the projection of the visual media to the user. Other exemplary systems and methods for controlling visual media projection of a user in a vehicle are also disclosed.

Description

[0001] introduce

[0002] The information provided in this section is for the purpose of generally presenting the context of the present disclosure. From the description in this section, the work of the currently named inventors and the aspects of the description that may not constitute prior art at the time of filing should neither be explicitly nor implicitly admitted to be prior art of the present disclosure.

[0003] The present disclosure relates to a feedback system for optimizing display operation.

[0004] Vehicles are often used in ride-sharing scenarios. In such examples, multiple passengers sometimes share the same vehicle, including the same passenger space (e.g., rear passenger space) for the duration of the trip. In some cases, the ride-sharing vehicle may have an interactive display system, such as a virtual display system, to provide information, entertainment, etc. to the passengers. Summary of the invention

[0005] A system for controlling visual media projection of a user in a vehicle, comprising: a projector module configured to project visual media to a user in the vehicle; a user tracking sensor configured to sense whether the user is looking at the visual media; and a control module in communication with the projector module and the user tracking sensor. The control module is configured to detect whether the user is looking at the projector module based on a first signal from the user tracking sensor, control the projector module to start visual media projection to the user in response to detecting that the user is looking at the projector module, detect an interruption of the user's viewing of the visual media based on a second signal from the user tracking sensor, and control the projector module to pause visual media projection to the user in response to detecting an interruption of the user's viewing of the visual media.

[0006] In other features, the user tracking sensor includes at least one of an eye tracking sensor and a face tracking sensor.

[0007] In other features, the system also includes an occupancy detection sensor configured to sense parameters associated with a seat in the vehicle, a control module communicating with the occupancy detection sensor, and the control module configured to detect whether the seat is occupied by an object representing an individual based on the sensed parameters, and to control the projector module to initiate projection of visual media to the user in response to detecting that the seat is occupied by an object representing an individual.

[0008] In other features, the occupancy detection sensor includes at least one of a biometric sensor positioned in the seat and a weight sensor positioned in the seat.

[0009] In other features, the projector module includes a retinal projector module configured to project visual media directly onto a retina of a user in the vehicle.

[0010] In other features, the control module is configured to generate an alert to a user in response to detecting the interruption.

[0011] In other features, the alarm includes at least one of an audible alarm and a tactile alarm.

[0012] In other features, the control module is configured to detect that the user is looking at the projector module again after the interruption, and in response to detecting that the user is looking at the projector module again after the interruption, control the projector module to restart projection of the visual media to the user.

[0013] In other features, the control module is configured to control the projector module to restart projection of the visual media at an interruption point in the visual media.

[0014] In other features, the control module is configured to detect that the user looks at the projector module again after the interruption, determine a period of time between detecting an interruption in the user's viewing of visual media and detecting that the user looks at the projector module again after the interruption, and control the projector module to pause projection of visual media to the user only if the period of time is greater than a defined threshold.

[0015] In other features, a vehicle includes the system, a first seat, and a second seat opposite the first seat, and the projector module is positioned in the first seat and configured to project visual media toward the second seat.

[0016] In other features, a user tracking sensor is positioned in the first seat and aligned with the projector module.

[0017] A method for controlling visual media projection for a user in a vehicle, comprising: detecting whether a user is looking at a projector module based on a first signal received from a user tracking sensor; in response to detecting that the user is looking at the projector module, controlling the projector module to start visual media projection for the user; detecting an interruption in the user's viewing of the visual media based on a second signal from the user tracking sensor; and, in response to detecting an interruption in the user's viewing of the visual media, controlling the projector module to pause visual media projection for the user.

[0018] In other features, the method further includes detecting whether the seat is occupied by an object representing a person based on sensed parameters received from the occupancy detection sensor.

[0019] In other features, controlling the projector module to initiate projection of the visual media to the user includes controlling the projector module to initiate projection of the visual media in response to detecting that the seat is occupied by an object representing the individual.

[0020] In other features, the user tracking sensor includes at least one of an eye tracking sensor and a face tracking sensor, and the occupancy detection sensor includes at least one of a biometric sensor positioned in the seat and a weight sensor positioned in the seat.

[0021] In other features, the method further comprises generating an alert to a user in response to detecting the interruption.

[0022] In other features, the alarm includes at least one of an audible alarm and a tactile alarm.

[0023] In other features, the method further comprises detecting that the user is looking at the projector module again after the interruption, and in response to detecting that the user is looking at the projector module again after the interruption, controlling the projector module to restart projection of the visual media to the user.

[0024] In other features, controlling the projector module to restart projection of the visual media to the user includes controlling the projector module to restart projection of the visual media at an interruption point in the visual media.

[0025] In other features, the method further includes detecting that the user again looks at the projector module after the interruption, and determining a period between detecting an interruption in the user viewing of the visual media and detecting that the user again looks at the projector module after the interruption.

[0026] In other features, controlling the projector module to pause projection of the visual media includes controlling the projector module to pause projection of the visual media only if the time period is greater than a defined threshold.

[0027] Further areas of applicability of the present disclosure will become apparent from the detailed description, the claims and the accompanying drawings.The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present disclosure should be more fully understood from the detailed description and accompanying drawings, in which:

[0029] Figure 1 is a functional block diagram of an exemplary system for controlling projection of visual media to an occupant according to the present disclosure;

[0030] Figure 2 According to the present disclosure, Figure 1 systems that are part of a vehicle;

[0031] Figure 3 is a functional block diagram of an exemplary system in a vehicle in which an exiting occupant interrupts another occupant's viewing of visual media in accordance with the present disclosure;

[0032] Figure 4is a functional block diagram of an exemplary system in a vehicle according to the present disclosure, wherein an object on a seat interrupts an occupant's viewing of visual media;

[0033] Figure 5 is a functional block diagram of an exemplary system in a vehicle according to the present disclosure in which a seated occupant interrupts another occupant's viewing of visual media; and

[0034] Figure 6-7 is a flow chart of an exemplary control process for controlling projection of visual media to an occupant according to the present disclosure. DETAILED DESCRIPTION

[0035] In a ride-sharing scenario, multiple passengers in a vehicle may share the same passenger space for the duration of the trip. Sometimes, such a ride-sharing vehicle may have an interactive display system (e.g., a virtual display system) for the passengers. In some cases, during entry and / or exit of the vehicle, etc., one occupant's display may be obscured by another occupant and / or object due to the position of the occupant and / or object. The obstruction of the display may interrupt the occupant's ability to view and / or interact with the display system.

[0036] The system according to the present disclosure enables detection of occlusion of the display and then, in response to the occlusion, subsequent control of the display to re-interface with the viewer. As a result, the inconvenience caused by the interruption to the viewer can be reduced, and in some cases, the inconvenience caused by the interruption to the viewer can be minimized.

[0037] Reference now Figure 1 , a block diagram of an exemplary system 100 for controlling the projection of visual media to a user (eg, occupant, viewer, passenger, etc.) in a vehicle is shown. Figure 1 As shown in FIG, system 100 generally includes a control module 102, various sensors, one or more projector modules 104, and an optional alarm generator 114. Figure 1 In an example of the system 100, the sensors may include one or more user tracking sensors (e.g., facial tracking sensors 110, eye tracking sensors 112, etc.) and / or one or more occupancy detection sensors (e.g., occupant classification sensors 106, biometric sensors 108, etc.). In various embodiments, the system 100 may include two or more occupant classification sensors 106, two or more biometric sensors 108, two or more facial tracker sensors 110, and two or more eye tracker sensors 112.

[0038] although Figure 1The system 100 is shown as including specific modules, but it should be understood that one or more other modules may be used if desired. In addition, although the system 100 is shown as including multiple separate modules, any combination of modules and / or their functions may be integrated into one or more modules. Figure 1 The system 100 is shown as including certain sensors, but it should be understood that the system 100 and / or other systems may also include more or fewer sensors, sensors with different functions, etc.

[0039] In various embodiments, the modules and sensors of the system 100 may communicate with each other and may share parameters via a network 116, such as a controller (or car) area network (CAN). In such examples, the parameters may be shared via one or more data buses of the network 116. In this way, a given module and / or sensor may make various parameters available to other modules and / or sensors via the network 116.

[0040] Figure 1 The system 100 may be used on any suitable vehicle, such as an electric vehicle (e.g., a pure electric vehicle, a plug-in hybrid electric vehicle, etc.), an internal combustion engine vehicle, etc. In addition, the system 100 may be used on an autonomous vehicle, a semi-autonomous vehicle, etc. For example, Figure 2 Depicted include Figure 1 The system 100 is a vehicle 200 that includes a control module 102 and a projector module 104 and one or more sensors 250 (e.g., an occupant classification sensor 106, a heart rate sensor 108, a face tracker sensor 110, an eye tracker sensor 112, etc.) in communication with the control module 102 and / or the projector module 104. Although the system 100 is generally described with respect to one vehicle (e.g., the vehicle 200), it should be understood that the system 100 may also be implemented in other suitable applications, such as other modes of transportation (e.g., airplanes, trains, etc.), within a room (e.g., a waiting room, etc.), etc.

[0041] In various embodiments, a shared vehicle (e.g., vehicle 200) may allow multiple passengers to share the same passenger space for a period of time. In some examples, not all passengers will enter and exit the vehicle at the same time. This often creates a scenario where one or more passengers entering and / or exiting the vehicle may interfere with other passengers' viewing of visual media (e.g., via an infotainment system as further explained below) during entry and exit.

[0042] For example, Figure 3A system 300 of a vehicle in such a scenario is depicted. As shown, the system 300 includes two opposing seats 350, 352, an occupant (e.g., a passenger, a user, etc.) 354 located on the seat 350, and another occupant 356 getting off the vehicle. The system 300 also includes Figure 1 The control module 102, the sensor groups 106, 112 and the two projector modules 104. Figure 3 As shown, exiting occupant 356 interrupts occupant 354 from viewing visual media output from one of projector modules 104 .

[0043] In other examples, one or more physical objects may interfere with other passengers' viewing of visual media (e.g., via an infotainment system). For example, an occupant may place their personal items (e.g., bags, luggage, packages, animals, etc.) on unoccupied seats and in the viewing area of ​​other fellow occupants of the vehicle.

[0044] Figure 4 A system 400 of a vehicle in such a scenario is depicted. As shown, the system 400 includes Figure 3 The opposing seats 350 and 352, Figure 3 The system 400 also includes an occupant 354 located on the seat 350 and an object 456 located on the seat 352. Figure 3 The control module 102, the sensor groups 106, 112 and the projector module 104. Figure 4 As shown, object 456 interrupts occupant 354 from viewing visual media output from one of projector modules 104 .

[0045] exist Figure 3-4 In the example of FIG. 1 , the sensor groups 106, 112 and one projector module 104 are positioned on each seat 350, 352. With this arrangement, the projector module 104 and the eye tracker sensor 112 can be aligned so that the projector module 104 and the eye tracker sensor 112 have the same viewing angle relative to the passenger 354. In other examples, any of the sensors 106, 112 and / or the projector module 104 can be positioned at other suitable locations in the vehicle. For example, one of the projector modules 104 can be positioned on a vehicle compartment wall opposite to the occupant 354 and / or at other suitable locations that are generally in line with the occupant 354. In addition, if desired, the eye tracker sensor 112 can be offset from the projector module 104 and / or positioned in other suitable areas of the vehicle so that the eye tracker sensor 112 can observe the eye(s) of the occupant 354.

[0046] In addition, although Figure 3-4The systems 300, 400 are shown as including an occupant classification sensor 106 and an eye tracker sensor 112, but it should be understood that other suitable user tracking sensors (e.g., a facial tracker sensor 110, etc.) and / or other suitable occupancy detection sensors (e.g., a biometric sensor 108, etc.) may be employed if desired.

[0047] For example, Figure 5 Depicted with Figure 3-4 The systems 300, 400 are substantially similar to the system 500, but wherein the system 500 includes the biometric sensor 108. Figure 5 In the example of FIG. 5 , the system 500 also includes Figure 3 The opposing seats 350 and 352 are occupied by passengers 354 and 356 respectively. Figure 3 The control module 102, the sensor groups 106, 112 and the projector module 104 in FIG. Figure 5 As shown, occupant 356 interrupts occupant 354 from viewing visual media output from one of projector modules 104 .

[0048] Continue to refer Figure 1 , a shared vehicle can provide personalized or group infotainment features to the users in the vehicle. Such infotainment features are often implemented using a projection-based system (e.g., projector module 104). For example, as referenced herein, Figure 1 The projector module 104 generally projects visual media to the occupants of the vehicle. For example, when the projector module 104 is located at a seat (e.g., Figure 3-5 When the projector module 104 is on the seat 352 of the opposite seat (for example, Figure 3-5 Projecting visual media to occupants of the seat 350) and / or the opposing seat.

[0049] In various embodiments, projector module 104 can be any suitable device for projecting visual media. For example, projector module 104 can provide a virtual reality display to the occupant, which is physically projected directly into the pupil of the occupant. In such an example, projector module 104 can include a retinal projector module to project visual media directly onto one or both retinas of the occupant. In this example, the retina can be used as a display. In other examples, projector module 104 can rely on an external display. For example, projector module 104 can project visual media onto an intermediate display such as an optical element (e.g., a diffuser, etc.).

[0050] Figure 1The alarm generator 114 generally controls the activation of one or more alarms to notify one or more occupants of the vehicle of the interruption. For example, and as further explained herein, the alarm generator 114 may generate an alarm (e.g., an audible alarm, a tactile alarm, a visual alarm, etc.) to the occupant in response to detecting the interruption.

[0051] Figure 1 The occupancy detection sensor 106 generally detects or senses parameters associated with a seat in the vehicle. Thus, the occupancy detection sensor can help detect whether a seat is occupied. For example, the occupant classification sensor 106 can detect the size and / or shape of an object, the weight of an object, etc. For example, the occupant classification sensor 106 can be a sensor located on a seat (e.g., Figure 3-5 In other examples, the occupant classification sensor 106 may be a camera, a radar sensor, etc., for sensing the size and / or shape of an object. In addition and / or as an alternative, the biometric sensor 108 may be used to detect whether the seat is occupied. For example, the biometric sensor 108 may be a heart rate sensor, a breathing sensor, and / or other suitable sensors positioned in the seat for detecting the biometrics of the occupant (e.g., heart rate, breathing, etc.).

[0052] Figure 1 The user tracking sensors generally detect or sense whether the occupant is looking at visual media or looking in the direction of the projector module 104. For example, the facial tracker sensor 110 may include a camera for tracking the facial movements of the occupant. Similarly, the eye tracker sensor 112 may include a camera for tracking the eyes of the occupant. In various embodiments, the user tracking sensors may be positioned in the occupant's opposite seat or other suitable location and / or aligned with the projector module 104 as explained herein.

[0053] In some examples, the user tracking sensor may include other suitable types of sensors. For example, a location sensor such as a GPS sensor and / or other location data collection sensor may be employed to track the real-time location of each occupant in the vehicle.

[0054] Continue to refer Figure 1 , the control module 102 can detect whether the occupant is looking at the projector module 104. For example, the control module 102 can receive one or more signals from the face tracker sensor 110, the eye tracker sensor 112, and / or other suitable user tracking sensors that provide feedback indicating that the occupant is looking at the projector module 104. Based on the received signals, the control module 102 can determine whether the occupant is looking at the projector module 104.

[0055] Then, in response, the control module 102 may control the projector module 104 to initiate projection of visual media to the passenger. For example, after determining that the occupant is looking at the projector module 104, the control module 102 may send a control signal to the projector module 104, thereby instructing the projector module 104 to project the visual media. In such an example, the projector module 104 may project the visual media directly into the pupils of the occupant or onto an intermediate display, as explained herein.

[0056] In some examples, control module 102 can control various parameters associated with visual media. For example, an occupant can provide user input to control module 102 via gestures detected by one or more of the user tracking sensors, or otherwise interact with control module 102. In such examples, control module 102 can adjust volume, adjust image brightness, adjust content, etc. based on the detected occupant's gestures.

[0057] In various embodiments, the control module 102 can control the projector module 104 to project visual media to the occupant only when additional conditions apply. For example, the control module 102 can control the projector module 104 to start visual media projection in response to detecting that the seat is occupied by an object similar to an individual. For example, the control module 102 can receive one or more signals with sensing parameters from the occupant classification sensor 106, the biometric sensor 108 and / or other suitable occupancy detection sensors, and then detect whether the seat is occupied by an object representing an individual based on the sensing parameters. For example, the parameters can include sensed weight, detected size, detected shape, etc. After determining that the seat is occupied by an individual, the control module 102 can send a control signal to the projector module 104 to start visual media projection.

[0058] Additionally, the control module 102 can detect interruptions in the occupant's viewing of the visual media. In such an example, any obstruction of the direct path between the projector module 104 and / or the display and the occupant can interfere with viewing. For example, an occupant may be interrupted by another passenger or object (e.g., Figure 3 and Figure 5 Crew 356, Figure 4 The occupant may be unable to view the visual media due to an object 456 (such as an object in the vehicle 456, etc.) blocking the projected media, the occupant has looked away, etc. In other scenarios, the control module 102 may receive one or more additional signals from the facial tracker sensor 110, the eye tracker sensor 112, and / or other suitable user tracking sensors indicating that the occupant is no longer viewing the visual media. In other examples, the control module 102 may receive a signal based on GPS or other location data relative to the destination indicating that an interruption will occur soon (and therefore be detected). Then, based on the received signal, the control module 102 may detect an interruption in the occupant's viewing of the visual media.

[0059] The control module 102 may then control the projector module 104 to pause the projection of the visual media to the passenger in response to detecting the interruption. For example, the control module 102 may send a control signal to the projector module 104 to pause (e.g., stop projecting, etc.) the visual media so that the passenger may not miss a portion of the media due to the interruption (e.g., due to an occupant, an object, etc.).

[0060] In various embodiments, the control module 102 may again control the projector module 104 to project the visual media to the passenger. For example, in response to detecting that the occupant is looking at the projector module 104 again after the interruption (e.g., the obstruction no longer exists), the control module 102 may control the projector module 104 to restart the projection of the visual media to the passenger. In some examples, the control module 102 may control the projector module 104 to restart the projection at the interruption point in the visual media (e.g., the point when the media was paused). In other examples, the control module 102 may control the projector module 104 to restart the projection of the media in real time, as if the media was not paused.

[0061] In some embodiments, the control module 102 can detect the duration of the interruption and then continue projection from when the original occlusion occurred to when the occupant ends. For example, in response to detecting the interruption, the control module 102 can start a timer. Once the control module 102 detects that the occupant is looking at the projector module 104 again (e.g., the occupant no longer exists), the control module 102 can determine the period between detecting the initial interruption and detecting that the occupant is looking at the projector module again after the interruption based on the timer.

[0062] Then, the control module 102 can take different action plans based on the determined time period. For example, the control module 102 can compare the determined time period with a defined threshold. If the determined time period is greater than the defined threshold, the control module 102 can control the projector module 104 to pause the projection of visual media to the occupants, and then restart the projection at the interruption point. However, if the determined time period is less than the defined threshold, the control module 102 can control the projector module 104 to pause the projection of visual media to the occupants, and restart the projection of the media in real time as if the media was not paused. Alternatively, if the determined time period is less than the defined threshold, the projector module 104 may not pause the projection.

[0063] The defined threshold may be any suitable time period. For example, the defined threshold may be any suitable time from 10 milliseconds to 100 milliseconds. In various embodiments, the threshold may depend on the type of media being presented. In such an example, if the projected media is a movie, advertisement, etc., the time period may be short (e.g., representing a brief interruption). However, if the projected media is a map, a user-selected feature (e.g., an advertisement, a movie, a program, etc.), the time period may be longer. In other examples, the threshold may be based on the frame rate of the projected media. For example, if the frame rate is 60 frames per second, the threshold may be at least 15 milliseconds.

[0064] In various embodiments, if an interruption occurs, the control module 102 can generate an alert for the occupants. For example, after detecting the interruption (e.g., in response to detecting the interruption, a period of time after detecting the interruption, etc.), the control module 102 can send a control signal to the alert generator 114. In response, the alert generator 114 can activate one or more alerts to notify one or more occupants of the vehicle of the interruption. For example, the alert generator 114 can generate a visual alert, generate an audible alert (e.g., a voice command), generate a tactile alert, etc.

[0065] Figure 6-7 It shows that Figure 1 The system 100 is used to control a vehicle (eg, Figure 2 200). Although the exemplary control processes 600, 700 are referenced to a control module 102 including a vehicle 200, the exemplary control processes 600, 700 are described with reference to a control module 102. Figure 1 Although described with reference to the system 100 of the present invention, any of the control processes 600, 700 may be adopted by any suitable system.

[0066] exist Figure 6 , the control process 600 begins at 602, where the control module 102 detects whether a user (e.g., an occupant, a passenger, etc.) is looking at a projector module (e.g., the projector module 104). In various embodiments, the control module 102 may make this determination based on one or more signals from one or more user tracking sensors, such as those described above. Figure 1 The face tracker sensor 110, the eye tracker sensor 112, etc. of FIG. 602. If no at 602, control returns to 602. If yes, control proceeds to 604.

[0067] At 604, the control module 102 controls the projector module to initiate projection of visual media to the user. For example, as described above, the control module 102 may generate and send a control signal to the projector module, thereby instructing the projector module to project visual media. Control then proceeds to 606.

[0068] At 606, the control module 102 determines whether an interruption occurred while the user was viewing the visual media. In various embodiments, the control module 102 may determine whether an interruption occurred while the user was viewing the visual media based on the information from the above-described Figure 1 The determination is made based on one or more signals of the facial tracker sensor 110, the eye tracker sensor 112, etc. If not, control returns to 606, where the projector module continues to project visual media to the user. If yes, control proceeds to 608.

[0069] At 608, the control module 102 controls the projector module to pause the visual media projection. For example, as described above, this control can be accomplished by generating a control signal and sending it to the projector module, thereby instructing the projector module to pause. Control then proceeds to 610.

[0070] At 610 , the control module 102 determines the output signal based on, for example, Figure 1 The face tracker sensor 110, the eye tracker sensor 112, etc. of the control module 102 are used to detect whether the user is looking at the projector module again. If yes, the control proceeds to 612, where the control module 102 controls the projector module to restart the projection of visual media to the user as described above. Then, the control returns to 606.

[0071] If false at 610, control may proceed as follows Figure 6 For example, control may end after a defined period of time without user eye engagement (e.g., the interruption time exceeds a threshold). In other examples, control may also end after a defined number of attempts to detect whether the user is looking at the projector module again, etc. In such an example, control may return to the number of times defined at 610 before ending.

[0072] exist Figure 7 , the control process 700 begins at 702, where the control module 102 detects whether a user (e.g., an occupant, a passenger, etc.) is looking at a projector module (e.g., projector module 104). For example, as described above, the control module 102 may be based on Figure 1 The face tracker sensor 110, the eye tracker sensor 112, etc. of the embodiment of the present invention make this determination. If yes at 702, control proceeds to 704. If no, control proceeds to 712.

[0073] At 704, the control module 102 determines whether a user is detected. For example, the control module 102 may detect a user from one or more occupancy detection sensors (e.g., Figure 1 The control module 102 receives one or more signals having sensing parameters on the occupant classification sensor 106, the biometric sensor 108, etc., and then detects whether the seat is occupied by an object representing an individual based on the sensing parameters. If no at 704, control returns to 702. If yes, control proceeds to 706, where the control module 102 controls the projector module to project visual media to the user, as described above. Control then proceeds to 708.

[0074] At 708, the control module 102 determines whether the timer was previously set (e.g., turned on, started, etc.). If so, control returns to 702. If not, control proceeds to 710, where the control module 102 starts a timer to track the time since the start of projecting media. Then, control returns to 702.

[0075] At 712, the control module 102 determines whether a user is detected while the projector module is projecting visual media based on, for example, one or more occupancy detection sensors (e.g., as explained above). If yes at 712, control proceeds to 714, where the control module 102 determines whether the period corresponding to the interruption (e.g., when the user is not looking at the projector module determined at 702) is less than a defined threshold. If yes at 714, control returns to 706, where the control module 102 controls the projector module to project visual media to the user, as described above. If no at 714, control proceeds to 716. If no at 712, control proceeds to 716, where the control module 102 controls the projector module to pause or stop the projection of visual media, as described above. Then, control returns to 702. In such an example, if the control module 102 detects that the user is looking at the projector module again (at 702) and determines that the user is an occupant of the seat (at 704), the control module 102 can control the projector module to restart the visual media projection at the interruption point (as described above).

[0076] Some of the aspects explained above are summarized below by means of numbered examples.

[0077] Example 1. A system for controlling visual media projection for a user in a vehicle, the system comprising:

[0078] a projector module configured to project visual media to a user in the vehicle;

[0079] a user tracking sensor configured to sense whether the user is looking at the visual media; and

[0080] a control module in communication with the projector module and the user tracking sensor, the control module being configured to:

[0081] detecting whether the user is looking at the projector module based on a first signal from the user tracking sensor;

[0082] In response to detecting that the user is looking at the projector module, controlling the projector module to initiate projection of visual media to the user;

[0083] detecting an interruption in viewing of the visual media by the user based on a second signal from the user tracking sensor; and

[0084] In response to detecting an interruption in viewing of the visual media by the user, the projector module is controlled to pause projecting the visual media to the user.

[0085] Example 2. The system of claim 1, wherein the user tracking sensor comprises at least one of an eye tracking sensor and a face tracking sensor.

[0086] Example 3. The system of Example 1, further comprising an occupancy detection sensor configured to sense a parameter associated with a seat in the vehicle, wherein:

[0087] the control module being in communication with the occupancy detection sensor; and

[0088] The control module is configured to detect whether the seat is occupied by an object representing an individual based on the sensed parameters, and to control the projector module to initiate projection of visual media to the user in response to detecting that the seat is occupied by the object representing an individual.

[0089] Example 4. The system of Example 3, wherein the occupancy detection sensor comprises at least one of a biometric sensor located in the seat and a weight sensor located in the seat.

[0090] Example 5. The system of Example 1, wherein the projector module comprises a retinal projector module configured to project the visual media directly onto a retina of the user in the vehicle.

[0091] Example 6. The system of example 1, wherein the control module is configured to generate an alert to the user in response to detecting an interruption.

[0092] Example 7. The system of Example 6, wherein the alarm comprises at least one of an audible alarm and a tactile alarm.

[0093] Example 8. A system according to Example 1, wherein the control module is configured to detect that the user looks at the projector module again after an interruption, and in response to detecting that the user looks at the projector module again after an interruption, control the projector module to restart the projection of visual media to the user.

[0094] Example 9. The system of Example 8, wherein the control module is configured to control the projector module to restart the projection of the visual media at an interruption point in the visual media.

[0095] Example 10. A system according to Example 1, wherein the control module is configured to detect that the user looks at the projector module again after an interruption, determine a time period between detecting an interruption in the user's viewing of the visual media and detecting that the user looks at the projector module again after the interruption, and control the projector module to pause projection of the visual media to the user only if the time period is greater than a defined threshold.

[0096] Example 11. A vehicle comprising the system according to Example 1, a first seat, and a second seat opposite the first seat, wherein the projector module is positioned on the first seat and is configured to project visual media toward the second seat.

[0097] Example 12. The vehicle of Example 11, wherein the user tracking sensor is positioned on the first seat and aligned with the projector module.

[0098] Example 13. A method for controlling visual media projection for a user in a vehicle, the method comprising:

[0099] detecting whether the user is looking at the projector module based on a first signal received from a user tracking sensor;

[0100] In response to detecting that the user is looking at the projector module, controlling the projector module to initiate projection of visual media to the user;

[0101] detecting an interruption in viewing of the visual media by the user based on a second signal from the user tracking sensor; and

[0102] In response to detecting an interruption in viewing of the visual media by the user, the projector module is controlled to pause projecting the visual media to the user.

[0103] Example 14. The method according to Example 13, wherein the method further comprises:

[0104] detecting whether the seat is occupied by an object representing an individual based on sensing parameters received from the occupancy detection sensor; and

[0105] Controlling the projector module to initiate projection of visual media to the user includes controlling the projector module to initiate projection of the visual media in response to detecting that a seat is occupied by an object representing an individual.

[0106] Example 15. The method according to Example 14, wherein:

[0107] The user tracking sensor comprises at least one of an eye tracking sensor and a face tracking sensor; and

[0108] The occupancy detection sensor includes at least one of a biometric sensor positioned in the seat and a weight sensor positioned in the seat.

[0109] Example 16. The method of Example 13, further comprising generating an alert for the user in response to detecting an interruption.

[0110] Example 17. The method of Example 16, wherein the alarm comprises at least one of an audible alarm and a tactile alarm.

[0111] Example 18. The method according to Example 13, further comprising:

[0112] detecting that the user is looking at the projector module again after the interruption; and

[0113] In response to detecting that the user is again looking at the projector module after the interruption, the projector module is controlled to restart projection of visual media to the user.

[0114] Example 19. The method of Example 18, wherein controlling the projector module to restart projection of the visual media to the user comprises controlling the projector module to restart projection of the visual media at an interruption point in the visual media.

[0115] Example 20. The method of Example 13, wherein the method further comprises:

[0116] detecting that the user again looks at the projector module after the interruption, and determining a period between detecting an interruption in viewing of visual media by the user and detecting that the user again looks at the projector module after the interruption; and

[0117] Controlling the projector module to pause the visual media projection includes controlling the projector module to pause the visual media projection only when the time period is greater than a defined threshold.

[0118] The foregoing description is essentially illustrative only, and there is no intention to limit the present disclosure, its application or use. The broad teachings of the present disclosure can be implemented in various forms. Therefore, although the present disclosure includes specific examples, the true scope of the present disclosure should not be limited thereto, because after studying the drawings, the specification and the appended claims, other modifications will become apparent. It should be understood that one or more steps in the method can also be performed in different orders (or simultaneously) without changing the principles of the present disclosure. In addition, although each of the embodiments is described above as having certain features, any one or more of those features described with reference to any embodiment of the present invention can also be implemented in the features of any one of the other embodiments and / or combined with the features of any one of the other embodiments, even if the combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and the replacement of one or more embodiments with each other remains within the scope of the present disclosure.

[0119] Various terms are used to describe the spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.), including "connected," "engaged," "coupled," "adjacent," "next to," "on top of," "above," "below," and "disposed." Unless explicitly described as "direct," when a relationship between a first and a second element is described in the above disclosure, the relationship can be a direct relationship in which there are no other intervening elements between the first and second elements, but can also be an indirect relationship in which there are one or more intervening elements (spatially or functionally) between the first and second elements. As used herein, the phrase at least one of A, B, and C should be interpreted to mean a logical (A or B or C) using a non-exclusive logical "or," and should not be interpreted to mean "at least one of A, at least one of B, and at least one of C."

[0120] In the drawings, the direction of the arrow, as indicated by the arrow, generally represents the flow of information (e.g., data or instructions) of interest to the illustration. For example, when component A and component B exchange various information, but the information transmitted from component A to component B is relevant to the illustration, the arrow may point from component A to component B, and the unidirectional arrow does not imply that no other information is transmitted from component B to component A. In addition, for information sent from component A to component B, component B may also send a request for the information or a confirmation of receipt of the information to component A.

[0121] In this application, including the following definitions, the term "module" or the term "controller" may be replaced with the term "circuit". The term "module" may refer to, be part of, or include: an application specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the functionality; or a combination of some or all of the above, such as in a system on a chip.

[0122] The module may include one or more interface circuits. In some examples, the interface circuit may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functions of any given module of the present disclosure may be distributed in multiple modules connected via the interface circuits. For example, multiple modules may allow load balancing. In other examples, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.

[0123] The term "code" as used above may include software, firmware and / or microcode, and may refer to a program, a routine, a function, a class, a data structure and / or an object. The term "shared processor circuit" covers a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" covers a processor circuit that executes some or all of the code from one or more modules in combination with an additional processor circuit. References to multi-processor circuits cover multi-processor circuits on discrete dies, multi-processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" covers a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" covers a memory circuit that stores some or all of the code from one or more modules in combination with additional memory.

[0124] The term "memory circuit" is a subset of the term "computer-readable medium". The term "computer-readable medium" as used herein does not encompass transient electrical or electromagnetic signals propagated through a medium (such as on a carrier wave). Therefore, the term "computer-readable medium" may be considered to be both tangible and non-transitory. Non-limiting examples of non-transitory, tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tapes or hard drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0125] The apparatus and methods described in this application may be implemented in part or in whole by a special purpose computer created by configuring a general purpose computer to perform one or more specific functions contained in a computer program. The above-mentioned function blocks, flow chart components and other elements are used as software specifications, which can be converted into a computer program by routine work of a skilled technician or programmer.

[0126] The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include or rely on stored data. The computer program may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0127] A computer program may include: a descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language) or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated by a compiler from a source code; (iv) source code executed by an interpreter; (v) source code compiled and executed by a just-in-time compiler, etc. For example, a computer program may be used from languages ​​including C, C++, C#, Objective C, Swift, Haskell, Go, SQL, R, Lisp, Fortran, Perl, Pascal, Culli, OCaml, HTML5 (Hypertext Markup Language, Revision 5), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Visual Lua, MATLAB, SIMULINK, and The syntax of the language is used to write source code.

Claims

1. A system for controlling projection of visual media to a user in a vehicle, the system comprising: a projector module configured to project visual media to a user in the vehicle; a user tracking sensor configured to sense whether the user is looking at the visual media; as well as a control module in communication with the projector module and the user tracking sensor, the control module being configured to: detecting whether the user is looking at the projector module based on a first signal from the user tracking sensor; In response to detecting that the user is looking at the projector module, controlling the projector module to initiate projection of visual media to the user; detecting an interruption in viewing of the visual media by the user based on a second signal from the user tracking sensor; as well as In response to detecting an interruption in viewing of the visual media by the user, the projector module is controlled to pause projecting the visual media to the user.

2. The system according to claim 1, wherein: The user tracking sensor includes at least one of an eye tracking sensor and a face tracking sensor.

3. The system of claim 1 , further comprising an occupancy detection sensor configured to sense a parameter associated with a seat in the vehicle, wherein: the control module being in communication with the occupancy detection sensor; as well as The control module is configured to detect whether the seat is occupied by an object representing an individual based on the sensed parameters, and to control the projector module to initiate projection of visual media to the user in response to detecting that the seat is occupied by the object representing an individual.

4. The system according to claim 3, wherein: The occupancy detection sensor includes at least one of a biometric sensor located in the seat and a weight sensor located in the seat.

5. The system according to claim 1, wherein: The projector module includes a retinal projector module configured to project the visual media directly onto a retina of the user in the vehicle.

6. The system according to claim 1, wherein: The control module is configured to detect that the user is looking at the projector module again after the interruption, and in response to detecting that the user is looking at the projector module again after the interruption, control the projector module to restart projection of visual media to the user.

7. The system according to claim 6, wherein: The control module is configured to control the projector module to restart the projection of the visual media at an interruption point in the visual media.

8. The system according to claim 1, wherein: The control module is configured to detect that the user looks at the projector module again after an interruption, determine a time period between detecting an interruption in the user's viewing of the visual media and detecting that the user looks at the projector module again after the interruption, and control the projector module to pause projection of the visual media to the user only if the time period is greater than a defined threshold.

9. A vehicle comprising the system according to claim 1, a first seat and a second seat opposite to the first seat, wherein: The projector module is positioned on the first seat and is configured to project visual media toward the second seat.

10. The vehicle according to claim 9, wherein: The user tracking sensor is positioned on the first seat and aligned with the projector module.

Images