Method for controlling the output of an audio data stream or video data stream in an audio zone of a vehicle, computer-readable medium, system, and vehicle

Abstract

The invention relates to a method for controlling the output of an audio data stream or a video data stream in an audio zone of a vehicle, the method comprising: providing a first video data stream on a first display device in the interior of the vehicle; providing a second video data stream on a second display device in the interior of the vehicle; determining whether a first occupant of the vehicle's audio zone is paying attention to the first video data stream on the first display device and to the second video data stream of the second display device; determining whether a second occupant of the vehicle's audio zone is paying attention to the first video data stream of the first display device and to the second video data stream of the second display device;and controlling the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the vehicle's audio zone depending on the attention of the first occupant of the vehicle's audio zone and the attention of the second occupant of the vehicle's audio zone.

Description

[0001] The invention relates to a method for controlling the output of an audio data stream of a video data stream in an audio zone of a vehicle. The invention further relates to a computer-readable medium for controlling the output of an audio data stream of a video data stream in an audio zone of a vehicle, a system for controlling the output of an audio data stream of a video data stream in an audio zone of a vehicle, and a vehicle comprising the system for controlling the output of an audio data stream of a video data stream in an audio zone of a vehicle.

[0002] Current vehicles may have a screen for passenger entertainment and one or more screens in a second row of seats that can display different content, such as different video streams. However, current vehicles have a smaller number of auditory zones compared to the number of screens that can display content. This can lead to interference between auditory zones, and for each auditory zone, it is necessary to determine which audio stream should be output to the occupants of the auditory zone among multiple video streams.

[0003] DE 10 2021 126 916 A1 describes a method for operating a vehicle infotainment system. The method includes determining whether a passenger in the front passenger seat or a rear seat is using an end-user device; and adjusting an output mode of the infotainment system at the passenger's seat depending on the determined use of the end-user device.

[0004] US 2013 / 0030645 A1 describes an infotainment system for providing content to multiple occupants of a vehicle. The infotainment system includes an occupant detector configured to receive characteristic data of the vehicle occupants and create a profile for each occupant of the vehicle, a recommendation engine that analyzes the profiles of the vehicle occupants, and a content delivery engine that delivers content to one or more vehicle occupants based on the analysis of the occupant profiles.

[0005] It is therefore an object of the invention to efficiently improve the output of an audio data stream of a video data stream in an audio zone of a vehicle. In particular, an object of the invention is to efficiently improve the output of an audio data stream to a plurality of video data streams in an audio zone of a vehicle.

[0006] This object is achieved by the features of the independent claims. Advantageous embodiments and further developments of the invention emerge from the dependent claims.

[0007] According to a first aspect, the invention is characterized by a method for controlling the output of an audio data stream or a video data stream in an audio zone of a vehicle. The method can be a computer-implemented method and / or a control unit-implemented method. The vehicle can be a motor vehicle. The method comprises providing a first video data stream on a first display device in an interior of the vehicle and providing a second video data stream on a second display device in the interior of the vehicle.The method further comprises determining an attention level of a first occupant of the audio zone of the vehicle with respect to the first video data stream on the first display device and the second video data stream of the second display device, and determining an attention level of a second occupant of the audio zone of the vehicle with respect to the first video data stream of the first display device and the second video data stream of the second display device. The method further comprises controlling an output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone of the vehicle as a function of the attention level of the first occupant of the audio zone of the vehicle and the attention level of the second occupant of the audio zone of the vehicle.

[0008] Advantageously, the method can dynamically control an audio data stream for individual vehicle occupants based on their attention. This can efficiently improve the comfort of audio output from a multitude of video data streams on displays in the interior.

[0009] According to an advantageous embodiment, the method may further comprise determining a content type of the first video data stream, determining a content type of the second video data stream, and controlling an output of the audio data stream of the first video data stream or the second video data stream in the audio zone of the vehicle depending on the attention of the first occupant of the vehicle, the attention of the second occupant of the vehicle, the content type of the first video data stream, and the content type of the second video data stream. This allows the output of the audio data stream to be controlled more precisely.

[0010] According to a further advantageous embodiment, the first display device and / or the second display device can be a display, a head-up display, a mobile device coupled to the vehicle, and / or a projection display in the interior of the vehicle. This allows the audio output for a variety of display devices to be efficiently controlled.

[0011] According to a further advantageous embodiment, the attention of the first occupant of the vehicle's audio zone and / or the second occupant of the vehicle's audio zone can be determined depending on the respective occupant's viewing direction toward the first video data stream of the first display device and / or the second video data stream of the second display device. The viewing direction of the respective occupant of the vehicle's audio zone can preferably be captured using one or more interior cameras of the vehicle. This allows the attention of multiple occupants with respect to multiple video data streams from display devices of the vehicle to be efficiently determined.

[0012] According to a further advantageous embodiment, the attention of the first occupant of the vehicle's audio zone and / or the second occupant of the vehicle's audio zone can be determined depending on the number of fixations of the respective occupant on the first video data stream of the first display device and / or the second video data stream of the second display device. The number of fixations of the respective occupant of the vehicle's audio zone can be recorded using one or more interior cameras of the vehicle. This allows the attention of multiple occupants with respect to multiple video data streams from display devices of the vehicle to be determined efficiently.

[0013] According to a further advantageous embodiment, the method can further comprise determining a number of fixations of the first occupant on the first video data stream of the first display device and / or on the second video data stream of the second display device relative to a total number of fixations of the first occupant within a sliding time interval with a predetermined time duration, and determining a number of fixations of the second occupant on the first video data stream of the first display device and / or on the second video data stream of the second display device relative to a total number of fixations of the second occupant within a sliding time interval with a predetermined time duration.The attention of the first occupant of the audio zone of the vehicle with respect to the first video data stream of the first display device and the second video data stream of the second display device may correspond to the determined number of fixations of the first occupant on the first video data stream of the first display device and / or on the second video data stream of the second display device relative to the total number of fixations of the first occupant within the sliding time interval with the predetermined time duration.The attention of the second occupant of the audio zone of the vehicle with respect to the first video data stream of the first display device and the second video data stream of the second display device can correspond to the determined number of fixations of the second occupant on the first video data stream of the first display device and / or on the second video data stream of the second display device relative to the total number of fixations of the second occupant within the sliding time interval with the predetermined time duration. This allows the attention of multiple occupants with respect to multiple video data streams from display devices of the vehicle to be efficiently determined.

[0014] According to a further advantageous embodiment, the number of fixations of the respective occupant in the audio zone of the vehicle and the total number of fixations of the respective occupant in the audio zone of the vehicle within the sliding time interval with the specified time duration can be recorded using one or more interior cameras of the vehicle. This allows for efficient recording of occupant gaze fixations with respect to video data streams from the vehicle's display devices.

[0015] According to a further advantageous embodiment, controlling the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone of the vehicle depending on the attention of the first occupant of the audio zone of the vehicle and the attention of the second occupant of the audio zone of the vehicle can comprise outputting the audio data stream associated with the first video data stream or the second video data stream in the audio zone of the vehicle whose occupant has the highest attention of all occupants of the audio zone with regard to the first video data stream or the second video data stream. This allows the output of the audio data stream to be controlled efficiently.

[0016] According to a further advantageous embodiment, controlling the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone of the vehicle depending on the attention of the first occupant in the audio zone of the vehicle and the attention of the second occupant in the audio zone of the vehicle can include outputting the audio data stream associated with the first video data stream or the second video data stream on a mobile audio output device of the respective occupant coupled to the vehicle if the attention of the respective occupant does not correspond to the highest attention of all occupants in the audio zone of the vehicle. This allows the output of the audio data stream to be controlled efficiently.

[0017] According to a further advantageous embodiment, controlling the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone of the vehicle depending on the attention of the first occupant of the audio zone of the vehicle and the attention of the second occupant of the audio zone of the vehicle can include delaying a change in the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone of the vehicle if the attention of the vehicle occupant with the highest attention of all occupants of the audio zone no longer corresponds to the highest attention of all occupants of the vehicle for a predetermined delay time interval. This allows the output of the audio data stream to be controlled efficiently.

[0018] According to a further aspect, the invention is characterized by a computer-readable medium for controlling an output of an audio data stream of a video data stream in an audio zone of a vehicle, wherein the computer-readable medium comprises instructions which, when executed on a computer and / or a control unit, carry out the method described above.

[0019] According to a further aspect, the invention is characterized by a system for controlling an output of an audio data stream of a video data stream in an audio zone of a vehicle, wherein the system is designed to carry out the method described above.

[0020] According to a further aspect, the invention is characterized by a vehicle comprising the above-described system for controlling an output of an audio data stream of a video data stream in an audio zone of the vehicle.

[0021] Further features of the invention emerge from the claims, the figures, and the description of the figures. All features and combinations of features mentioned above in the description, as well as the features and combinations of features mentioned below in the description of the figures and / or shown alone in the figures, can be used not only in the respective specified combination, but also in other combinations or even on their own.

[0022] A preferred embodiment of the invention is described below with reference to the accompanying drawings. Further details, preferred embodiments, and developments of the invention will emerge from these. In detail, schematically Fig. 1 shows an exemplary method for controlling an output of an audio data stream of a video data stream in an audio zone of a vehicle, and Fig. 2 an exemplary system for controlling an output of an audio stream of a video stream in an audio zone of a vehicle.

[0023] In detail, Fig. 1 shows a method 100 for controlling the output of an audio data stream of a video data stream in an audio zone of a vehicle. The method can couple the output of the audio data stream of a video data stream to the attention of vehicle occupants. The audio zone of the occupant with the highest attention to a video data stream of a display device of the vehicle can be determined by the method for outputting the audio data stream of the video data stream.

[0024] The method 100 may provide a first video data stream on a first display device in an interior of the vehicle. For example, the first display device may be a display device of an entertainment system of a second row of seats in a vehicle. For example, the first display device may be a display device of an entertainment system of a front passenger of the vehicle. The method 100 may provide 104 a second video data stream on a second display device in the interior of the vehicle. For example, the second display device may be a display device of an entertainment system of a second row of seats in a vehicle. For example, the second display device may be a display device of an entertainment system of a front passenger of the vehicle.

[0025] The method 100 can determine 106 an attention of a first occupant of the audio zone of the vehicle with respect to the first video data stream on the first display device and the second video data stream of the second display device. The method 100 can further determine 108 an attention of a second occupant of the audio zone of the vehicle with respect to the first video data stream of the first display device and the second video data stream of the second display device. The attention can be determined by one or more interior cameras of the vehicle. The interior camera(s) can be used to detect how attentively occupants are following one or more video data streams in the vehicle. Specifically, the attention can be determined using a number of fixations of an occupant of the vehicle with respect to a video data stream of a display device.A fixation can be defined as an eye of a vehicle occupant dwelling on a fixation point for approximately 0.3 seconds. A fixation point can be a point within the range of a display device displaying a video stream. For each occupant and / or for each display device with a video stream, attention can be determined using the number of fixations. For example, for a sliding time interval, the number of fixations on a display device relative to a total number of fixations for an occupant can be detected. This value can be detected for each occupant of an audio zone. By comparing this value, the occupant with the highest proportion of fixations on a display device in the sliding interval can be determined.The occupant with the highest proportion of fixations on a display device in the sliding interval can be determined as the occupant with the highest attention on a display device.

[0026] Finally, the method 100 can control 110 the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone of the vehicle depending on the attention of the first occupant in the audio zone of the vehicle and the attention of the second occupant in the audio zone of the vehicle. The method 100 can output the audio data stream of that video data stream in the audio zone of several occupants whose attention is highest with respect to the video data stream of a display device. Other occupants in the audio zone can only visually perceive the video data streams or listen to an audio data stream associated with another video data stream via a coupled mobile audio output device, for example, via headphones coupled to the vehicle.

[0027] In detail, Fig.2 shows an exemplary system 200 for controlling the output of an audio data stream or a video data stream in an audio zone 204, 206 of a vehicle 202. The vehicle may include two audio zones 204, 206. Furthermore, the vehicle may include multiple display devices 208, 210, 212. Multiple occupants 214, 216, 218, 220 may be located in the interior of the vehicle. The occupants may be captured by an interior camera 222. The occupants 214, 216, 218, 220 may view different display devices 208, 210, 212. The interior camera 222 can capture the gaze, gaze direction, and / or fixations of the occupants 214, 216, 218, 220 of the vehicle 202. A video data stream can be provided to each display device 208, 210, 212, wherein the video data stream can be different on each display device 208, 210, 212. For example, the interior camera 222 can capture fixations 226, 228, 230 of the occupant 220 of the audio zone 206 for the display devices 208, 210, 212.For example, the interior camera 222 can capture fixations 232, 234 of the occupant 218 of the audio zone 206 for the display devices 208, 210. For example, the interior camera 222 can capture fixations 224 of the occupant 216 of the audio zone 204 for the display device 212. The system 200 can execute the method 100 in the vehicle 202 to provide a corresponding audio data stream associated with a video data stream of a display device 208, 210, 212 in the respective audio zone 204, 206.

[0028] Advantageously, the method can dynamically control an audio data stream for individual vehicle occupants based on their attention. This can efficiently improve the comfort of audio output from a multitude of video data streams on displays in the interior. List of reference symbols 100 procedures 102 Providing a first video data stream 104 Providing a second video stream 106 Determining an attention 108 Determining an attention 110 Controlling an output of an audio stream 200 systems 202 vehicles 204 Audio Zone 206 Audio Zone 208 display device 210 display device 212 display device 214 inmates 216 inmates 218 inmates 220 inmates 222 Interior camera 224 fixations 226 fixations 228 fixations 230 fixations 232 fixations 234 fixations

Claims

[1] A method (100) for controlling an output of an audio data stream of a video data stream in an audio zone (204, 206) of a vehicle (202), the method (100) comprising: Providing (102) a first video data stream on a first display device (208, 210, 212) in an interior of the vehicle (202); Providing (104) a second video data stream on a second display device (208, 210, 212) in the interior of the vehicle (202); Determining (106) an attention of a first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) with respect to the first video data stream on the first display device (214, 216, 218, 220) and the second video data stream of the second display device (214, 216, 218, 220); Determining (106) an attention of a second occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) with respect to the first video data stream of the first display device (214, 216, 218, 220) and the second video data stream of the second display device (214, 216, 218, 220); and Controlling (110) an output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone (204, 206) of the vehicle (202) depending on the attention of the first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) and the attention of the second occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202), wherein controlling the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone (204, 206) of the vehicle (202) depending on the attention of the first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) and the attention of the second occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) comprises: Outputting the audio data stream associated with the first video data stream or the second video data stream in the audio zone (204, 206) of the vehicle (202) whose occupant (214, 216, 218, 220) has the highest attention of all occupants (214, 216, 218, 220) of the audio zone (204, 206) with respect to the first video data stream or the second video data stream. [2] The method (100) of claim 1, the method further comprising: Determining a content type of the first video stream; Determining a content type of the second video stream; and Controlling an output of the audio data stream, the first video data stream or the second video data stream in the audio zone (204, 206) of the vehicle (202) depending on the attention of the first occupant (214, 216, 218, 220) of the vehicle (202), the attention of the second occupant (214, 216, 218, 220) of the vehicle (202), the content type of the first video data stream and the content type of the second video data stream. [3] Method according to one of the preceding claims, wherein the first display device (208, 210, 212) and / or the second display device (208, 210, 212) is a display, a head-up display, a mobile terminal coupled to the vehicle, and / or a projection display in the interior of the vehicle (202). [4] Method according to one of the preceding claims, wherein the attention of the first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) and / or of the second occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) is determined as a function of a viewing direction of the respective occupant (214, 216, 218, 220) to the first video data stream of the first display device (208, 210, 212) and / or the second video data stream of the second display device (208, 210, 212); wherein the viewing direction of the respective occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) is preferably detected with one or more interior cameras (222) of the vehicle (202). [5] Method (100) according to one of the preceding claims, wherein the attention of the first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) and / or of the second occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) is determined as a function of a number of fixations of the respective occupant (214, 216, 218, 220) on the first video data stream of the first display device (208, 210, 212) and / or the second video data stream of the second display device (208, 210, 212); wherein the number of fixations of the respective occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) is recorded with one or more interior cameras (222) of the vehicle (202). [6] Method (100) according to one of the preceding claims, the method (100) comprising: Determining a number of fixations of the first occupant (214, 216, 218, 220) on the first video data stream of the first display device (208, 210, 212) and / or on the second video data stream of the second display device (208, 210, 212) relative to a total number of fixations of the first occupant (214, 216, 218, 220) within a sliding time interval with a predetermined time duration; Determining a number of fixations of the second occupant (214, 216, 218, 220) on the first video data stream of the first display device (208, 210, 212) and / or on the second video data stream of the second display device (208, 210, 212) relative to a total number of fixations of the second occupant (214, 216, 218, 220) within a sliding time interval with a predetermined time duration; wherein the attention of the first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) with respect to the first video data stream of the first display device (208, 210, 212) and the second video data stream of the second display device (208, 210, 212) corresponds to the determined number of fixations of the first occupant (214, 216, 218, 220) on the first video data stream of the first display device (208, 210, 212) and / or on the second video data stream of the second display device (208, 210, 212) relative to the total number of fixations of the first occupant (214, 216, 218, 220) within the sliding time interval with the predetermined time duration; and / or wherein the attention of the second occupant (214, 216, 218, 220) of the audio zone of the vehicle with respect to the first video data stream of the first display device (208, 210, 212) and the second video data stream of the second display device (208, 210, 212) corresponds to the determined number of fixations of the second occupant (214, 216, 218, 220) on the first video data stream of the first display device (208, 210, 212) and / or on the second video data stream of the second display device (208, 210, 212) relative to the total number of fixations of the second occupant (214, 216, 218, 220) within the sliding time interval with the predetermined time duration. [7] Method according to claim 6, wherein the number of fixations of the respective occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) and the total number of fixations of the respective occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) within the sliding time interval with the predetermined time duration are recorded with one or more interior cameras (222) of the vehicle (202). [8] Method according to one of the preceding claims, wherein controlling the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone (204, 206) of the vehicle (202) depending on the attention of the first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) and the attention of the second occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) comprises: Outputting the audio data stream associated with the first video data stream or the second video data stream on a mobile audio output device of the respective occupant (214, 216, 218, 220) coupled to the vehicle (202) if the attention of the respective occupant (214, 216, 218, 220) does not correspond to the highest attention of all occupants (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202). [9] Method according to one of the preceding claims, wherein controlling the output of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone (204, 206) of the vehicle (202) depending on the attention of the first occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) and the attention of the second occupant (214, 216, 218, 220) of the audio zone (204, 206) of the vehicle (202) comprises: Delaying a change of the audio data stream of the first video data stream and the audio data stream of the second video data stream in the audio zone (204, 206) of the vehicle (202) if the attention of the occupant (214, 216, 218, 220) of the vehicle (202) with the highest attention of all occupants (214, 216, 218, 220) of the audio zone (204, 206) no longer corresponds to the highest attention of all occupants (214, 216, 218, 220) of the vehicle (202) for a predetermined delay time interval. [10] A computer-readable medium for controlling an output of an audio data stream of a video data stream in an audio zone (204, 206) of a vehicle (202), the computer-readable medium comprising instructions which, when executed on a computer and / or a control device, carry out the method (100) according to any one of claims 1 to 9. [11] System (200) for controlling an output of an audio data stream of a video data stream in an audio zone (204, 206) of a vehicle (202), wherein the system (200) is adapted to carry out the method (100) according to any one of claims 1 to 9. [12] A vehicle comprising the system (200) for controlling an output of an audio data stream of a video data stream in an audio zone (204, 206) of the vehicle (202) according to claim 11.

Images