Audio arbitration for multimedia streams

Abstract

The invention provides an audio arbitration for a multimedia stream. In a vehicle, a computer may receive a first content dataset including a first graphics component and a first audio stream and a second content dataset including a second graphics component and a second audio stream; and selecting a first audio output mode of the first audio stream and a second audio output mode of the second audio stream based on a priority determination comprising a priority of the first content data set and / or a priority of the second content data set, wherein the priority determination is based on the vehicle state.

Description

Technical Field

[0001] The infotainment system described in this article provides a way for multiple occupants of a vehicle to simultaneously and independently view, listen to, and interact with different datasets of content. Background Technology

[0002] Computers included in the vehicle can output content datasets to display and audio devices for vehicle occupants to view, listen to, and interact with. Content data can include a wide range of information, such as vehicle data, vehicle service notifications, navigation maps, entertainment data (such as cable television, movies, video games, and the internet via a web browser), and / or cellular data (such as text messages). Summary of the Invention

[0003] The infotainment system simultaneously outputs graphical components from two or more content datasets to a display screen. Each user can have an associated audio device, such as a speaker pointing to the user's location or position in the vehicle (i.e., the position the user is sitting in the vehicle cabin (hereinafter referred to as the user's "cabin position")) or a wireless headset associated with that user. The content data typically has an audio stream. For the corresponding audio device, the infotainment system determines the priority ranking of the associated user and selects the audio stream to be output to the audio device based on that priority ranking. The priority ranking assigns a relative priority to content data 300, 302 for the associated user. The infotainment system selects the audio stream from the content data that ranks highest in that user's priority ranking. The infotainment system determines the user's priority ranking based on the content type of the content data and the user's gaze direction. The content type is the category of the content data, such as navigation, podcasts, streaming video, etc. The gaze direction indicates which graphical component the user is looking at. Therefore, the infotainment system enables the vehicle computer to output audio streams from content datasets to specific audio devices, so that only those users who wish to listen to the content data can listen to it.

[0004] Therefore, this disclosure includes a system comprising a computer for a vehicle, the computer including a processor and a memory storing instructions executable by the processor, the instructions including instructions for performing: receiving a first content dataset and a second content dataset, the first content dataset including a first graphics component and a first audio stream, the second content dataset including a second graphics component and a second audio stream; and selecting a first audio output mode of the first audio stream and a second audio stream based on a priority determination including a priority of the first content dataset and / or a priority of the second content dataset, wherein the priority determination is based on vehicle status.

[0005] The first graphics component can be output from the first display, and the second graphics component can be output from the second display.

[0006] The vehicle status can be normal movement, and the priority can be determined based on the normal movement status.

[0007] The vehicle status can be parked, and the priority can be determined based on the parked status.

[0008] The computer can receive third content data, which includes a third graphics component and a third audio stream and has a priority based on vehicle status.

[0009] The first audio output mode and / or the second audio output mode may include muting or suppressing audio.

[0010] Selecting an audio output mode can include choosing an audio device to output an audio stream.

[0011] The computer can determine whether the audio device is configured for audio isolation and select the output mode based on whether the audio device is configured for audio isolation.

[0012] The priority of the first content dataset and the second content dataset can be based on the identity of the user associated with the first content dataset or the second content dataset.

[0013] The priority of the first content dataset and the second content dataset can be based on the content type of the first content dataset and the second content dataset.

[0014] The priority of the first and second content datasets can be based on the user's gaze direction.

[0015] A method includes: receiving a first content dataset and a second content dataset, the first content dataset including a first graphics component and a first audio stream, the second content dataset including a second graphics component and a second audio stream; selecting a first audio output mode of the first audio stream and a second audio output mode of the second audio stream based on a priority determination including a priority of the first content dataset and / or a priority of the second content dataset, wherein the priority determination is based on vehicle status.

[0016] The first graphics component can be output from the first display, and the second graphics component can be output from the second display.

[0017] The vehicle status can be normal movement, and the priority can be determined based on the normal movement status.

[0018] The vehicle status can be parked, and the priority can be determined based on the parked status.

[0019] The computer can receive third content data, which includes a third graphics component and a third audio stream and has a priority based on vehicle status.

[0020] The first audio output mode and / or the second audio output mode may include muting or suppressing audio.

[0021] Selecting an audio output mode can include choosing an audio device to output an audio stream.

[0022] The computer can determine whether the audio device is configured for audio isolation and select the output mode based on whether the audio device is configured for audio isolation.

[0023] The priority of the first content dataset and the second content dataset can be based on the identity of the user associated with the first content dataset or the second content dataset.

[0024] The priority of the first content dataset and the second content dataset can be based on the content type of the first content dataset and the second content dataset.

[0025] The priority of the first content dataset and the second content dataset can be based on the identity of the user associated with the first content dataset or the second content dataset. Attached Figure Description

[0026] Figure 1 This is a block diagram of an example vehicle system.

[0027] Figure 2 This is a top-down illustrative view of the vehicle, exposing the passenger compartment for explanation.

[0028] Figure 3 This is an illustration of an example dual-view monitor.

[0029] Figure 4 This is a flowchart of an example process for outputting audio data. Detailed Implementation

[0030] Referring to the accompanying drawings, in which like reference numerals indicate like parts throughout several views, vehicle system 100 includes a computer 104 for vehicle 102, the computer including a processor and a memory. The memory stores instructions executable by the processor, including instructions for performing the following operations: outputting a first graphics component to display screen 108 for first content data 300 having a first graphics component and a first audio stream; and outputting a second graphics component to display screen 108 simultaneously with outputting the first graphics component to display screen 108 for second content data 302 having a second graphics component and a second audio stream. Furthermore, for a first user 200 associated with a first audio device 110, computer 104 can determine a first priority ranking of the first content data 300 and the second content data 302 based on the content type of the second content data 300 and the second content data 302. By detecting a first gaze direction of the first user 200, computer 104 can select a first choice for either the first audio stream or the second audio stream based on the first priority ranking and output the first selection to the first audio device 110. Furthermore, for a second user 202 associated with the second audio device 110, the computer 104 can determine a second priority ranking of the first content data 300 and the second content data 302 based on the content types of the first content data 300 and the second content data 302, as well as the second gaze direction of the second user 202. Then, the computer 104 can select a second choice between the first audio stream and the second audio stream based on the second priority ranking, and can output the second choice to the second audio device 110 simultaneously with outputting the first choice to the first audio device 110.

[0031] Exemplary system components

[0032] refer to Figure 1 Vehicle 102 includes vehicle system 100. Vehicle system 100 includes computer 104 with memory including instructions executable by computer 104 to perform processes and operations as described herein. Computer 104 may be communicatively coupled to sensors 106, displays 108, audio devices 110, and other components in vehicle 102 via vehicle communication network 114. Vehicle 102 may be any passenger or commercial vehicle, such as sedan, truck, SUV, crossover, van, minivan, taxi, bus, ICE (internal combustion engine), BEV (battery electric vehicle), hybrid vehicle, PHEV (plug-in hybrid electric vehicle), etc.

[0033] As mentioned above, the vehicle computer 104 (hereinafter referred to as "vehicle computer 104" or "computer 104") includes a processor and memory. The memory includes one or more forms of computer-readable medium and stores instructions executable by the computer 104 to perform various operations, including those disclosed herein. For example, the computer 104 may be a general-purpose computer having a processor and memory as described above, and / or may include electronic control units (ECUs) or controllers for specific functions or sets of functions, and / or dedicated electronic circuitry including ASICs (Application-Specific Integrated Circuits) manufactured for specific operations (e.g., ASICs for processing and / or transmitting sensor data). In another example, the computer 104 may include an FPGA (Field-Programmable Gate Array), which is manufactured as a user-configurable integrated circuit. Typically, hardware description languages ​​such as VHDL (Very High Speed ​​Integrated Circuit Hardware Description Language) are used in electronic design to describe digital and mixed-signal systems such as FPGAs and ASICs. For example, an ASIC is manufactured based on VHDL programming provided before manufacturing, while the logic components within an FPGA may be configured based on VHDL programming (e.g., stored in memory electrically connected to the FPGA circuitry). In some examples, computer 104 may include a combination of processors, ASICs, and / or FPGA circuitry. Computer 104 may be multiple computers coupled together.

[0034] The memory can be of any type (e.g., hard disk drive, solid-state drive, server, or any volatile or non-volatile media). The memory can store collected data transmitted from sensor 106. The memory can be a separate device from computer 104, and computer 104 can retrieve the information stored in the memory via network 114 in vehicle 102 (e.g., via CAN bus, wireless network, etc.). Alternatively or additionally, the memory can be part of computer 104 (e.g., as memory of computer 104).

[0035] Computer 104 may include being programmed to: operate one or more of the vehicle components, such as propulsion (e.g., controlling the speed of vehicle 102 by controlling one or more of an internal combustion engine, an electric motor, a hybrid engine, etc.), steering, interior and / or exterior lights, display 108, audio device 110, etc.; and determine whether and when computer 104 (not a human operator) controls such operation.

[0036] Computer 104 is typically arranged for communication over vehicle communication network 114, which may include buses in vehicle 102, such as Controller Area Network (CAN), and / or other wired and / or wireless mechanisms. Alternatively or additionally, where computer 104 actually comprises multiple devices, vehicle communication network 114 may be used for communication between the devices represented herein as computer 104. Further, as mentioned below, various controllers and / or sensors 106 may provide data to computer 104 via vehicle communication network 114.

[0037] Via vehicle network 114, computer 104 can transmit messages to and / or receive messages (e.g., CAN messages) from various devices and / or components in vehicle 102 (e.g., sensors 106, ECUs, etc.). Alternatively or additionally, where computer 104 actually comprises multiple devices, vehicle communication network 114 can be used for communication between devices represented herein as computer 104. Further, as mentioned below, various controllers and / or sensors 106 can provide data to computer 104 via vehicle communication network 114.

[0038] Sensor 106 can provide data about the occupants of vehicle 100. Sensor 106 may be a camera and may detect electromagnetic radiation within a certain wavelength range. For example, the sensor may detect visible light, infrared radiation, ultraviolet light, or a range of wavelengths including visible light, infrared light, and / or ultraviolet light. For example, the camera may be a charge-coupled device (CCD), complementary metal-oxide-semiconductor (CMOS), or any other suitable type.

[0039] Vehicle 102 may include multiple displays, such as a first display 108-1 (referred to herein as the “panoramic display”) and a second display 108-2 (referred to herein as the “console display”) (displays 108-1 and 108-2 are collectively referred to herein as “display 108”). Displays 108-1 and 108-2 display visual data, such as two-dimensional visual data, to the occupants of vehicle 102. Displays 108-1 and 108-2 may be of any suitable type for displaying content clearly visible to the respective occupants, such as light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), plasma displays, digital light processing technology (DLPT), etc. Display 108 may display the visual data via a screen in monochrome or color, and the visual data may be updated at a certain frame rate, such as, for example, 60 frames per second. The displayed visual data may be a static image, in which most of the two-dimensional area does not change with the frame, or it may be a dynamic image, in which most of the two-dimensional area changes with the frame. The visual data to be displayed on display 108 may be generated by a display controller. The display controller is a computing device such as an ECU that can receive data to be displayed on the display 108 in a visual format from a computer 104, another vehicle ECU, or an external computing device 118 via a server 116.

[0040] Display 108 can be configured to output content datasets 300 and 302. Display 108 can be further configured to output the same content datasets 300 and 302 or different content datasets. Additionally, one or more displays 108-1 (e.g., a panoramic display) can be configured to output multiple content datasets 300 and 302, provided that the specifications of display 108-1 (e.g., screen size) are capable of supporting multiple programs (e.g., the screen is large enough to allow viewing multiple programs).

[0041] Some displays 108-2 may allow for user interaction. For example, display 108-2 may be a conventional touchscreen display, allowing the user to provide input to computer 104 via display 108-2, such as selecting content datasets 300, 302 to be output by display 108-2. Additionally, interaction with data not included in the multiple content datasets 300, 302 (e.g., vehicle maintenance data) may be possible via touchscreen technology. The touchscreen may be any suitable type for receiving input from the user (e.g., resistive, capacitive, infrared, etc.).

[0042] Audio device 110 can be any suitable device configured to output sound to the occupants of vehicle 102 (e.g., first user 200 and second user 202). For example, audio device 110 can be a speaker, a personal device (e.g., headphones), etc. A speaker is an electroacoustic transducer that converts electrical signals into sound. A speaker can be any suitable type for producing sound (e.g., dynamic) that is audible to a corresponding user. A portable device can be any suitable device for emitting sound to a single user, such as in-ear or over-ear headphones, a portable speaker, etc. A personal device can be connected via a wired connection (such as an audio jack) or a wireless connection (such as Bluetooth). TM It is connected to the vehicle network 114. The audio device 110 may be part of the infotainment system and is thus configured to output audio associated with the content datasets 300, 302.

[0043] Computer 104 can select an audio output mode for the audio stream. The audio output mode is a provision of content data 300, 302, which includes the volume (i.e., decibel level) of the audio stream and the output location of the audio stream (i.e., the specifications from one or more of the plurality of available audio devices 110 used to output the audio stream). Computer 104 can change all aspects of the audio output mode, or it can change one aspect of the audio output mode while keeping another or other aspects unchanged. For example, computer 104 can adjust the audio output mode of content datasets 300, 302 by reducing the volume of the audio stream and / or changing which audio devices 110 are outputting the audio stream. Computer 104 can adjust the audio output mode based on a priority ranking as described below.

[0044] Vehicle users 200 and 202 can be associated with audio device 110; that is, a specific audio device 110 can be assigned to a particular user 200 or 202. Computer 104 can store user associations, enabling the assignment of audio device 110 to the corresponding user 200 or 202. Audio device 110 can be associated with user 200 or 202 based on its placement relative to the vehicle compartment location (e.g., the front and rear areas of the compartment). Computer 104 can store associations between vehicle audio device 110 and compartment location. The stored associations can be specified based on any desired criteria during the development of computer 104. Continuing the example, the stored associations can associate audio device 110 with the nearest passenger compartment location. For example, if the first user 200 is in the operator's compartment position 208 of vehicle 102, any audio device 110 (e.g., a speaker) supported by the surface of vehicle 102 that is closer to the operator's compartment position 208 than any other compartment position 208, or that is oriented to emit sound toward the operator's compartment position 208, may be associated with the first user 200.

[0045] The association of audio device 110 can also be based on the most recent user 200, 202, etc., who wants to select content data 300, 302 on audio device 110. After receiving the selection of content data 300, 302, computer 104 can use image data collected by sensor 106 to determine which user 200, 202 selected content data 300, 302, and associate user 200, 202 with the device 110 on which user 200, 202 selected content data 300, 302. As another example, if a second user 202 has access to a personal audio device 110 (e.g., headphones) that communicates with computer 104 (e.g., via a wireless connection), then that audio device 110 can be associated with the second user 202. As a further example, computer 104 can associate users 200, 202 with audio device 110 based on user input. Users 200 and 202 can specify to computer 104 which audio devices 110 will be associated with each user 200 and 202 (e.g., via input to one of the displays 108).

[0046] refer to Figure 2The image shows the passenger compartment 210 of vehicle 102. Passenger compartment 210 can accommodate users 200, 202 of vehicle 102. In this example, user 200 is located in the operator's compartment position 208, and user 202 is located in the right rear compartment position 208. Passenger compartment 210 includes one or more compartment positions 208, such as one or more compartment positions 208 located in the front row of passenger compartment 210 and one or more compartment positions 208 located in the second row behind the front row. Passenger compartment 210 may also include a third row (not shown) of compartment positions 208 located at the rear of passenger compartment 210. The position and orientation of the compartment positions 208 and their components can be selected by users 200, 202.

[0047] Still referencing Figure 3 This diagram shows a forward view of the vehicle cabin 210, where the vehicle 102 can support multiple displays, such as displays 108-1 and 108-2. Display 108 can be supported by various surfaces of the vehicle, such as the dashboard 204, console 206, etc. In this example, display 108-1 is a panoramic display including a screen that can be supported by the dashboard 204. Panoramic display 108-1 can be located on the top surface (e.g., the uppermost surface) of the dashboard 204. Panoramic display 108-1 can be located above the steering wheel and / or above the instrument cluster, and / or can display graphical data that makes up the instrument cluster. Therefore, panoramic display 108-1 can be viewed by users 200 and 202, thereby reducing the need for users 200 and 202 to shift their gaze away from the direction of travel of the vehicle 102.

[0048] The first display 108-1 may be a panoramic display (e.g., it may have an aspect ratio greater than that of a typical widescreen display, i.e., greater than 16:9). As shown, the panoramic display 108-1 extends from a first end of the dashboard 204 to a second end of the dashboard. For example, the panoramic display 108-1 may begin at a first side of the supporting windshield of the vehicle 102 and end at a second side of the supporting windshield of the vehicle 102. As another example, the panoramic display 108-1 may begin and end at a point on the dashboard 204 that is typically between the two sides of the vehicle 102. For example, the panoramic display 108-1 may extend for at least half the length from one side to the other.

[0049] The second display 108-2 (console display) can be supported by the central console 206. The console display 108-2 can be part of an infotainment system, allowing content datasets 300, 302 that can be played by the panoramic display 108-1 to also be played by the console display 108-2. The console display 108-2 can be visible from a range of angles including occupants (i.e., passengers and operators) in multiple locations within the vehicle 102. For example, the console display 108-2 can be located below the panoramic display 108-1, and the console display 108-2 can be laterally centered relative to the vehicle 102. The aspect ratio of the console display 108-2 may not be as wide as that of the panoramic display 108-1. For example, the aspect ratio can be standard (i.e., 4:3) or widescreen (i.e., 16:9).

[0050] The audio device 110 can be installed inside the vehicle 102 (e.g., in the passenger compartment 210). For example, the audio device 110 can be installed in the dashboard 204, console 206, etc. The audio device 110 can be installed inside the vehicle 102, such as... Figure 2 As shown. The audio device 110 can be a personal audio device (e.g., wireless headphones) or a dedicated in-vehicle audio device (e.g., a speaker), as shown. The audio device 110 can be supported by the vehicle 102 in locations around the interior of the vehicle such that users 200, 202 at different locations within the passenger compartment 210 can generally hear the audio stream at a similar volume. For example, the audio device 110 (e.g., those audio devices installed in the vehicle 102) can be positioned on the dashboard 204, console 206, sides of the vehicle 102, etc.

[0051] like Figure 3 As shown, displays 108-1 and 108-2 can display corresponding content datasets 300 and 302. For the purposes of this disclosure, "content dataset" means a data stream that includes a graphics component (i.e., one or more visually detectable images) and an associated audio stream. In various embodiments, content data 300 and 302 can be pre-recorded and streamed from memory, and / or can be generated or streamed in real time from a remote source. Vehicle system 100 can output multiple content datasets 300 and 302 simultaneously, for example, as Figure 3The first content data 300 and the second content data 302 are shown. The first content data 300 may include a first graphics component and a first audio stream, and the second content data 302 may include a second graphics component and a second audio stream. The graphics component and the audio stream are described in further detail below. In the example shown, the panoramic display 108-1 is outputting graphics components from the first content data 300 and the second content data 302. The console display 108-2 is outputting graphics components from the first content data 300. In this non-limiting example, the first content data 300 is a navigation program including a map. The second content data 302 is a weather forecast.

[0052] Content datasets 300 and 302 can be selected by users 200 and 202. That is, users 200 and 202 can interact with vehicle computer 104 (e.g., via a touchscreen interface of displays 108-1 and 108-2 or another input device) to request one or both of displays 108-1 and 108-2 to output one or more content datasets 300 and 302. Alternatively or additionally, content datasets 300 and 302 can be output based on instructions stored by vehicle computer 104. For example, computer 104 can cause navigation content data to be output even when users 200 and 202 have not yet requested content datasets 300 and 302. As a further example, computer 104 can cause notification content data 300 and 302 to be output based on the vehicle being in a state where it is necessary to prioritize notification content, as described in further detail below.

[0053] Content datasets 300 and 302 include corresponding graphical components. For example, first content data 300 may include a first graphical component, and second content data 302 may include a second graphical component. For the purposes of this disclosure, a graphical component is defined as visual data (e.g., depending on the video or image display format, one or more frames in an image and / or image stream are provided as a pixel array) included in content data 300 and 302. The graphical component can be a static or dynamic display of information, video streams, etc. For example, a graphical component of the first content data 300 as a navigation program (such as...) Figure 3 (As shown) will be a map, which may include nodes indicating the relative position of vehicle 102. For example, as... Figure 3 As shown, the graphical component of the second content data 302 in the weather program will be a display of various weather information (such as temperature, cloud cover, etc.). As another example, if the first content data 300 is a movie, etc., and the second content data 302 is a video call, then the first graphical component will be the video stream of the movie, and the second graphical component will be the video stream of the video call.

[0054] Content datasets 300 and 302 also include corresponding audio streams. First content data 300 may include a first audio stream, and second content data 302 may include a second audio stream. For the purposes of this disclosure, an audio stream is defined as audio data included in content datasets 300 and 302. The audio streams of content data 300 and 302 can be output from audio device 110 in conjunction with output graphics components from displays 108-1 and 108-2 for the corresponding content datasets 300 and 302. The audio streams of the content data can be synchronized to the graphics components of the content data 300 and 302 (e.g., audio accompanying streaming video, audio navigation instructions triggered when the current location reaches a predefined point along a route displayed in the graphics component, etc.). For example, common formats for providing streaming video with associated audio typically include mechanisms for synchronizing audio and video. Continuing the previous example, if first content data 300 is a movie and second content data 302 is a video call, then the first audio stream would be the audio associated with the movie, and the second audio stream would be the audio of the video call.

[0055] Displays 108-1 and 108-2 can display visual information that is not part of the content data 300 and 302. For example, display 108-1 can display vehicle information as part of the instrument cluster (e.g., vehicle speed, engine speed, engine temperature, etc.) without accompanying audio data.

[0056] Audio devices 110 can be supported around vehicle 102 or installed in the passenger compartment 210. Furthermore, computer 104 can selectively output audio streams to some, but not all, audio devices 110, while simultaneously outputting graphics components, including audio streams, contained in content data 300, 302, to one or more displays 108. That is, computer 104 can simultaneously output different audio streams to different audio devices 110. Therefore, when playing the first and second audio streams simultaneously, users 200, 202 can hear the first audio stream without hearing or being interrupted by the second audio stream. Outputting multiple audio streams from different audio devices 110 in a manner that minimizes or reduces interference from the audio streams to users 200, 202 to whom the audio streams are not directed is referred to herein as "audio isolation."

[0057] Device 110 can be configured for audio isolation. For example, audio isolation can be achieved when an audio stream is output to a personal audio device 110 (such as headphones), where users 200, 202 receive sound from the audio stream and prevent or reduce other sounds from reaching the user's ears. In some examples, audio isolation can be achieved by changing the audio output mode of the audio stream to output the audio stream via a designated audio device 110 (e.g., a vehicle speaker) arranged to cancel or interfere with sounds from each other. For example, if the audio device 110 is a speaker, computer 104 can output the audio stream at a volume that is generally clear to the respective user but inaudible or at least unclear to other users 200, 202, which can be facilitated by the speaker being directed towards the vehicle compartment position 208 where users 200, 202 are seated. Alternatively, computer 104 can execute a suitable active noise cancellation algorithm to produce canceled audio data that cancels (i.e., reduces) the audio data stream from another user 200, 202. Active noise cancellation algorithms can invert or phase-shift the sound waves contained in the audio stream, thereby producing canceled audio data. Outputting canceled audio data can cause destructive interference to the sound that generated the input audio data. For example, canceled audio data used to cancel a first audio stream can be added to a second audio stream to isolate a second user 202 (i.e., minimize or reduce interference to the second user). Such techniques can be useful when users 200, 202 do not have access to personal audio devices 110 (e.g., Bluetooth headsets), which typically perform audio isolation by blocking audio streams that users 200, 202 do not want to hear. In examples where user 202 does have access to personal audio devices 110, computer 104 can associate those personal audio devices 110 with a specific user 202 (e.g., when the user connects a headset to computer 104 via Bluetooth and designates the headset as belonging to user 202).

[0058] Audio device 110 may be positioned such that effective audio isolation is not possible. Such audio device 110 is referred to herein as "non-isolated". Computer 104 may determine whether audio isolation is possible based on the positioning of audio device 110. For example, computer 104 may store in memory an identifier indicating whether an audio device 110 installed in passenger compartment 210 is isolated or non-isolated. When determining that the first audio device 110 and the second audio device 110 are non-isolated audio devices 110 (e.g., in the case where audio device 110 is a vehicle speaker positioned in compartment 210 to output sound), the computer may output the same audio stream to the first audio device and the second audio device, rather than attempting to output different audio streams (e.g., the first audio stream and the second audio stream) to different non-isolated audio devices 110.

[0059] Computer 104 can cause more than one content data 300, 302 to be output simultaneously. That is, computer 104 can cause a first graphics component and a first audio stream of the first content data 300 to be output via displays 108-1, 108-2 and audio device 110, while also outputting a second graphics component and a second audio stream via displays 108-1, 108-2 and audio device 110. Typically, the first and second audio streams of the corresponding content data sets 300, 302 are played via different audio devices 110, or one of the audio streams is cut off (e.g., muted) and not played on audio device 110, so that the other audio stream can be played on audio device 110.

[0060] Alternatively, the graphical components of content datasets 300 and 302 can be played simultaneously on different displays 108-1 and 108-2, or simultaneously on the same display 108 (e.g., panoramic display 108-1). For example, panoramic display 108-1 can cause graphical components of a navigation program to be output on the left side of the display while graphical components of a weather forecast program are output on the right side of the display. Furthermore, when displays 108-1 and 108-2 are outputting graphical components from multiple content datasets 300 and 302, displays 108-1 and 108-2 can rearrange additional graphical components, stop outputting additional graphical components, or cause additional graphical components to be output (for reasons described below) while also outputting other graphical components.

[0061] Computer 104 can cause the output of graphical components and audio streams of content datasets 300 and 302 based on a priority ranking of multiple content datasets 300 and 302. As used herein, a "priority ranking" is a list of content datasets 300 and 302 arranged according to numerical values ​​(i.e., "priority scores") indicating the relative weight or importance of the content datasets 300 and 302. Content datasets 300 and 302 can be assigned a baseline (i.e., default) priority score or associated with said priority score. The baseline priority score can be stored in a lookup table or the like, which is stored in the memory of computer 104 and specified by the device manufacturer (such as a vehicle manufacturer). When content datasets 300 and 302 are to be output, computer 104 can retrieve the baseline priority lookup table and find the baseline priority of the corresponding content dataset 300 or 302. Computer 104 can then assign the corresponding baseline priorities to content datasets 300, 302, and can then adjust the corresponding priorities based on dynamic (i.e., changing over time and possibly in real time) factors such as vehicle status (e.g., moving or parked), content type, and content subtype, as further described below.

[0062] Computer 104 can cause the highest priority content datasets 300 and 302 to be output, and the lower priority content datasets 300 and 302 to be paused. Alternatively, computer 104 can adjust the audio output mode of one (or more) lower priority content datasets 300 and 302 by decreasing (or “suppressing”) the volume of the associated audio stream instead of stopping the output. The adjustment of the audio output mode of the lower priority content data 300 and 302 can be based on the vehicle status (described below).

[0063] The priority ranking of content datasets 300 and 302 can be dynamically determined by computer 104 (e.g., a storage-based algorithm). For example, the priority ranking could be a set of scores, each ranging from 1 to 100, where lower scores indicate lower importance and higher scores indicate higher importance. Each content dataset 300 and 302 can have a corresponding score, and the priority ranking is an ordering of content datasets 300 and 302 based on the scores (e.g., in descending order from the highest to the lowest score). Alternatively, the priority ranking could be an ordering of content datasets 300 and 302 without individual scores (e.g., based on a set of rules).

[0064] In cases where audio isolation is unavailable (e.g., when the only audio device communicating with computer 104 is a vehicle speaker that cannot provide noise cancellation or interference), computer 104 may determine a single priority ranking that includes all content datasets 300, 302 applicable to all non-isolated audio devices 110. Alternatively, when audio isolation is available (e.g., when a first user 200 is listening to content datasets 300, 302 via a dedicated non-isolated vehicle speaker and a second user 202 is simultaneously listening to multiple content datasets 300, 302 via headphones), computer 104 may allow users 200, 202 associated with isolated device 110 to override the priority ranking for isolated device 110, such that the only content data 300, 302 output by isolated audio device 110 is the content data 300, 302 selected by the associated user 200, 202. In other words, computer 104 can determine the priority ranking of content datasets 300, 302 to be used by all non-isolated audio devices 110, and can disregard isolated devices 110 when determining the priority ranking and output audio stream. Alternatively, computer 104 can determine the individual priority ranking of each of the isolated devices and the priority ranking of all non-isolated devices.

[0065] Computer 104 can adjust the audio output mode of content datasets 300 and 302 based on priority scores or rankings. For example, content datasets 300 and 302 with a priority ranking lower than other content datasets 300 and 302 can be muted or have their volume reduced. When the priority ranking is based on scores, computer 104 can reduce the volume of content datasets 300 and 302 by the maximum possible percentage of each priority score difference between them and the highest-ranked data dataset 300 and 302 (e.g., if content datasets 300 and 302 are 10 priority scores lower than the highest-priority content dataset 300 and 302, computer 104 can reduce the volume of said content dataset by 10%). Alternatively, computer 104 can mute all data datasets 300 and 302 that are not at the top of the priority ranking.

[0066] Computer 104 can determine a priority ranking (i.e., a score) based on adding to or subtracting from a baseline priority according to dynamic factors. The factors and adjustments given by computer 104 can be included in a stored lookup table (priority ranking lookup table). The priority ranking lookup table can specify factors influencing priority, such as the content type of content datasets 300 and 302, the presence of dialogue between users 200 and 202, the gaze direction of users 200 and 202, the identity of users 200 and 202 who requested each content dataset (in the event of audio isolation being unavailable), and / or preferences input by users 200 and 202, as each will be described sequentially below. Computer 104 can calculate a score for each content dataset 300 and 302 based on adjustments specified by the priority ranking lookup table corresponding to each factor. The priority ranking lookup table can be specified by equipment manufacturers, such as vehicle manufacturers, and can specify adjustments to the priority ranking in response to various dynamic factors (e.g., vehicle speed, whether the vehicle is moving or parked, etc.).

[0067] Computer 104 can assign corresponding numerical values ​​to factors (e.g., numerical values ​​for each content dataset in different content datasets 300, 302). These numerical values ​​may include values ​​for the presence of dialogue and values ​​for the absence of dialogue, values ​​for the graphical components of content data 300, 302 displayed on screen 108-1 when user 200 looks at them, and values ​​for when user 200 looks away from content data 300, 302 displayed on screen 108-1, etc. Computer 104 can then combine the numerical values ​​of each content data into a single score (e.g., a weighted sum of the numerical values). Computer 104 can then rank the content data 300, 302 according to their scores. An example expression that computer 104 can use to calculate priority scores is represented by Equation 1:

[0068] Equation 1:

[0069] Priority score = Baseline priority score + F1 + F2 + F3 + ... + F N

[0070] F refers to the adjustment to be made based on the corresponding factors specified by the priority ranking lookup table (e.g., add 10 for gaze direction, subtract 5 for ad subtype, etc.).

[0071] Table 1 shows an example priority ranking lookup table that computer 104 can use (among other lookup tables) to adjust priority rankings, where the priority ranking is based on scores. Table 1 is specifically for the content subtypes (described in further detail below):

[0072]

[0073] Table 1

[0074] As another example of how computer 104 can determine priority ranking, computer 104 can execute a pre-stored set of rules to rank multiple content datasets 300, 302 (e.g., content datasets 300, 302 that user 200 is viewing are ranked higher than other content datasets 300, 302 of the same content type; content datasets 300, 302 of certain content types are ranked higher than other types, regardless of the direction of user 200's gaze; etc.).

[0075] Table 2 shows an example priority ranking lookup table that computer 104 can use (among other lookup tables) to adjust priority rankings, where the priority ranking is rule-based. Table 2 is specifically for the content subtypes (described in further detail below):

[0076]

[0077] Table 2

[0078] User selection can affect the priority ranking of content datasets 300 and 302, where audio isolation is unavailable (e.g., where only one priority ranking exists for all audio devices 110). For example, computer 104 can increase the priority score or placement of content datasets 300 and 302 requested by user 200 located in the left front position 208 of carriage 210. In the score-based priority ranking example, if first user 200 requests first content data 300 and second user 202 requests second content data 302, the priority ranking lookup table (mentioned above) can specify that the priority score of first content data 300 is increased by 10, and the priority score of second content data 302 remains unchanged in the priority ranking of all non-isolated audio devices 110. Alternatively or additionally, in the rule-based priority ranking example, computer 104 can be programmed to specify that any content data 300 or 302 selected by user 200 is always assigned a higher priority ranking in the priority ranking of non-isolated audio devices 110 than content datasets 300 or 302 selected by any other user 202.

[0079] Priority ranking can be partly based on the content type of content data 300 and 302. As used herein, "content type" refers to a qualitative classification based on the subject matter of the specific material presented by content data 300 and 302. For example, a content type could be a navigation program, podcast, streaming video, vehicle notification, etc. A priority ranking lookup table can specify (regardless of priority score) that some content types are always given the highest priority relative to other content types. For example, a priority ranking lookup table can specify that content datasets 300 and 302, as notifications (e.g., traffic alerts), can always be given the highest priority in priority ranking.

[0080] The content types of content data 300 and 302 may have further content subtypes. Content subtypes are further classifications of the topics presented by the types of content data 300 and 302. Not all content types include content subtypes. As an example of content subtypes, if the content type of content data 300 and 302 is a podcast, then the content subtype could be "advertisement," and another content subtype could be "podcast content." The content subtype of the same content data 300 and 302 that is playing or streaming can change over time (e.g., the advertisement portion of a streaming video and the non-advertisement portion of a streaming video could be corresponding subtypes of the podcast content type played at different times in the podcast). Computer 104 can determine the content subtype of content data 300 and 302 based on included and / or associated tags or other metadata indicating the content subtype. Alternatively or additionally, computer 104 can determine the content subtype of content data 300 and 302 based on a trained neural network (described below).

[0081] Computer 104 determines priority ranking at least in part based on the content subtype of content data 300, 302. Computer 104 can change the determination in real time at least in part based on the content subtype (e.g., in response to a change in the content subtype of content data 300, 302). When priority ranking is based on scores, computer 104 can refer to a priority ranking lookup table specifying how to adjust scores based on content subtype (e.g., it is necessary to lower the priority score by 10 for an advertisement content subtype). For example, if first content data 300 is a commercial advertisement, computer 104 can determine the content subtype of first data 300 as an advertisement and lower the score accordingly. As another example, computer 104 can give a higher priority score or place it in the priority ranking in response to the content subtype of content data 300, 302 being a telephone call. For example, user 200 receives a telephone call as second content data 302. Computer 104 assigns a higher priority score to second content data 302 than to first content data 300, causing first content data 300 to be muted or have its volume reduced. In the case of rule-based priority ranking, computer 104 can refer to a priority ranking lookup table that specifies the relative ranking of each content data set 300, 302 based on the content subtype of the data. For example, the table can specify that content datasets 300, 302 of the advertising subtype are always placed below content datasets 300, 302 of the non-advertising content subtype.

[0082] A neural network can be trained to determine content subtypes to identify the content subtypes of content datasets 300 and 302 based on the training process. When training the deep neural network, a training dataset comprising example content datasets 300 and 302 belonging to various content subtypes can be used. The training dataset may include thousands of example content datasets 300 and 302, each belonging to a specific content subtype. The deep neural network can be executed multiple times on the content datasets 300 and 302, wherein each execution compares the output predictions to ground truth to determine a loss function. The loss function is backpropagated from the output layer to the input layer via the deep neural network to adjust the weights, which manage the processing of each layer to minimize the loss function. When the loss function reaches a user-determined minimum for the training dataset, the deep neural network training is considered complete, and the weights indicated by the minimum loss function can then be stored along with the trained deep neural network.

[0083] Computer 104 can adjust the scores of content data 300, 302 based on the dialogue between users 200, 202. As used herein, "dialogue" refers to a conversation between users 200, 202, not a conversation between users 200, 202 and a third party (e.g., via a telephone call that could be considered a content subtype). Computer 104 can detect the dialogue based on known voice recognition systems. For example, computer 104 can utilize a trained neural network (using a method similar to that described above) to recognize speech. When audio sensor 106 detects audio and computer 104 determines, based on the neural network, that the audio is the speech of one or more users 200, 202, computer 104 can act accordingly. Since the dialogue does not involve any virtual data, it is not "content data." Therefore, typically, when a dialogue is detected, computer 104 can adjust the audio output mode of the audio stream to pause content datasets 300, 302, mute the content datasets, or reduce the volume of the content datasets. As an example where priority ranking is score-based, computer 104 could lower the score of all content datasets 300 and 302 based on determining that users 200 and 202 are talking. As another example, computer 104 determines that users 200 and 202 are talking. Then, computer 104 lowers the score of all content datasets 300 and 302, causing the audio streams of content datasets 300 and 302 to reduce their volume or be muted. As an example, in the case where priority ranking is rule-based, computer 104 could directly lower the volume of content datasets 300 and 302 based on detecting a conversation.

[0084] Computer 104 detects the gaze direction of users 200 and 202. Computer 104 can first use any suitable eye detection algorithm to detect the eyes, such as shape-based techniques using elliptical or complex eye models; feature-based techniques, such as detecting local features, detecting filter responses, or detecting the pupil and iris; appearance-based techniques; hybrid techniques of the above; and so on. Computer 104 can then use any suitable gaze tracking algorithm (e.g., model-based techniques, interpolation-based techniques, appearance-based techniques, visible light-based techniques, etc.) to detect the gaze direction.

[0085] Computer 104 can adjust priority ranking based on the gaze direction of users 200, 202, for example, relative to the graphical components of content datasets 300, 302 (e.g., display screens 108-1, 108-2 displaying the graphical components). Computer 104 can determine and / or adjust the priority ranking of audio devices 110 associated with users 200, 202 based on the gaze direction. In an example where priority ranking is score-based, computer 104 can increase or decrease the score of content data 300, 302 based on the gaze direction. If computer 104 determines that users 200, 202 are viewing the graphical components of content data 300, 302 (i.e., the gaze direction of users 200, 202 is aimed at a portion of display screens 108-1, 108-2 including the content data), computer 104 can increase the score of the content data. Alternatively or additionally, computer 104 can decrease the score of content datasets 300, 302 that users 200, 202 are not viewing. Computer 104 can consult a priority ranking lookup table that specifies how to adjust priority scores based on gaze direction. The lookup table can specify that if the user's eyes are directly viewing the graphics component and the user's head is angled towards the graphics component, computer 104 will increase the priority score of the data by 20. Alternatively, the lookup table can specify that if the user's eyes and head are away from the graphics component, computer 104 will not adjust the priority score. In a rule-based priority ranking example, computer 104 can assign a higher priority to content data 300, 302 that the user's gaze direction is pointing to than to other content datasets 300, 302.

[0086] Computer 104 can determine the priority ranking of the content datasets 300 and 302 for the respective isolated audio devices, and computer 104 can determine the priority ranking based on the gaze direction as described above. For example, computer 104 can determine a first priority ranking for non-isolated devices and a second priority ranking for isolated devices. User 200 associated with the isolated device can shift their gaze direction from the graphics component of the first content dataset 300 to the graphics component of the second content dataset 302. Computer 104 can then retrieve a priority ranking lookup table and adjust the priority scores of the first content dataset 300 and the second content dataset 302 accordingly, such that the audio stream of the second content dataset 302 is output to the isolated device.

[0087] Computer 104 can determine and / or adjust the priority ranking of content datasets 300 and 302 based on requests from users 200 and 202. That is, users 200 and 202 can (possibly based on personal preferences or other reasons) request (e.g., via a touchscreen) to raise or lower the priority ranking of one of the content datasets 300 and 302. In a score-based priority ranking example, users 200 and 202 can lower the priority score of the first content dataset 300 so that if the first content dataset 300 is a talk show hosted by a specific individual that users 200 and 202 dislike, they can alternatively view / listen to the second content dataset 302. In a rule-based priority ranking example, users 200 and 202 can simply specify to computer 104 which content dataset 300 and 302 will be given the highest priority.

[0088] As described above, computer 104 can determine and / or adjust the priority ranking of content datasets 300 and 302 based on vehicle status. "Vehicle status" as used herein refers to the state of vehicle 102 as described by physical measurements of vehicle 102 and / or the environment surrounding vehicle 102. For example, vehicle status may include one or more of the following: ambient temperature surrounding vehicle 102, passenger compartment temperature of vehicle 102, speed of vehicle 102, and the movement state of vehicle 102 (e.g., moving or not moving and / or parked). Computer 104 can determine vehicle status based on data collected by sensor 106.

[0089] A list of vehicle states may be stored (e.g., by the vehicle manufacturer) in the memory of computer 104. Computer 104 can then determine one or more vehicle states in real time or near real time based on physical measurements of vehicle 102 and / or the environment in and / or around vehicle 102. For example, when vehicle 102 is operating without abnormal environmental conditions (e.g., operating on a road in full sunlight without precipitation), computer 104 may determine that vehicle 102 is in a “normal movement” state. As another example, vehicle 102 may be in a “parked” state when the vehicle’s transmission is in park. As a further example, when the user has selected a direction as one of the content datasets 300, 302, the vehicle may be in a “navigation” state (e.g., a state where audible navigation instructions may be accompanied by the display of a map and / or navigation instructions), or when computer 104 determines that vehicle 102 is not at the center of the edge of the lane and / or not within a specified distance of the edge of the lane, the vehicle may be in an “abnormal movement” state, such as a “potential lane violation” state. The vehicle status can be stored together with the corresponding weights of the priority scores to be assigned to content data 300 and 302 when computer 104 determines the status.

[0090] As an example of vehicle states, vehicle 102 may be traveling on the road in a "normal movement" state, but may then move laterally within a specified lateral distance of the lane lines. Computer 104 may then determine that vehicle 102 is in a "potential lane violation" state. The "potential lane violation" state indicates that the priority score of the "notification" set of content data 300, 302 (e.g., a content dataset notifying user 200 of lateral movement) should be weighted to give that content dataset 300, 302 the highest priority for audio playback compared to other content datasets 300, 302 currently streaming in vehicle 102. Alternatively, the priority of other content datasets 300, 302 may be reduced. Vehicle 102 may remain in the "potential lane violation" state until vehicle 102 is no longer within the specified distance of the lane lines and remains so for a predetermined time (e.g., three or five seconds), and / or until the user removes the "notification" set of content data 300, 302.

[0091] As a further example of vehicle status, vehicle 102 can be driving on the road in a "normal movement" state. Vehicle 102 is then brought into a parking lot and parked. Computer 104 can then determine that the current vehicle status is a "parked" state. In the parked state, the graphical and / or audio components of content datasets 300, 302 can now be output by the first display 108-1 (wherein the normal state, such data 300, 302 components could be prevented from being output by the first display 108-1, for example, because it includes entertainment content unsuitable for normal movement or other movement states).

[0092] Computer 104 can dynamically adjust priority scores based on dynamic factors such as the vehicle state described above. Therefore, computer 104 can proactively switch between output and pause of content datasets 300 and 302 based on dynamically determined priority scores. For example, the first content data 300 played via the first display 108-1 may be the only data played at the first time. Then, at the second time, users 200 and 202 may request playback of the second content dataset 302 via the second display 108-2. Furthermore, computer 104 can determine that the priority score of the second content data 302 is higher than the priority score of the first content data 300. Therefore, computer 104 can pause or mute the playback or streaming of the first content data 300 after the second time while playing the second content data 302. Then, at the third time, the score of the second content data 302 can be dynamically changed, causing the second content data 302 to drop below the priority ranking of the first content data 300. Then, computer 104 can pause or mute the second content data 302 and play the first content data 300 at or after the third time.

[0093] Computer 104 can select an audio stream from one of content datasets 300 and 302 based on their priority ranking, wherein the graphics components from the first content dataset 300 and the second content dataset 302 are played simultaneously by one or both of two displays 108-1 and 108-2. In this case, computer 104 can select the audio stream from the content dataset 300 or 302 that has the highest priority ranking.

[0094] Computer 104 can cause the selected audio stream of users 200, 202 to be output to the audio device 110 associated with those users 200, 202. Priority scores for the audio streams can be assigned and / or adjusted based on the user's identity associated with display 108. For example, a user identified as an adult can be given a higher priority score than a user identified as a younger user (or vice versa).

[0095] If the corresponding priority score drops below a score threshold, computer 104 can mute the audio streams of content datasets 300 and 302. The score threshold can be a value specified by the vehicle manufacturer and stored in the memory of computer 104 and / or can be selected by users 200 and 202.

[0096] Instead of muting the audio streams of content data sets 300 and 302, computer 104 can reduce the volume of the audio streams of content data sets 300 and 302 based on their priority scores (but not to zero decibels). That is, computer 104 can increase or decrease the volume of the audio streams of content data sets 300 and 302, where the priority score increases or decreases. For example, the volume of content data sets 300 and 302 could be a percentage of the maximum volume corresponding to a priority score within a priority ranking. A priority score of 50 might cause computer 104 to adjust the audio stream volume to 50% of its maximum volume.

[0097] Computer 104 can pause or mute content datasets 300 and 302 (or at least one of the graphics components and audio streams of content datasets 300 and 302) based on a change threshold of one of them. The change threshold can be a range of values ​​on a priority scale (e.g., 1-100). Computer 104 can allow simultaneous output of two content datasets 300 and 302 (provided that audio isolation or personal audio device 110 is available for those content datasets 300 and 302 that include audio streams), wherein both content datasets 300 and 302 have priority scores within the change threshold. Otherwise, if content datasets 300 and 302 have priority scores that change by an amount greater than the change threshold, computer 104 can pause or mute content datasets 300 and 302 with lower priority scores. For example, the first content dataset 300 may have a priority score of 50, and the second content dataset 302 may have a priority score of 60. If the change threshold is 20, both content datasets 300 and 302 can be played simultaneously. However, if the change threshold is 5, computer 104 can pause or mute the first content data.

[0098] refer to Figures 1 to 3 Description Figure 4 An example process 400 is shown for determining the priority ranking of content datasets 300 and 302 and for displaying content datasets 300 and 302 based on the corresponding priority ranking. The process can be executed according to program instructions executed by computer 104.

[0099] The process begins at box 410, where computer 104 determines the content type (e.g., podcast, navigation, alarm, etc.) of content datasets 300, 302 when multiple sets of content, typically associated with corresponding displays 108, are played in vehicle 102.

[0100] Next, in box 415, computer 104 identifies audio device 110 that receives content sets 300, 302. For example, device 110 may be identified as a front cabin speaker, a rear cabin speaker, a Bluetooth headset, a portable device, etc.

[0101] Next, in decision box 420, computer 104 determines whether any isolated device 110 exists in vehicle 102 (i.e., whether any audio device 110 identified in box 415 and receiving the corresponding content sets 300, 302 is an isolated device 110). If an isolated device exists, the process continues to box 425. Otherwise, the process continues to box 430.

[0102] In box 425, computer 104 has determined that an isolated audio device 110 is present in vehicle 102. Computer 104 removes the isolated device from the priority ranking calculation because the isolated device 110 may only output those content datasets 300, 302 specifically selected by users 200, 202 of the isolated audio device 110, and this audio output is generally not audible to other occupants of the vehicle 102.

[0103] Next, in box 430, computer 104 retrieves the stored baseline priority scores assigned to each content dataset based on the content type of each content dataset 300, 302.

[0104] Next, in box 435, computer 104 determines one or more vehicle states based on the aforementioned stored vehicle state diagram.

[0105] Next, in decision box 440, computer 104 determines whether vehicle 102 is in a vehicle state where it is necessary to adjust the baseline priority score. For example, as explained above, a vehicle in a “potential lane violation state” may require priority score adjustment. Furthermore, priority scores can be adjusted, for example, based on user data such as gaze direction or data of subtypes such as content datasets 300, 302. If priority score adjustment is required, process 400 proceeds to box 445. Otherwise, the process continues to box 455.

[0106] In box 445, computer 104 retrieves a lookup table, etc., which specifies the adjustment to be made to the priority score of each content dataset 300, 302 based on the current vehicle status and / or the aforementioned factors (e.g., content subtype, gaze direction, potential lane violation status, etc.).

[0107] Next, in box 450, computer 104 adjusts the priority scores of content datasets 300 and 302. For example, computer 104 can make such adjustments based on the lookup table and Equation 1 above.

[0108] Next, in box 455, computer 104 outputs the audio streams of the corresponding content datasets 300 and 302 according to their priority scores. As mentioned above, the highest priority content datasets 300 and 302 can be output, while the lower priority content datasets 300 and 302 can have their output modes adjusted to be lowered or muted.

[0109] Next, in box 460, computer 104 determines whether to continue process 400. For example, once process 400 is started, computer 104 can return to box 410 to continue determining priority ranking. However, process 400 may end after an input or event occurs that terminates process 400, such as user 200, 202 stopping the operation of vehicle 102 (e.g., shutting off a propulsion system such as the engine), user providing input to end process 400, etc. If process 400 is to continue, the process returns to box 410. Otherwise, process 400 ends.

[0110] This disclosure has been described in an illustrative manner, and it should be understood that the terminology used is intended to be descriptive in nature and not restrictive. In light of the foregoing teachings, many modifications and variations of this disclosure are possible, and this disclosure may be practiced in ways other than those specifically described. The adjectives “first” and “second” are used throughout this document as identifiers and are not intended to indicate importance, order, or quantity. The use of “in response to,” “after determining,” etc., indicates a causal relationship, not merely a temporal one. The operations, systems, and methods described herein should always be implemented and / or performed in accordance with applicable user manuals and / or safety guidelines.

[0111] According to the present invention, a system is provided having a computer for a vehicle, the computer including a processor and a memory, the memory storing instructions executable by the processor, the instructions including instructions for performing the following operations: receiving a first content dataset and a second content dataset, the first content dataset including a first graphics component and a first audio stream, the second content dataset including a second graphics component and a second audio stream; and selecting a first audio output mode of the first audio stream and a second audio output mode of the second audio stream based on a priority determination including a priority of the first content dataset and / or a priority of the second content dataset, wherein the priority determination is based on vehicle status.

[0112] According to an embodiment, the first graphics component is output by a first display, and the second graphics component is output by a second display.

[0113] According to an embodiment, the vehicle status can be a normal moving state, and the priority is determined based on the normal moving state.

[0114] According to an embodiment, the vehicle status is a parked state, and the priority is determined based on the parked state.

[0115] According to an embodiment, the instructions also include instructions for performing the following operations: receiving third content data, the third content data including a third graphics component and a third audio stream and having a priority based on vehicle state.

[0116] According to an embodiment, the first audio output mode and / or the second audio output mode include muting or suppressing audio.

[0117] According to an embodiment, selecting an audio output mode includes selecting an audio device to output an audio stream.

[0118] According to an embodiment, the instructions include further instructions for performing the following operations: determining whether the audio device is configured for audio isolation, and selecting an output mode based on whether the audio device is configured for audio isolation.

[0119] According to an embodiment, the priority of the first content dataset and the second content dataset is based on the identity of the user associated with the first content dataset or the second content dataset.

[0120] According to an embodiment, the priority of the first content dataset and the second content dataset is based on the content type of the first content dataset and the second content dataset.

[0121] According to an embodiment, the priority of the first content dataset and the second content dataset is based on the user's gaze direction.

[0122] According to the present invention, a method includes: receiving a first content dataset and a second content dataset, the first content dataset including a first graphics component and a first audio stream, the second content dataset including a second graphics component and a second audio stream; and selecting a first audio output mode of the first audio stream and a second audio output mode of the second audio stream based on a priority determination including a priority of the first content dataset and / or a priority of the second content dataset, wherein the priority determination is based on vehicle state.

[0123] In one aspect of the invention, a first graphics component is output by a first display, and a second graphics component is output by a second display.

[0124] In one aspect of the invention, the vehicle is in a normal moving state, and a priority is determined based on the normal moving state.

[0125] In one aspect of the invention, the vehicle state is a parked state, and a priority is determined based on the parked state.

[0126] In one aspect of the invention, the method includes receiving third content data, the third content data including a third graphics component and a third audio stream and having a priority based on vehicle state.

[0127] In one aspect of the invention, the first audio output mode and / or the second audio output mode include muting or suppressing audio.

[0128] In one aspect of the invention, selecting an audio output mode includes selecting an audio device to output an audio stream.

[0129] In one aspect of the invention, the method includes determining whether an audio device is configured for audio isolation, and selecting an output mode based on whether the audio device is configured for audio isolation.

[0130] In one aspect of the invention, the priority of the first content dataset and the second content dataset is based on the identity of the user associated with the first content dataset or the second content dataset.

Claims

1. A method comprising: Receive a first content dataset and a second content dataset, wherein the first content dataset includes a first graphics component and a first audio stream, and the second content dataset includes a second graphics component and a second audio stream; as well as A first audio output mode for the first audio stream and a second audio output mode for the second audio stream are selected based on a priority determination that includes the priority of the first content dataset and / or the priority of the second content dataset, wherein the priority determination is based on vehicle status.

2. The method of claim 1, wherein the first graphics component is output by a first display, and the second graphics component is output by a second display.

3. The method of claim 1, wherein the vehicle state is a normal moving state, and the priority is determined based on the normal moving state.

4. The method of claim 1, wherein the vehicle state is a parked state, and the priority is determined based on the parked state.

5. The method of claim 1, further comprising receiving third content data, the third content data including a third graphics component and a third audio stream and having a priority based on the vehicle state.

6. The method of claim 1, wherein the first audio output mode and / or the second audio output mode includes muting or suppressing audio.

7. The method of claim 1, wherein selecting the audio output mode includes selecting an audio device to output the audio stream.

8. The method of claim 1, further comprising determining whether the audio device is configured for audio isolation, and selecting the output mode based on whether the audio device is configured for audio isolation.

9. The method of claim 1, wherein the priority of the first content dataset and the second content dataset is based on the identity of the user associated with the first content dataset or the second content dataset.

10. The method of claim 1, wherein the priority of the first content dataset and the second content dataset is based on the user's gaze direction.

11. The method of claim 1, wherein the first content data and the second content data are selected by the user.

12. The method of claim 1, wherein the first display is a panoramic display.

13. The method of claim 1, further comprising outputting the first graphics component and the second graphics component from the first display.

14. The method of claim 1, wherein the priority of the first content dataset and the second content dataset is based on a priority score.

15. A remote computer comprising a processor and a memory storing instructions executable by the processor to perform the method as claimed in any one of claims 1 to 14.

Images