Cabin multi-tone area management method, related device and communication system

Abstract

The invention provides a cockpit multi-tone area management method, a related device and a communication system. The cabin system may angry the plurality of screens into a plurality of ride zones according to a profile. Each riding area can be associated with a sound area. The cabin system can dynamically create and delete the sound area according to the access and removal of the sounding device. The cabin system can perform unified management on audio focus, volume, audio routing and media session in a sound area. The application corresponding to one driving area can apply for the audio focus in the sound area associated with the driving area, and the sound production equipment in the sound area is used for playing the audio. Audio playing in different sound areas can be independent and does not affect each other. Therefore, different users in the vehicle can use different sound areas to listen to the audio conveniently, and the audio listening experience of the users in the vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent automobile technology, and in particular to a cockpit multi-sound zone management method, related devices and communication systems. Background Art

[0002] With the development of electronic products, more and more electronic products are installed in vehicles. Vehicles are also becoming more and more intelligent. The cockpit system of an intelligent vehicle can be an intelligent and networked vehicle-mounted product, which can interact intelligently with people, roads, and the vehicle itself. Today's cockpit system can support more and more screens and sound devices. Based on these screens and sound devices, the cockpit system can be divided into one or more sound zones. The cockpit system needs to manage the sound zones to provide users sitting in different positions in the vehicle with a better driving experience. Summary of the invention

[0003] The present application provides a cockpit multi-zone management method, related devices and communication system, which can dynamically create or delete audio zones according to changes in external sound devices in the cockpit system. The cockpit system can uniformly manage the audio focus, volume, audio routing, and media sessions in a sound zone. In addition, the audio playback in each sound zone can be independent and does not affect each other. This can facilitate different users in the vehicle to listen to audio in different sound zones, thereby improving the user's audio listening experience in the vehicle.

[0004] In the first aspect, the present application provides a method for managing multiple audio zones in a cockpit, which is applied to a vehicle, wherein the vehicle includes an in-vehicle speaker and multiple screens, wherein the multiple screens include a first screen and a second screen, and both the first screen and the second screen are associated with a first audio zone corresponding to the in-vehicle speaker. The vehicle may detect that a first application on the second screen requests to play a first audio, and plays the first audio through the in-vehicle speaker; the vehicle may detect that a second application on the first screen requests to play a second audio, and plays the second audio through the in-vehicle speaker, and pauses the first audio or reduces the volume of the first audio; the vehicle may detect an input operation on the second screen that accesses a first sound device, and the vehicle connects to the first sound device and creates a second audio zone corresponding to the first sound device; wherein the audio zone associated with the second screen changes from the first audio zone to the second audio zone; the vehicle may detect again that a first application requests to play the first audio, and plays the first audio through the first sound device.

[0005] It can be seen that when the vehicle has no external sound device, the vehicle may include a first sound zone associated with the in-vehicle speaker. All screens in the vehicle may be associated with the first sound zone. When the vehicle is connected to an external sound device, the vehicle may create a new sound zone, and the corresponding screen may be associated with the newly created sound zone. In this way, the sound zone associated with the screen may be adjusted dynamically, so that the audio applied on different screens may be separated and played on different sound devices. In this way, different users may connect different sound devices to different screens, and then use different sound devices to listen to the audio they want to listen to without affecting each other. This may improve the audio listening experience of users sitting in different positions in the vehicle.

[0006] In combination with the first aspect, in some embodiments, a first configuration file is stored in the vehicle, and the first configuration file is used to indicate the correspondence between multiple screens and driving areas, wherein any one of the multiple screens corresponds to a driving area, the sound zone associated with the screen corresponding to the same driving area remains consistent, and the sound zones associated with screens corresponding to different driving areas are the same or different.

[0007] The cockpit systems of different models may contain different screens. In the above embodiment, the driving and riding areas of different models can be configured by simply modifying the configuration files for different models. The cockpit systems of different models can create driving and riding areas according to their corresponding configuration files.

[0008] In combination with the first aspect, in some embodiments, the vehicle includes a first audio focus stack corresponding to the first sound zone, and the first audio focus stack includes audio focus information of an application running in the foreground or background on a screen associated with the first sound zone; after the vehicle detects an input operation of the first sound device connected to the second screen, it can create a second audio focus stack corresponding to the second sound zone, and migrate the audio focus information of the application running in the foreground or background on the second screen from the first audio focus stack to the second audio focus stack.

[0009] In combination with the first aspect, in some embodiments, the vehicle includes a first audio routing table corresponding to a first sound zone, the first audio routing table includes application information of an application running in the foreground or background on a screen associated with the first sound zone, and a first audio channel of an in-vehicle speaker is configured in the first audio routing table, which is used to indicate that the audio of the application corresponding to the application information in the first audio routing table is routed to the first audio channel; after the vehicle detects an input operation of accessing a first sound device on the second screen, it can create a second audio routing table corresponding to the second sound zone, and migrate the application information of the application running in the foreground or background on the second screen from the first audio routing table to the second audio routing table; and create a second audio channel of the first sound device, and configure the second audio channel in the second audio routing table, the second audio channel configured in the second audio routing table is used to indicate that the audio of the application corresponding to the application information in the second audio routing table is routed to the second audio channel.

[0010] In combination with the first aspect, in some embodiments, the vehicle includes a first media session stack corresponding to a first sound zone, and the first media session stack includes a media session of an application running in the foreground or background on a screen associated with the first sound zone; after the vehicle detects an input operation of a first sound device connected to a second screen, it can create a second media session stack corresponding to a second sound zone, and migrate the media session of an application running in the foreground or background on the second screen from the first media session stack to the second media session stack.

[0011] In combination with the first aspect, in some embodiments, before the vehicle detects an input operation on the second screen for accessing the first sound device, when an input operation on the first screen for viewing audio playback information is detected, the vehicle can display a first list on the first screen according to the first media session stack, and the first list includes the audio playback information indicated by the first media session stack; when an input operation on the second screen for viewing the audio playback information is detected, the vehicle displays the first list on the second screen according to the first media session stack.

[0012] In combination with the first aspect, in some embodiments, the first list includes audio playback information of a first audio; when the second audio is played by the in-vehicle speaker and the first audio is paused, the vehicle detects an input operation on the audio playback information of the first audio in the first list on the first screen, and the input operation on the audio playback information of the first audio in the first list is used to play the first audio; in combination with the first aspect, in some embodiments, the first audio is played through the in-vehicle speaker and the second audio is paused.

[0013] In combination with the first aspect, in some embodiments, after the vehicle detects an input operation on the second screen for accessing the first sound device, it can detect an input operation on the first screen for viewing audio playback information, and display a second list on the first screen according to the first media session stack, the second list including the audio playback information indicated by the first media session stack, and the second list does not include the audio playback information of the first audio; detects an input operation on the second screen for viewing the audio playback information, and displays a third list on the second screen according to the second media session stack, the third list including the audio playback information indicated by the second media session stack, and the second list includes the audio playback information of the first audio.

[0014] In combination with the first aspect, in some embodiments, after the vehicle is connected to the first sound-emitting device, an input operation on the second screen for adjusting the volume to the first volume is detected, and the volume of the audio played by the first sound-emitting device is adjusted to the first volume.

[0015] In combination with the first aspect, in some embodiments, after the vehicle is connected to the first sound-emitting device, an input operation on the first screen for adjusting the volume to a second volume is detected, and the volume of the audio played by the in-vehicle speaker is adjusted to the second volume.

[0016] It can be seen that the volume in different sound zones can be adjusted independently without affecting each other.

[0017] In combination with the first aspect, in some embodiments, the first screen displays a first interface of a second application, and the first interface is a playback interface of a second audio. After the vehicle is connected to the first sound device, when an input operation to migrate the first interface from the first screen to the second screen is detected, the vehicle can display the first interface on the second screen, cancel the display of the first interface on the first screen, and switch the second audio from the in-vehicle speakers to the first sound device to continue playing.

[0018] In combination with the first aspect, in some embodiments, after the vehicle switches the second audio from the in-vehicle speaker to the first sound device to continue playing, when it detects that the third application on the first screen requests to play the third audio, the vehicle plays the third audio through the in-vehicle speaker.

[0019] In combination with the first aspect, in some embodiments, after the vehicle switches the second audio from the in-vehicle speaker to the first sound device to continue playing, when it detects that the first application on the second screen requests to play the first audio, the vehicle plays the first audio through the first sound device and pauses the second audio or lowers the playback volume of the second audio.

[0020] In combination with the first aspect, in some embodiments, the vehicle includes a first audio focus stack, a first audio routing table, and a first media session stack corresponding to a first audio zone, and a second audio focus stack, a second audio routing table, and a second media session stack corresponding to a second audio zone, the first audio focus stack including audio focus information of the second application, the first audio routing table including application information of the second application, and the first media session stack including the media session stack of the first application; after the vehicle detects an input operation to migrate the first interface from the first screen to the second screen, the audio focus information of the second application can be migrated from the first audio focus stack to the second audio focus stack, the application information of the second application can be migrated from the first audio routing table to the second audio routing table, and the media session of the second application can be migrated from the first media session stack to the second media session stack.

[0021] It can be seen that during the screen migration process, the migrated application does not need to adapt the multimedia display interface and audio stream according to the screen after the screen migration. Since the application information of the second application mentioned above is migrated to the audio routing table associated with the second sound zone, the second application can find itself in the audio routing table associated with the second sound zone, and then transmit the audio of the second application to the audio channel of the first sound device according to the audio routing table corresponding to the second sound zone. This can achieve the seamless switching of the audio of the second application from the in-car speaker to the first sound device, maintaining the continuity of the audio playback. The seamless switching of the audio of the second application from the in-car speaker to the first sound device mentioned above can mean that when the audio of the second application is switched to the first sound device, the first sound device can continue to play the audio from the position where the in-car speaker stops playing the audio.

[0022] In combination with the first aspect, in some embodiments, the second interface of the first application is displayed on the second screen, and the second interface is the playback interface of the first audio; the vehicle again detects that the first application requests to play the first audio, and after playing the first audio through the first sound device, when the input operation of sharing the second interface from the second screen to the first screen is detected, the vehicle displays the second interface on the first screen and switches the first audio from the first sound device to the in-vehicle speakers for continued playback.

[0023] In combination with the first aspect, in some embodiments, the vehicle also includes a third screen, the third screen is associated with the third sound zone of the second sound device, the second sound device is connected to the vehicle, the vehicle detects an input operation to share the second interface from the second screen to the third screen, and displays the second interface on the third screen.

[0024] In combination with the first aspect, in some embodiments, after the vehicle switches the first audio from the first sound-emitting device to the in-vehicle speaker to continue playing, when it detects that the fourth application on the second screen requests to play the fourth audio, the vehicle plays the fourth audio through the first sound-emitting device.

[0025] In combination with the first aspect, in some embodiments, the vehicle detects that the fifth application on the third screen requests to play the fifth audio, and plays the fifth audio through the second sound-emitting device.

[0026] In combination with the first aspect, in some embodiments, after the vehicle switches the first audio from the first sound-emitting device to the in-vehicle speaker to continue playing, when it detects that the sixth application on the first screen requests to play the sixth audio, the vehicle plays the sixth audio through the in-vehicle speaker and pauses playing the first audio.

[0027] In combination with the first aspect, in some embodiments, the vehicle includes a first audio focus stack and a first audio routing table corresponding to the first audio zone, and a second audio focus stack and a second audio routing table corresponding to the second audio zone, the second audio focus stack includes audio focus information of the first application, and the second audio routing table includes application information of the first application; after the vehicle detects an input operation to share the second interface from the second screen to the first screen, the audio focus information of the first application is migrated from the second audio focus stack to the first audio focus stack, and the application information of the first application is migrated from the second audio routing table to the first audio routing table.

[0028] It can be seen that during the screen sharing process, the shared application does not need to adapt the multimedia display interface and audio stream according to the screen receiving the screen sharing. Since the audio routing strategy of the first application mentioned above is migrated to the audio routing table associated with the first audio zone, the audio track of the first application can find itself in the audio routing table associated with the first audio zone, and then transmit the audio of the first application to the audio channel of the in-car speaker according to the audio routing table associated with the first audio zone. This can achieve seamless switching of the audio of the first application from the first sound device to the in-car speaker, maintaining the continuity of audio playback.

[0029] In combination with the first aspect, in some embodiments, after the vehicle is connected to the first sound-emitting device, when it is detected that the first sound-emitting device is disconnected, the vehicle changes the sound zone associated with the second screen from the second sound zone to the first sound zone; when the in-vehicle speaker is playing the second audio, it is detected that the first application on the second screen requests to play the first audio, the first audio is played through the in-vehicle speaker, and the second audio is paused or the volume of the second audio is lowered.

[0030] In combination with the first aspect, in some embodiments, the vehicle includes a first audio focus stack corresponding to the first sound zone, and a second audio focus stack corresponding to the second sound zone. After the vehicle detects that the first sound device is disconnected, the audio focus information in the second audio focus stack is migrated to the first audio focus stack, and the second audio focus stack is deleted.

[0031] In combination with the first aspect, in some embodiments, the vehicle includes a first audio routing table corresponding to the first audio zone, and a second audio routing table corresponding to the second audio zone. After the vehicle detects that the first sound-emitting device is disconnected, the application information in the second audio routing table is migrated to the first audio routing table, and the second audio routing table is deleted.

[0032] In combination with the first aspect, in some embodiments, the vehicle includes a first media session stack corresponding to the first audio zone and a second media session stack corresponding to the second audio zone. After the vehicle detects that the first sound device is disconnected, the media session in the second media session stack is migrated to the first media session stack, and the second media session stack is deleted.

[0033] In combination with the first aspect, in some embodiments, the vehicle further includes a fourth screen, the fourth screen is consistent with the sound zone associated with the second screen, and after the first application is detected to request to play the first audio again, the vehicle plays the first audio through the first sound device, and then detects that the seventh application on the fourth screen requests to play the seventh audio, plays the seventh audio through the first sound device, and pauses the first audio or reduces the volume of the seventh audio. The fourth screen and the second screen may correspond to the same driving zone.

[0034] In a second aspect, the present application provides a vehicle, which may include a screen, an in-vehicle speaker, a memory, and a processor. The screen may be used to display a user interface. The in-vehicle speaker may be used to play audio. The memory may be used to store a computer program. The processor may be used to call the computer program so that the vehicle executes any possible implementation method in the first aspect.

[0035] In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a vehicle, causes the vehicle to execute any possible implementation method in the first aspect.

[0036] In a fourth aspect, the present application provides a computer program product, which may include computer instructions. When the computer instructions are run on a vehicle, the vehicle executes any possible implementation method in the first aspect.

[0037] In a fifth aspect, the present application provides a chip, which is applied to a vehicle, and the chip includes one or more processors, and the processor is used to call computer instructions to enable the vehicle to execute any possible implementation method as in the first aspect.

[0038] It can be understood that the vehicle provided in the second aspect, the computer-readable storage medium provided in the third aspect, the computer program product provided in the fourth aspect, and the chip provided in the fifth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a schematic diagram of the relationship between a driving area and a screen provided in an embodiment of the present application;

[0040] Figure 2 is a structural schematic diagram of a vehicle 100 provided in an embodiment of the present application;

[0041] Figure 3 It is a schematic diagram of a cockpit audio partition architecture provided by an embodiment of the present application;

[0042] Figure 4is a schematic diagram of the architecture of a cockpit system provided in an embodiment of the present application;

[0043] Figure 5 It is a schematic diagram of a cockpit audio partition architecture when no external sound device is connected, provided in an embodiment of the present application;

[0044] Figure 6 It is a schematic diagram of a cockpit audio partition architecture when an external headset 1 and a headset 2 are connected, provided in an embodiment of the present application;

[0045] Figure 7 It is a flow chart of a method for creating a sound zone provided in an embodiment of the present application;

[0046] Figure 8 is a schematic diagram of an external sound device for a cockpit system provided in an embodiment of the present application;

[0047] Fig. 9 This is a flow chart of a method for creating an audio channel provided by an embodiment of the present application;

[0048] Fig.10 and Fig.11 It is a schematic diagram of a specific process of creating a sound zone by a cockpit system provided in an embodiment of the present application;

[0049] Fig.12 It is a flow chart of a method for deleting an audio channel when a sound device is removed provided by an embodiment of the present application;

[0050] Fig.13 and Fig.14 is a schematic diagram of a specific process of deleting a sound zone in a cockpit system provided in an embodiment of the present application;

[0051] Figures 15A to 15D is a schematic diagram of an audio playback scenario in which some sound-emitting devices are removed provided in an embodiment of the present application;

[0052] Fig.16 is a schematic diagram of audio stream transmission in different audio zones provided by an embodiment of the present application;

[0053] Fig.17 is a schematic diagram of applying for audio focus provided by an embodiment of the present application;

[0054] Fig.18 is a schematic diagram of adjusting the volume of a multimedia playback application provided in an embodiment of the present application;

[0055] Fig.19 is a schematic diagram of a system application for adjusting volume provided in an embodiment of the present application;

[0056] Figures 20A to 20D Schematic diagrams of some volume adjustment scenarios provided in embodiments of the present application;

[0057] Fig.21 is a schematic diagram of a media session management provided by an embodiment of the present application;

[0058] Fig. 22 is a schematic diagram of a screen migration method provided by an embodiment of the present application;

[0059] Fig.23 and Fig.24 is a schematic diagram of a screen migration scenario provided by an embodiment of the present application;

[0060] Fig.25 is a schematic diagram of a screen sharing method provided by an embodiment of the present application;

[0061] Fig.26 and Fig. 27 is a schematic diagram of a screen sharing scenario provided in an embodiment of the present application;

[0062] Figures 28A to 28F These are some other screen sharing scene diagrams provided in the embodiments of the present application. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in a "or" relationship.

[0064] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0065] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0066] The term "user interface (UI)" in the following embodiments of the present application refers to the medium interface for interaction and information exchange between an application (APP) or an operating system (OS) and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that can be recognized by the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the screen of an electronic device.

[0067] The cockpit system may include multiple screens and multiple sound devices. Among these multiple screens, there may be multiple applications applying for audio focus in order to obtain permission to use the sound device for audio playback. The cockpit system needs to manage the application for audio focus, determine which applications can obtain audio focus, and route the audio of the application that obtains audio focus to the appropriate sound device. When a user operation to adjust the volume is received, the cockpit system needs to determine which sound device's volume to adjust. It can be seen that the audio zone management in the cockpit system involves audio routing, volume management, audio focus management and other contents.

[0068] The cockpit system can also include the vehicle's in-vehicle operating system. The in-vehicle operating system can be used to manage and control in-vehicle software and hardware resources, support the vehicle's upper-layer software development and data connection, and support the operation of upper-layer software. The in-vehicle operating system can also be used to control the operation of applications and provide multiple forms of human-computer interaction interfaces.

[0069] The cockpit system may also be called cockpit, smart cockpit, etc.

[0070] In some embodiments, a cockpit system based on the Android Open Source Project (AOSP) includes an audio routing-related management module, a volume-related management module, and an audio focus-related management module. Among them, the audio routing-related management module can match the audio content (context) and the playback address (address) according to car_audio_configuration.xml, and then generate an audio routing strategy. The volume-related management module can be used to manage the volume in each audio zone, and classify different contexts into different volume groups (VolumeGroup) for management. The audio focus-related management module manages the audio focus according to the focus strategy of one audio corresponding to one audio focus. These management modules work independently, which is not conducive to the unified management of the audio zones by the cockpit system. The above-mentioned cockpit system based on AOSP lacks unified management of media sessions applied in multiple audio zones.

[0071] In addition, in the scenario where audio flows from one audio zone to another in the cockpit system, the above-mentioned cockpit system based on AOSP requires the audio playback application to adapt (for example, the audio is adapted to different audio zones, the audio playback interface is adapted to the screens corresponding to different audio zones, etc.). Different audio and video applications have different adaptation methods, which not only increases the workload of audio and video application developers, but also affects the user's audio listening experience.

[0072] The present application provides a cockpit multi-zone management method, which can manage each audio zone independently. Among them, the cockpit system can uniformly manage the audio focus, volume, audio routing, and media session in a sound zone. The cockpit system can divide the vehicle's built-in sound devices (such as in-car speakers) into a sound zone, and divide the sound devices connected to the cockpit system (such as headphones) into one or more sound zones. The cockpit system can dynamically create and delete sound zones based on the access and removal of sound devices.

[0073] The present application constructs a hierarchical cockpit audio partition architecture, which divides the audio system part in the cockpit system into multiple levels, and can support flexible configuration of the screen and audio in the cockpit. Among them, the audio system part in the cockpit system can be divided into four levels: screen, driving area, sound zone, and sound device. The screen in the cockpit system can be divided into multiple driving areas. Each driving area can be associated with a sound zone. An application on a screen can apply for audio focus in the sound zone associated with the driving area to which this screen belongs. That is, the audio applied on a screen can be played by the sound device corresponding to the sound zone associated with the driving area to which this screen belongs. The relationship between the above-mentioned screen and the driving area can be determined according to the configuration file. The association relationship between the above-mentioned driving area and the sound zone can be determined according to the correspondence between the sound device corresponding to the sound zone and the screen. The correspondence between the sound device and the screen can be dynamically variable. Therefore, the association relationship between the driving area and the sound zone is also dynamically variable. Based on the above-mentioned hierarchical cockpit audio partition architecture, different models can implement the multi-sound zone management method of the present application through configuration.

[0074] In scenarios where audio streaming is required, the cockpit system can uniformly migrate the audio focus, audio routing, and media session of the audio playback application in one audio zone to another audio zone, so that the sound device playing the audio can be seamlessly switched from the sound device corresponding to one audio zone to the sound device corresponding to another audio zone. The sound device corresponding to the other audio zone can continue to play the above audio. In this way, the audio is transferred from one sound device to another without causing the audio to be interrupted or played from the beginning, which can bring a good audio listening experience to the user, and the audio playback application is unaware of the above audio streaming scenario, and the audio playback application does not need to adapt.

[0075] To facilitate understanding, some concepts involved in this application are introduced below.

[0076] 1. Audio Focus

[0077] Audio focus can refer to the permission to use a sound device to play audio. When an APP obtains audio focus, this APP can use the sound device to play audio. When an APP loses audio focus, this APP will pause or stop playing audio, or lower the volume of playing audio. It is understandable that when multiple APPs use the same sound device to play audio at the same time, the audio of these multiple APPs will be mixed together. After mixing, the user may not be able to hear the content being played clearly. In order to avoid all applications using a sound device to play audio at the same time, usually only one application can obtain audio focus in the same time period. When an application holds audio focus, other applications can still apply for audio focus or seize audio focus.

[0078] The types of audio focus can include: long focus, short focus, ducking focus, and exclusive focus. When applying for audio focus, the application can apply for any of the above types of audio focus as needed.

[0079] Long focus (AUDIOFOCUS_GAIN) can indicate an audio focus that needs to be occupied for a long time. When the audio focus of the application is a long focus, the application will stop playing audio after losing the long focus. In some embodiments, after the application holding the long focus stops playing audio, other applications that previously played audio will not resume playing. For example, when music APP1 holds the audio focus to play music, in response to the operation of opening music APP2 to play music, music APP2 will apply for a long focus. Music APP2 can seize the audio focus of music APP1 and start playing music. When music APP2 pauses playing music, music APP1 will not actively resume playing. Among them, when music APP2 seizes the audio focus of music APP1, the cockpit system can delete the audio focus information of music APP1 in the audio focus stack. At this time, music APP1 needs to reapply for audio focus if it wants to play music. That is to say, when an application applies for a long focus, the cockpit system can clear the audio focus stack. The application corresponding to the audio focus information at the top of the audio focus stack can be the application that currently holds the audio focus and uses the sound device to play audio.

[0080] Short focus (AUDIOFOCUS_TRANSIENT) can represent an audio focus that is occupied for a short time. Short focus can cause the application that loses the audio focus to pause playback, and the application that loses the audio focus can resume playback after the short focus is abandoned. For example, when the music APP1 holds the audio focus and plays music, the phone application receives an incoming call. The phone application can apply for short focus. The phone application can seize the audio focus of the music APP1 and start playing the incoming call prompt tone. When the call ends, the phone application can actively abandon the short focus, and the music APP1 can regain the audio focus and resume playing music. Among them, when the phone application seizes the audio focus of the music APP1, the cockpit system can place the audio focus information of the phone application at the top of the stack in the audio focus stack, and place the audio focus information of the music APP1 below the top of the stack (that is, the last bit of the audio focus information of the phone application). When the call ends, the cockpit system can delete the audio focus information of the phone application in the audio focus stack. In this way, the audio focus information of the music APP1 in the audio focus stack is back at the top of the stack. Music APP1 can actively resume playback after the call of the phone application ends.

[0081] Ducking focus (AUDIOFOCUS_TRANSIENT_MAY_DUCK) can indicate an audio focus that is occupied for a short time. Unlike short focus, ducking focus allows the application that previously acquired the audio focus to lower the volume and play at the same time as the application that newly acquired the ducking focus. For example, in a scenario where you are listening to music and navigating at the same time, when music APP1 holds the audio focus and plays music, the navigation application needs to play the navigation prompt tone. The navigation application can apply for ducking focus. The navigation application can seize the audio focus of music APP1 and start playing the navigation prompt tone. When the navigation application plays the navigation prompt tone, music APP1 can lower the volume and play music at the same time to ensure that the user can hear the navigation prompt tone more clearly. When the navigation prompt tone is played, the navigation application can actively give up the ducking focus, and music APP1 can restore the volume of playing music. Among them, when the navigation application seizes the audio focus of music APP1, the cockpit system can place the audio focus information of the navigation application at the top of the stack in the audio focus stack, and place the audio focus information of music APP1 below the top of the stack. When the navigation prompt sound is played, the cockpit system can delete the audio focus information of the navigation application in the audio focus stack. In this way, the audio focus information of the music APP1 in the audio focus stack is relocated to the top of the stack. The music APP1 can actively restore the volume of the music playback.

[0082] Exclusive focus (AUDIOFOCUS_TRANSIENT_EXCLUSIVE) can indicate audio focus that is occupied for a short time. Unlike short focus and evasive focus, exclusive focus has the highest priority, which allows an application that applies for exclusive focus to seize the audio focus of other applications, but does not allow other applications to seize the exclusive focus when applying for audio focus.

[0083] There can be one or more audio zones in the cockpit system. One audio zone has one audio focus. The applications holding the audio focus in different audio zones can be different. Therefore, each audio zone can correspond to an audio focus stack. The cockpit system can manage the audio focus stack of each audio zone based on the type of focus requested by the application. Multiple applications corresponding to a driving zone can compete for audio focus in the audio zone associated with this driving zone.

[0084] 2. Audio Routing

[0085] Each sound device in the cockpit system has its own audio channel, which can be used to transmit audio playback data.

[0086] Audio routing refers to the relationship between an application and the audio channel it uses when playing audio. The cockpit system can determine which audio channel the application's audio is transmitted through based on the audio routing, that is, which sound device the application's audio is played on.

[0087] 3. MediaSession

[0088] The media session can be a bridge between the cockpit system and the multimedia playback application. The multimedia playback application can include an APP for playing audio and / or video. The multimedia playback application can create a media session when playing audio and / or video. The media session can inform the cockpit system that the multimedia playback application is playing multimedia content (such as audio and / or video), and can also inform the cockpit system what multimedia content is being played and how to control the multimedia content being played (such as pause / play / fast forward / fast rewind).

[0089] In some embodiments, the cockpit system can display controls for controlling the playback of multimedia content in the user interface of the audio center based on the media session, which can facilitate the user to control the multimedia content outside the multimedia playback application.

[0090] 4. Occupant Zone

[0091] The driving area can be divided based on the screen in the cockpit system. The cockpit system can include one or more driving areas. One driving area can correspond to one or more screens.

[0092] Please refer to Figure 1, Figure 1 The diagram shows the relationship between the driving area and the screen.

[0093] Exemplarily, a cockpit system may include driving zone 0, driving zone 1, driving zone 2, and driving zone 3. The screen corresponding to driving zone 0 may include a central control screen, an instrument screen, and a head up display (HUD). The screen corresponding to driving zone 1 may include a co-pilot screen. The co-pilot screen may refer to a screen set in the area where the co-pilot seat is located, which is convenient for users sitting in the co-pilot seat to watch. The screen corresponding to driving zone 2 may include rear screen 1, rear screen 2, armrest screen, and rear curtain. Rear screen 1 and rear screen 2 may refer to screens set in the area where the rear seats are located, which is convenient for users sitting in the rear seats to watch. The screen corresponding to driving zone 3 may include a virtual screen. The virtual screen may be a screen set to use the external speakers configured outside the vehicle as an independent sound zone. The virtual screen may be a screen without a physical entity, or it may be a screen with a physical entity outside the vehicle. Driving zone 3 may be associated with the sound zone corresponding to the external speakers. In this way, the user can use the external speakers to play audio by transferring the audio stream to the virtual screen.

[0094] Occupant zone 0 may be referred to as OccupantZone0. Occupant zone 1 may be referred to as OccupantZone1. Occupant zone 2 may be referred to as OccupantZone2. Occupant zone 3 may be referred to as OccupantZone3. A screen corresponding to an occupant zone may be referred to as a screen in the occupant zone, or a screen bound to the occupant zone. The correspondence between an occupant zone and a screen may also be referred to as a binding relationship between an occupant zone and a screen.

[0095] It can be seen that the screen bound to the driving area 0 is mainly for the driver. The screen bound to the driving area 1 is mainly for the user sitting in the front passenger seat. The screen bound to the driving area 2 is mainly for the user sitting in the back seat. The screen bound to the driving area 3 can be used to serve the user outside the vehicle. The division of the driving area can be based on the distribution of the screens on the vehicle, dividing the vehicle into multiple areas: driver area, front passenger area, back seat area and outside area.

[0096] This application does not limit the division of the driving and passenger areas in the cockpit system. It is not limited to driving and passenger areas 0 to 3, and the cockpit system may include more or fewer driving and passenger areas. Figure 1 The binding relationship between the driving area and the screen is shown. The driving area in the cockpit system can be bound to more or fewer screens.

[0097] One or more applications can be configured on each screen in the cockpit system. Two different screens can be configured with different applications or with the same application. Take the central control screen and the co-pilot screen as an example. Both the central control screen and the co-pilot screen can be configured with an application market APP for installing APPs. In response to the operation of installing one or more applications in the application market APP on the central control screen, the cockpit system can configure the one or more applications on the central control screen. In response to the operation of installing another one or more applications in the application market APP on the co-pilot screen, the cockpit system can configure the other one or more applications on the central and co-pilot screens. In other words, the cockpit system can configure the application on the screen where the user operation for installing this application is performed. The embodiment of the present application does not limit the source of the applications in each screen.

[0098] The applications in the screen corresponding to the driving area may be referred to as applications corresponding to the driving area. When a driving area is bound to multiple screens, the applications corresponding to the driving area may include the applications in the multiple screens.

[0099] In some embodiments, a configuration file of the driving zone may be stored in the cockpit system. The configuration file may include binding rules between the screen and the driving zone. The cockpit system may create the driving zone according to the configuration file and determine the screen corresponding to the driving zone. The configuration file may be referred to as a first configuration file.

[0100] The cockpit systems of different models may contain different screens. In the above embodiment, the driving and riding areas of different models can be configured by simply modifying the configuration files for different models. The cockpit systems of different models can create driving and riding areas according to their corresponding configuration files.

[0101] 5. AudioZone

[0102] The audio zones may be divided based on the sound devices in the cockpit system. The cockpit system may include one or more audio zones. One audio zone may be associated with one or more sound devices.

[0103] For example, the cockpit system can divide all the in-vehicle speakers configured inside the vehicle into an audio zone and create audio zone 0 (AudioZone0). The sound device associated with audio zone 0 is the in-vehicle speaker. Optionally, the cockpit system can also divide the in-vehicle speaker into multiple audio zones. This embodiment of the application is not limited to this.

[0104] In some embodiments, when an external speaker is configured outside the vehicle, the cockpit system can divide the external speaker into a sound zone and create a sound zone associated with the external speaker.

[0105] In some embodiments, the cockpit system can be connected to one or more external sound-emitting devices, such as headphones. The headphones can include wired headphones, wireless headphones (such as Bluetooth headphones, etc.). When an external sound-emitting device, such as headphone 1, is detected, the cockpit system can divide headphone 1 into another audio zone and create audio zone 1 (AudioZone1). The sound-emitting devices associated with audio zone 1 can include headphone 1. When it is detected that headphone 1 is disconnected, the cockpit system can delete audio zone 1.

[0106] In some embodiments, the cockpit system can connect the audio zone 0 to the above Figure 1 As shown in the figure, the driving zone 0 is associated. In this way, the application corresponding to the driving zone 0 can compete for the audio focus in the audio zone 0, and use the in-car speakers to play audio after obtaining the audio focus.

[0107] In the case where there is only one audio zone, that is, audio zone 0, in the cockpit system, the cockpit system can associate all driving zones with audio zone 0. In this way, applications corresponding to all driving zones in the cockpit system compete for audio focus in audio zone 0, and play using the in-car speakers after obtaining the audio focus.

[0108] When the cockpit system is externally connected to the headset 1, the cockpit system can determine which screen the headset 1 is bound to. In response to an operation on a screen to connect an external sound device to the cockpit system, the cockpit system can bind the sound device to the screen. The sound device can be bound to the screen through which it is connected to the cockpit system. Then, the cockpit system can associate the sound zone corresponding to the sound device with the driving zone bound to the screen to which the sound device is bound.

[0109] For example, when earphone 1 is connected to the cockpit system through the co-pilot screen, the cockpit system can bind earphone 1 to the co-pilot screen. Then, the cockpit system can bind the audio zone 1 corresponding to earphone 1 to the Figure 1 The co-pilot screen is bound to driving zone 1. In this way, the application corresponding to driving zone 1 can compete for audio focus in audio zone 1, and use earphone 1 to play audio after obtaining audio focus.

[0110] Equipment hardware and software structure

[0111] The structure of the vehicle 100 involved in the present application and the software architecture of the cockpit system in the vehicle 100 are introduced below.

[0112] Figure 2 A schematic structural diagram of a vehicle 100 is shown as an example.

[0113] like Figure 2As shown, the vehicle 100 may include: a controller area network (CAN) bus 11, multiple electronic control units (ECU), an engine 13, a telematics box (T-box) 14, a transmission 15, a driving recorder 16, an anti-lock brake system (ABS) 17, a sensor system 18, a camera system 19, a microphone 20, a speaker 21, and the like.

[0114] The CAN bus 11 is a serial communication network that supports distributed control or real-time control and is used to connect various components of the vehicle 100. Any component on the CAN bus 11 can monitor all data transmitted on the CAN bus 11. The frames transmitted by the CAN bus 11 may include data frames, remote frames, error frames, and overload frames, and different frames transmit different types of data. In an embodiment of the present application, the CAN bus 11 can be used to transmit data involved in the multi-tone zone management method of various components. The specific implementation of the method can refer to the detailed description of the method embodiment below.

[0115] Not limited to the CAN bus 11, in some other embodiments, the various components of the vehicle 100 can also be connected and communicated through other methods. For example, the various components can also communicate through the vehicle Ethernet (Ethernet) local interconnect network (localinterconnect network, LIN) bus, FlexRay and common vehicle network system (media oriented systems, MOST) bus, etc., and the embodiments of the present application do not limit this. The following embodiments are described by each component communicating through the CAN bus 11.

[0116] The ECU is equivalent to the processor or brain of the vehicle 100, and is used to instruct the corresponding components to perform corresponding actions according to the instructions obtained from the CAN bus 11 or the operation input by the user. The ECU can be composed of a security chip, a microprocessor (microcontroller unit, MCU), a random access memory (random access memory, RAM), a read-only memory (random-only memory, ROM), an input / output interface (I / O), an analog / digital converter (A / D converter), and large-scale integrated circuits such as input, output, shaping, and driving.

[0117] There are many types of ECUs, and different types of ECUs can be used to achieve different functions.

[0118] The multiple ECUs in the vehicle 100 may include, for example, an engine ECU 121 , a telematics box (T-box) ECU 122 , a transmission ECU 123 , a driving recorder ECU 124 , an antilock brake system (ABS) ECU 125 , and the like.

[0119] The engine ECU 121 is used to manage the engine and coordinate the various functions of the engine, for example, it can be used to start the engine, shut down the engine, etc. The engine is a device that provides power to the vehicle 100. The engine is a machine that converts a certain form of energy into mechanical energy. The vehicle 100 can be used to convert the chemical energy of liquid or gas combustion, or convert electrical energy into mechanical energy and output power to the outside. The engine components can include two major mechanisms, the crank-connecting rod mechanism and the valve mechanism, as well as five major systems such as cooling, lubrication, ignition, energy supply, and starting system. The main components of the engine are the cylinder block, cylinder head, piston, piston pin, connecting rod, crankshaft, flywheel, etc.

[0120] The T-box ECU 122 is used to manage the T-box 14 .

[0121] T-box 14 is mainly responsible for communicating with the Internet, providing a remote communication interface for the vehicle 100, and providing services including navigation, entertainment, driving data collection, driving trajectory recording, vehicle fault monitoring, vehicle remote query and control (such as unlocking and closing, air conditioning control, window control, engine torque limit, engine start and stop, seat adjustment, query of battery power, fuel level, door status, etc.), driving behavior analysis, wireless hotspot sharing, road rescue, abnormal reminders, etc.

[0122] T-box14 can be used to communicate with the telematics service provider (TSP) and the electronic device on the user (such as the driver) side to realize the vehicle status display and control on the electronic device. When the user sends a control command through the vehicle management application on the electronic device, TSP will issue a request instruction to T-box14. After obtaining the control command, T-box14 sends a control message through the CAN bus and realizes the control of the vehicle 100, and finally feeds back the operation result to the vehicle management application on the user side electronic device. In other words, the data read by T-box14 through the CAN bus 11, such as vehicle condition report, driving report, fuel consumption statistics, violation query, location trajectory, driving behavior and other data, can be transmitted to the TSP background system through the network, and forwarded to the electronic device on the user side by the TSP background system for the user to view.

[0123] The T-box 14 may specifically include a communication module and a screen.

[0124] Among them, the communication module can be used to provide wireless communication functions, supporting the vehicle 100 to communicate with other devices through wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), ultra-wideband (UWB) and other wireless communication technologies. The communication module can also be used to provide mobile communication functions, supporting the vehicle 100 to communicate with other devices through global system for mobile communications (GSM), universal mobile telecommunications system (UMTS), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), 5G and future 6G and other communication technologies.

[0125] The communication module can establish a connection and communicate with other devices such as servers, user-side electronic devices, etc. through the vehicle to everything (V2X) communication technology (cellular V2X, C-V2X) based on the cellular network. C-V2X may include, for example, V2X (LTE-V2X) based on long term evolution (LTE), 5G-V2X, etc.

[0126] The screen can be used to provide a visual interface. The vehicle 100 may include one or more screens, for example, an instrument screen arranged in front of the driver's seat, a screen arranged above the seat for displaying the surrounding conditions, a HUD that projects information onto the windshield, and the like. The screens in the vehicle 100 may also include: a central control screen, a co-pilot screen, a rear screen, an armrest screen, a rear curtain, and the like. Among them, the rear seat area of ​​the vehicle 100 may be configured with one or more rear screens. For example, the area of ​​the left rear seat is configured with a screen. The area of ​​the right rear seat is configured with another screen. This application does not limit the number and position of the screens included in the vehicle 100.

[0127] T-box 14 may also be referred to as a vehicle system, a telematics processor, a vehicle gateway, etc., which is not limited in the embodiments of the present application.

[0128] The transmission ECU 123 is used to manage the transmission.

[0129] The transmission 15 is a mechanism that can be used to change the speed and torque of the engine. It can change the transmission ratio of the output shaft and the input shaft in a fixed or step-by-step manner. The components of the transmission 15 may include a speed change transmission mechanism, an operating mechanism, and a power output mechanism. The main function of the speed change transmission mechanism is to change the value and direction of the torque and speed; the main function of the operating mechanism is to control the transmission mechanism to achieve the change of the transmission ratio of the transmission, that is, to achieve gear shifting, so as to achieve speed and torque change.

[0130] The drive recorder ECU 124 is used to manage the drive recorder 16 .

[0131] The components of the driving recorder 16 may include a host, a vehicle speed sensor, data analysis software, etc. The driving recorder 16 refers to an instrument that records images and sounds of the vehicle during driving, including relevant information such as driving time, speed, and location. In the embodiment of the present application, when the vehicle is driving, the vehicle speed sensor collects the wheel speed and sends the vehicle speed information to the driving recorder 16 via the CAN bus.

[0132] The ABS ECU 125 is used to manage the ABS 17 .

[0133] ABS17 automatically controls the braking force when the vehicle is braking, so that the wheels are not locked and are in a rolling and sliding state, so as to ensure that the adhesion between the wheels and the ground is at the maximum. During the braking process, when the electronic control device determines that a wheel is tending to lock based on the wheel speed signal input by the wheel speed sensor, the ABS enters the anti-lock brake pressure adjustment process.

[0134] The sensor system 18 may include: an acceleration sensor, a vehicle speed sensor, a vibration sensor, a gyroscope sensor, a radar sensor, a signal transmitter, a signal receiver, and the like. The acceleration sensor and the vehicle speed sensor are used to detect the speed of the vehicle 100. The vibration sensor may be set under the seat, the seat belt, the seat back, the operation panel, the airbag, or other locations to detect whether the vehicle 100 has been hit and the location of the user. The gyroscope sensor may be used to determine the motion posture of the vehicle 100. The radar sensor may include a laser radar, an ultrasonic radar, a millimeter wave radar, and the like. The radar sensor is used to emit electromagnetic waves to illuminate the target and receive its echo, thereby obtaining information such as the distance from the target to the electromagnetic wave emission point, the rate of change of distance (radial velocity), the azimuth, and the height, so as to identify other vehicles, pedestrians, or roadblocks near the vehicle 100. The signal transmitter and the signal receiver are used to send and receive signals, which can be used to detect the location of the user. The signal may be, for example, ultrasonic waves, millimeter waves, lasers, and the like.

[0135] The camera system 19 may include multiple cameras, which are used to capture static images or videos. The cameras in the camera system 19 can be set in front of the vehicle, behind the vehicle, on the side, inside the vehicle, etc., to facilitate the realization of functions such as assisted driving, driving recording, panoramic view, and in-vehicle monitoring.

[0136] The sensor system 18 and the camera system 19 can be used to detect the surrounding environment so that the vehicle 100 can make corresponding decisions to cope with environmental changes. For example, they can be used to complete the task of paying attention to the surrounding environment during the automatic driving stage.

[0137] The microphone 20, also called a "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or outputting a voice command, the user can make a sound by approaching the microphone 20 with his mouth to input the sound signal into the microphone 20. The vehicle 100 can be provided with at least one microphone 20. In other embodiments, the vehicle 100 can be provided with two microphones 20, which can not only collect sound signals but also realize a noise reduction function. In other embodiments, the vehicle 100 can also be provided with three, four or more microphones 20 to form a microphone array to collect sound signals, reduce noise, identify the sound source, realize a directional recording function, etc.

[0138] In addition, the vehicle 100 may also include multiple interfaces, such as a USB interface, an RS-232 interface, an RS485 interface, etc., which can be connected to external electronic devices such as cameras, microphones, and headphones.

[0139] In the embodiment of the present application, the microphone 20 can be used to detect the voice command input by the user. The sensor system 18, the camera system 19, the T-box 14, etc. can be used to recognize the user's voice command. When the voice command is recognized, the T-box ECU 122 can execute the above voice command.

[0140] The speaker 21, which may also be referred to as a "speaker" or a "horn", may be used to convert electrical signals into sound signals. The speaker 21 may be used for audio playback. The vehicle 100 may be provided with at least one speaker 21. For example, the vehicle 100 may include a plurality of in-vehicle speakers. These plurality of in-vehicle speakers may be distributed at different locations in the vehicle so that users sitting at various locations in the vehicle may have a good audio listening experience. In some embodiments, the vehicle 100 may also include an external speaker. In some embodiments, the vehicle 100 may also be connected to an external sound device (such as a Bluetooth headset, etc.) and play audio through the external sound device.

[0141] In some embodiments, the memory in the vehicle 100 may be used to store the binding relationship between the vehicle and the user.

[0142] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the vehicle system. The vehicle 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0143] For example, the vehicle 100 may also include a separate memory, a battery, lights, wipers, an instrument panel, a transmission control unit (TCU), an auxiliary control unit (ACU), a passive entry passive start (PEPS), an on-board unit (OBU), a body control module (BCM), a charging port, and the like.

[0144] In the present application, the memory of the vehicle 100 may be used to store a configuration file of the driving zone. The configuration file may be used to determine the binding relationship between the screen of the vehicle 100 and the driving zone.

[0145] Figure 3 A schematic diagram of a cockpit audio partition architecture provided by the present application is exemplarily shown.

[0146] like Figure 3 As shown, the audio system part of the cockpit system can be logically divided into four levels: cockpit screen, driving area, sound area, and sound equipment.

[0147] The cockpit screen may include screens in the vehicle 100. For example, screen 0, screen 1, screen 2, screen 3, etc. Screen 0 may be a central control screen. Screen 2 may be an instrument screen. Screen 3 may be a co-pilot screen. Screen 4 may be a rear screen.

[0148] The driving area and the sound zone can refer to the introduction of the above-mentioned embodiment. For example, the driving area can include driving area 0, driving area 1, driving area 2, and so on. Driving area 0 can be bound to the central control screen and the instrument screen (i.e., screen 0 and screen 1). Driving area 1 can be bound to the co-pilot screen (i.e., screen 2). Driving area 2 can be bound to the rear screen (i.e., screen 3). The sound zone can include sound zone 0, sound zone 1, sound zone 2, and so on.

[0149] Driving zone 0 may be associated with audio zone 0. Driving zone 1 may be associated with audio zone 1. Driving zone 2 may be associated with audio zone 2. The embodiment of the present application does not limit the association relationship between the driving zone and the audio zone.

[0150] The sound device may include a sound device built into the vehicle 100 and an external sound device. For example, sound device 0, sound device 1, sound device 2, etc. Sound device 0 may be a speaker in the vehicle 100. Sound device 1 may be an external earphone 1. Sound device 2 may be an external earphone 2.

[0151] The above-mentioned sound zone 1 may be created when the sound device 1 is connected. Therefore, the sound device 1 may be associated with the sound zone 1. The above-mentioned sound zone 2 may be created when the sound device 2 is connected. Therefore, the sound device 2 may be associated with the sound zone 2.

[0152] In some embodiments, the sound device 1 is connected through the screen 2. The cockpit system can associate the sound zone 1 associated with the sound device 1 with the driving zone 1 bound to the screen 2. The sound device 2 is connected through the screen 3, and the cockpit system can associate the sound zone 2 associated with the sound device 2 with the driving zone 2 bound to the screen 3.

[0153] When the application in screen 0 needs to play audio, the cockpit system can complete the application's audio focus application through driving zone 0 and audio zone 0, and determine that the application's audio should be routed to the audio channel of sound device 0. In this way, the application in screen 0 can play audio through sound device 0. Similarly, when the application in screen 2 needs to play audio, the cockpit system can complete the application's audio focus application through driving zone 1 and audio zone 1, and determine that the application's audio should be routed to the audio channel of sound device 1. In this way, the application in screen 2 can play audio through sound device 1.

[0154] In the above cockpit audio partition architecture, the cockpit screen and the driving zone can be bound through a configuration file. The association between the driving zone and the audio zone, and the association between the audio zone and the sound device can be dynamically determined based on the connection or removal of the sound device.

[0155] The above-mentioned cockpit audio partition architecture can support flexible configuration of the driving area to which the screen belongs and the audio routing to the sound-emitting device, support dynamic creation and deletion of audio zones, and also support seamless switching of audio between sound-emitting devices in screen migration and screen sharing scenarios without the need for multimedia playback applications to adapt.

[0156] Figure 4 An exemplary schematic diagram of the architecture of a cockpit system provided by the present application is shown.

[0157] like Figure 4 As shown, the cockpit system may include an application layer, a driving area management layer, an audio framework layer (audioframework), a hardware abstraction layer (HAL), and a sound device.

[0158] 1. Application layer

[0159] The application layer may include one or more APPs. For example, multimedia playback applications, audio centers, cockpit voice assistants, Bluetooth, desktop services, and the like. Multimedia playback applications may be APPs used to play multimedia content (such as audio and / or video). Multimedia playback applications may include, but are not limited to: music APPs, phone APPs, video APPs, audio book APPs, radio APPs, navigation APPs, and the like. Multimedia playback applications may belong to third-party applications. Audio centers, cockpit voice assistants, Bluetooth, and desktop services may belong to system applications. The difference between third-party applications and system applications lies in whether they can globally manage and control other applications. Third-party applications cannot globally manage and control other applications. System applications can globally manage and control other applications.

[0160] In some embodiments, the above system applications are configured in multiple screens in the cockpit system. The system application in one screen can globally manage and control other applications in the same screen. The third-party applications in different screens may be different.

[0161] For example, the central control screen, co-pilot screen, rear screen and other screens can all include an audio center, cockpit voice assistant, Bluetooth, and desktop services.

[0162] The desktop service can be used to display the desktop. The desktop may include application icons for applications on the screen, controls for adjusting the volume, controls for adjusting the screen brightness, etc. It can be understood that the central control screen, the co-pilot screen, the rear screen, etc. can all display the desktop through the desktop service to facilitate user interaction with the screen.

[0163] Bluetooth can be used to connect Bluetooth devices, such as Bluetooth headsets. Bluetooth can provide a user interface for Bluetooth connection to facilitate users to select Bluetooth devices and complete the connection and removal of Bluetooth devices. It is understandable that if a screen receives a user operation to connect a Bluetooth headset, the Bluetooth headset can be bound to this screen after it is connected to the cockpit system. The central control screen, co-pilot screen, rear screen and other screens can all provide Bluetooth connection services via Bluetooth. In this way, the central control screen, co-pilot screen, rear screen and other screens can all be bound to Bluetooth headsets.

[0164] The cockpit voice assistant can be used to interact with the user by voice, recognize and execute the user's voice commands. For example, the user can use the cockpit voice assistant to open a music app to play music, or open a navigation app to navigate, etc. In some embodiments, the cockpit voice assistant can display the text content of its voice interaction with the user on the screen.

[0165] The audio center can be used to display the multimedia playlist in the screen. The multimedia playlist can include information about the multimedia content currently being played on the screen, and / or information about the multimedia content that has been played. In some embodiments, the multimedia playlist can also include controls for controlling the playback of multimedia content. In this way, the user can control the multimedia content in a screen through the audio center in the screen (such as pausing playback, switching playback content, etc.). The central control screen, co-pilot screen, rear screen and other screens can all display the multimedia playlists in their respective screens through the audio center.

[0166] In some embodiments, the audio center of a screen can obtain the media session of the application in the screen, and then determine the content in the multimedia playlist according to the media session. The audio center can also be called a media center or the like.

[0167] In some embodiments, the audio center of a screen may also obtain a media session in a media session stack associated with the driving zone where the screen is located, and then determine the content in the multimedia playlist according to the media session.

[0168] The cockpit system may also include more or fewer applications, which is not limited in the embodiments of the present application.

[0169] 2. Driving and Passenger Area Management

[0170] The driving area management layer may include: Media Session Management API (MediaSessionManager API), Media Session Service (MediaSessionService), Audio Management API (AudioManager API), Audio Service (AudioService), Cockpit Audio Management API (CarAudioManager API), and Cockpit Audio Service (CarAudioService). Among them, the Media Session Management API, Media Session Service, Audio Management API, and Audio Service may be native modules in the existing cockpit system (such as a cockpit system based on AOSP). The cockpit audio management API and the cockpit audio service may be modules added by this application to the cockpit system for unified management of audio focus, volume, audio routing, and media sessions. Among them, both the media session service and the audio service can communicate with the cockpit audio service.

[0171] In the cockpit system of the present application, the multimedia playback application may not need to be adapted and modified, and may still manage the media session through the media session management API and call the audio service through the audio management API to manage one or more of the audio focus, volume, and audio routing. When the media session service receives information about the multimedia playback application performing media session management, the media session service may instruct the cockpit audio service to manage the media session of the multimedia playback application based on the information. For example, the above-mentioned media session management information may include, but is not limited to: information about creating a media session, information about deleting a media session, information about the multimedia content being played, and so on. When the audio service receives information about the multimedia playback application performing audio management, the audio service may instruct the cockpit audio service to manage the audio focus, and / or volume, and / or audio routing in the multimedia playback application based on the information. The above-mentioned audio management information may include, but is not limited to: information about applying for audio focus, information about abandoning audio focus, information about adjusting volume, information about adjusting audio routing, and so on.

[0172] System applications in each screen (such as audio center, cockpit voice assistant, Bluetooth, desktop services, etc.) can call cockpit audio services through the cockpit audio management API to achieve unified management of audio focus, volume, and audio routing.

[0173] The cockpit audio service may include an in-vehicle audio feature part, an APP and screen association module, a screen and driving area association module, a driving area audio management module, and a cockpit media session management module.

[0174] The in-vehicle audio feature part may include audio-related logic of screen sharing features and audio-related logic of screen migration features. Screen sharing features may refer to sharing multimedia content played on one screen to one or more other screens. For example, the co-pilot screen is playing video A of a video APP. In response to the operation of sharing video A to the central control screen and the rear screen, the cockpit system may also play video A on the central control screen and the rear screen. In this way, the co-pilot screen, the central control screen and the rear screen all play video A. Among them, the audio-related logic of the screen sharing feature can be used to indicate the logic of audio flowing between sound-emitting devices in the screen sharing scenario. For example, the audio-related logic of the screen sharing feature may include but is not limited to: when the multimedia content played on one screen is shared to all screens bound to the driving area, the audio in the multimedia content is migrated to the in-car speakers and played by the in-car speakers. The above-mentioned audio-related logic of the screen sharing feature is only an exemplary description of the present application and should not be construed as a limitation on the present application.

[0175] The screen migration feature may refer to migrating multimedia content played on one screen to another screen. For example, the central control screen is playing video A of a video APP. In response to the operation of migrating video A to the co-pilot screen, the cockpit system may turn off video A in the central control screen and play video A in the co-pilot screen. In this way, the playback window of video A is migrated from the central control screen to the co-pilot screen. Among them, the audio-related logic of the screen migration feature can be used to indicate the logic of the flow of audio between sound devices in the screen migration scenario. For example, the audio-related logic of the screen migration feature may include but is not limited to: when the multimedia content played on one screen is migrated to another screen, the audio in the multimedia content is migrated from the sound device corresponding to the audio associated with the driving and riding area bound to this screen to the sound device corresponding to the audio associated with the driving and riding area bound to this other screen. The above-mentioned audio-related logic of the screen migration feature is only an exemplary description of the present application and should not be construed as a limitation on the present application.

[0176] The APP and screen association module includes a screen APP list. The screen APP list can record the information of the APP configured in each screen in the cockpit system. The APP configured in each screen in the cockpit system can be independent. Each screen can install APP independently. In response to an operation on a screen for installing an APP, the cockpit system can configure this APP in this screen after downloading and installing this APP. The application icon of this APP can be displayed on the desktop of this screen so that the user can use this APP. This APP can be called the APP installed on this screen. The desktop of other screens that do not have this APP installed may not contain the application icon of this APP. The APP and screen association module can manage the above-mentioned screen APP list according to the changes of the APP in the screen. The cockpit audio service determines which APPs are included in each screen based on the screen APP list.

[0177] The screen and driving area association module may include a configuration file of the driving area. The configuration file of the driving area may record the binding relationship between the screen and the driving area. The cockpit audio service may read the configuration file, create the driving area according to the configuration file, and bind the screen to the driving area.

[0178] The driving zone audio management module can be used to manage the audio of the driving zone corresponding application. The driving zone audio management module can obtain the driving zone configuration file from the above-mentioned screen and driving zone association module to determine the number of driving zones in the cockpit system and the binding relationship between the driving zone and the screen. The driving zone audio management module can create a driving zone and bind the screen to the driving zone.

[0179] In some embodiments, the passenger zone audio management module may include a passenger zone audio management module corresponding to each passenger zone. The passenger zone audio management module corresponding to a passenger zone may be used to manage the audio of the application corresponding to the passenger zone.

[0180] Exemplarily, the driving area audio management module may include a driving area 0 audio management module and a driving area 1 audio management module. The driving area 0 audio management module can be used to manage the audio of the corresponding application of the driving area 0. The driving area 1 audio management module can be used to manage the audio of the corresponding application of the driving area 1. For example, the driving area 0 may be the driving area bound to the central control screen, the instrument screen, and the HUD. The driving area 1 may be the driving area bound to the co-pilot screen. Not limited to the driving area audio management modules corresponding to the driving area 0 and the driving area 1, the driving area audio management module may also include more or fewer driving area audio management modules corresponding to the driving area.

[0181] The audio management module of the riding area 0 includes: a historical connection device list (historicalDeviceList), an application list (applicationList), active sound zone information (activeZone), and active device information (activeDevice). The historical connection device list in the audio management module of the riding area 0 may include information about the sound device that the screen in the riding area 0 has been bound to. The audio management module of the riding area 0 may update the historical connection device list therein according to the changes in the sound device bound to the screen in the riding area 0. The information of the sound device in the historical connection device list may include, but is not limited to: identification information of the sound device, volume configuration information of the sound device, etc. The volume configuration information of the sound device can be used to indicate the volume set in the sound device. The application list in the audio management module of the riding area 0 may include information about applications running in the foreground or background in the screen bound to the riding area 0. The applications in the application list in the audio management module of the riding area 0 may include applications installed by the screen in the riding area 0 and running on the screen in the riding area 0, as well as applications installed by screens in other riding areas and transferred to the screen in the riding area 0 to run. The applications in the application list in the audio management module of the riding area 0 may be the applications corresponding to the riding area 0. The audio management module of the riding area 0 may update the application list therein according to the changes of the screen running applications in the riding area 0. In some embodiments, the audio management module of the riding area 0 may obtain the activity information of the application from the activity manager service (AMS) to determine the screen running application in the riding area 0. Among them, activity is an application component that can provide an interactive window for the user to perform an operation, such as playing audio, taking pictures, making calls, etc. The active sound zone information in the audio management module of the riding area 0 may be used to indicate the sound zone currently associated with the riding area 0. The audio management module of the riding area 0 may update the active sound zone information therein according to the changes of the associated sound zone of the riding area 0. The active device information of the riding area 0 may be used to indicate the sound device currently bound to the screen in the riding area 0. The riding area 0 may update the active sound zone information therein according to the changes of the screen bound sound device in the riding area 0.

[0182] The audio management module of the driving zone 1 also includes: a list of historically connected devices, an application list, active audio zone information, and active device information. For details, please refer to the above introduction to the audio management module of the driving zone 0. I will not repeat it here.

[0183] The cabin media session management module can be used to manage the media sessions of the applications corresponding to each driving zone. The cabin media session management module can synchronize the information of the applications corresponding to the driving zone from the above-mentioned driving zone audio management module, and then determine which media sessions are included in each driving zone. In some embodiments, the cabin media session management module may include a media session management module corresponding to each driving zone. The media session management module corresponding to a driving zone can be used to manage the media sessions in this driving zone (i.e., the media sessions of the applications corresponding to this driving zone).

[0184] Exemplarily, the cockpit media session management module may include a driving zone 0 media session management module and a driving zone 1 media session management module. The driving zone 0 media session management module may be used to manage media sessions in the driving zone 0. The driving zone 1 media session management module may be used to manage media sessions in the driving zone 1. The cockpit media session management module is not limited to the media session management modules corresponding to the driving zone 0 and the driving zone 1, and the cockpit media session management module may include media session management modules corresponding to more or less driving zones.

[0185] The riding zone 0 media session management module may include an active media session stack. The active media session stack in the riding zone 0 media session management module may be used to indicate in which media session stack the media session of the corresponding application of the riding zone 0 is currently located. The media session stack in which the media session of the corresponding application of the riding zone 0 is located may change with the change of the audio zone associated with the riding zone 0.

[0186] The driving zone 1 media session management module may also include an active media session stack. For details, please refer to the above introduction to the driving zone 0 media session management module. No further details will be given here.

[0187] For example, if the audio zones associated with driving zone 1 and driving zone 0 are the same, the media session of the application corresponding to driving zone 1 and the media session of the application corresponding to driving zone 0 may be in the same media session stack.

[0188] The media session stack in which a media session of an application corresponding to a riding zone is located can be called a media session stack associated with this riding zone. In some embodiments, a media session stack associated with a riding zone may include not only the media session of the application corresponding to this riding zone, but also the media sessions of applications corresponding to other riding zones. For example, when riding zone 1 and riding zone 0 are associated with the same audio zone, the above-mentioned media session stack 0 includes both the media session of the application corresponding to riding zone 0 and the media session of the application corresponding to riding zone 1.

[0189] In some embodiments, the riding area management layer may further include a riding area video management module. The riding area video management module may be used to manage the video playback content of the riding area corresponding application. Among them, similar to the above-mentioned riding area audio management module, the riding area video management module may include video management modules corresponding to each riding area. For example, riding area 0 video management module, riding area 1 video management module, and so on. Riding area 0 video management module may be used to manage the video playback content corresponding to riding area 0. Riding area 1 video management module may be used to manage the video playback content corresponding to riding area 1. In the scenario of screen migration or screen sharing, the above-mentioned riding area video management module may be responsible for transferring the video playback interface of the application from one screen to other screens.

[0190] 3. Audio framework layer

[0191] The audio framework layer may include: media session management instantiation API (MediaSessionManagerEx API), media session service instantiation (MediaSessionServiceEx), audio zone management API, audio zone management module (AudioZoneService), audio policy management module (AudioPolicyManager), and audio channel management module (AudioFlinger).

[0192] The audio zone management API can be used by various modules and services in the driving zone audio management module layer to call the audio zone management module to manage the audio zone.

[0193] The audio zone management module can be used to manage the audio zone, for example, to create a new audio zone and the audio zone management module corresponding to the new audio zone, to delete the audio zone and the audio zone management module corresponding to the audio zone, and to manage and control the audio focus, audio routing, and volume in the audio zone. In some embodiments, the audio zone management module may include an audio zone management module corresponding to each audio zone. The audio zone management module corresponding to a audio zone can be used to manage the audio zone. Among the audio zone management modules, there is at least one audio zone management module corresponding to the audio zone: the audio zone management module corresponding to the audio zone associated with the in-vehicle speaker.

[0194] Exemplarily, the audio zone management module may include an audio zone 0 management module and an audio zone 1 management module. The audio zone 0 management module may be used to manage the audio focus, audio routing, and volume in the audio zone 0. The audio zone 1 management module may be used to manage the audio focus, audio routing, and volume in the audio zone 1. For example, the audio zone 0 may be associated with the driving zone 0, and the associated sound device may be an in-vehicle speaker. The audio zone 1 may be associated with the driving zone 1, and the associated sound device may be an external headset 1. The audio zone management module is not limited to the audio zone management modules corresponding to the audio zone 0 and the audio zone 1, and the audio zone management module may also include audio zone management modules corresponding to more or fewer audio zones.

[0195] The audio zone 0 management module may include: an audio routing module, a focus management module, and a volume management module. The audio routing module in the audio zone 0 management module may be used to manage the audio routing in the audio zone 0. The audio routing module may determine the audio routing strategy for the audio of the application corresponding to the driving zone associated with the audio zone 0. Among them, the audio of the application corresponding to the driving zone associated with the audio zone 0 may be routed to the audio channel corresponding to the audio zone 0. The focus management module in the audio zone 0 management module may be used to manage the audio focus of the audio zone 0. The application corresponding to the driving zone associated with the audio zone 0 may compete for the audio focus in the audio zone 0. The audio zone 0 management module may include an audio focus stack, and update the audio focus stack according to the information that the application corresponding to the driving zone associated with the audio zone 0 applies for the audio focus. The volume management module in the audio zone 0 management module may be used to manage the volume of the audio zone 0. The audio zone 0 management module may adjust the volume of the sound device associated with the audio zone 0 according to the instruction to adjust the volume.

[0196] The audio zone 1 management module may also include an audio routing module, a focus management module, and a volume management module. For details, please refer to the above introduction to the audio zone 0 management module. No further details will be given here.

[0197] The media session management instantiation API can be used by each module and service in the cockpit audio management module layer to call the media session service instantiation. For example, the cockpit media session management module can call the media session service instantiation through the media session management instantiation API to create a media session stack, delete a media session stack, and update a media session stack.

[0198] A media session service instantiation may include one or more media session stacks. The number of media session stacks may be consistent with the number of audio zones.

[0199] Exemplarily, the riding zone 0 media session management module may call the media session service instantiation to create the media session stack 0. The media session stack 0 may include the media session of the application corresponding to the riding zone 0. In the case where the audio zone associated with the riding zone 1 is different from the riding zone 0, the riding zone 1 media session management module may call the media session service instantiation to create the media session stack 1. The media session stack 1 may include the media session of the application corresponding to the riding zone 1. In some embodiments, when the audio zone associated with the riding zone 1 changes to the audio zone associated with the riding zone 0, the riding zone 1 media session management module may call the media session service instantiation to migrate the media session in the media session stack 1 to the media session stack 0, and delete the media session stack 0.

[0200] Not limited to media session stack 0 and media session stack 1, the media session service instantiation may also include more or less media session stacks.

[0201] The audio policy management module can be used to manage audio routing policies. The audio routing policy management module can create an audio routing table, delete a routing table, and update an audio routing table. Each audio zone can be associated with an audio routing table. An audio routing table associated with an audio zone can record the audio routing policy of the audio corresponding to the application associated with the driving zone of this audio zone. The audio routing table can be called AudioPolicyMix.

[0202] Exemplarily, the audio zone 0 management module may instruct the audio policy management module to create audio routing table 0. The audio zone 1 management module may instruct the audio policy management module to create audio routing table 1. Among them, audio routing table 0 may include information about applications corresponding to audio zone 0 and the driving zone. All audios applied in audio routing table 0 can be routed to the audio channel corresponding to audio zone 0, that is, played by the sound device associated with audio zone 0. Similarly, audio routing table 1 may include information about applications corresponding to audio zone 1 and the driving zone. All audios applied in audio routing table 1 can be routed to the audio channel corresponding to audio zone 1, that is, played by the sound device associated with audio zone 1.

[0203] When the driving zone associated with a sound zone changes, the sound zone management module corresponding to the sound zone can instruct the audio policy management module to update the audio routing table associated with the sound zone.

[0204] In some embodiments, the audio policy management module may also include a Bluetooth protocol module. The Bluetooth protocol module may be used to indicate the parameters and processes required for the Bluetooth-based audio channel to transmit the audio channel. For example, the Bluetooth protocol module may include the Bluetooth audio transmission model agreement (Advanced Audio Distribution Profile, A2DP). A2DP specifies the protocol stack software and usage methods for transmitting high-quality audio file data using Bluetooth asynchronous transmission channels. Based on A2DP, high-quality audio can be transmitted through Bluetooth sound, so that the audio in the cockpit system can be listened to using Bluetooth headphones.

[0205] The audio channel management module can be used to manage the audio channels in the cockpit system. The audio channel management module may include audio channels for the in-car speakers. There may be multiple audio channels for the in-car speakers, for example, audio channel 0 (Mix0), audio channel 1 (Mix1), audio channel 2 (Mix2), and so on. Among them, there may be multiple in-car speakers in the cockpit, and different types of audio may be processed differently to produce different playback sound effects. For example, the types of audio may include media audio, call audio, alarm audio, and so on. Different types of audio can be transmitted and played through different audio channels of the in-car speakers.

[0206] In some embodiments, when a sound device is connected to the cockpit system, the audio channel management module can create an audio channel for the sound device. When the external sound device is removed, the audio channel management module can delete the audio channel for the sound device. For example, when a Bluetooth headset 1 is connected to the cockpit system, the audio channel management module can create Bluetooth audio channel 1 (Bluetooth Mix 1) for the Bluetooth headset 1. When the Bluetooth headset 1 is disconnected from the cockpit system, the audio channel management module can delete Bluetooth audio channel 1.

[0207] In some embodiments, the driving zone audio management module can assign the audio channel of the in-car speaker to audio zone 0. In this way, the audio zone 0 management module can determine that the audio channel corresponding to audio zone 0 is the audio channel of the in-car speaker. In the case where the above-mentioned Bluetooth headset 1 is connected to the cockpit system through the screen bound to the driving zone 1, the driving zone audio management module can assign Bluetooth audio channel 1 to audio zone 1. In this way, the audio zone 1 management module can determine that the audio channel corresponding to audio zone 1 is Bluetooth audio channel 1.

[0208] Not limited to the audio channels of the in-car speakers and the Bluetooth audio channel 1, the audio channel management module may also include more or fewer audio channels.

[0209] 4. Hardware Abstraction Layer and Sound Devices

[0210] The hardware abstraction layer may be an interface layer between the cockpit system kernel and the hardware circuit, and may be used to abstract the hardware. The hardware abstraction layer may include an in-car speaker hardware abstraction module and a Bluetooth hardware abstraction module. Among them, the in-car speaker abstraction module may be an interface between the cockpit system and the in-car speaker. The cockpit system may control the in-car speaker to play audio through the in-car speaker hardware abstraction module. The Bluetooth hardware abstraction module may be an interface between the cockpit system and a Bluetooth headset. The cockpit system may control the Bluetooth headset connected to the cockpit system to play audio through the Bluetooth hardware abstraction module.

[0211] The sound device of the cockpit may include an in-vehicle speaker and a Bluetooth headset connected to the cockpit system. The present application does not limit the sound device of the cockpit. For example, the sound device of the cockpit may also include an external speaker configured outside the vehicle 100, a wired headset connected to the cockpit system, and the like.

[0212] Above Figure 4 The cockpit system shown is only an exemplary description of the present application. Figure 4 The cockpit system shown in FIG. 1 may further include the cockpit system of the vehicle 100. Figure 4 More or fewer modules, or combining some software modules, or splitting some software modules, or arranging software modules in different locations.

[0213] In some embodiments, the vehicle 100 may include multiple processor chips and multiple cockpit systems. Different audio zones may be in different processor chips and cockpit systems. For example, the central control screen and the co-pilot screen are controlled by a processor chip and a cockpit system. The rear screen is controlled by another processor chip and a cockpit system. The driving area where the central control screen and the co-pilot screen are located and the audio zones associated with the driving area can all be in the cockpit system that controls the central control screen. The driving area where the rear screen is located and the audio zones associated with the driving area can all be in the cockpit system that controls the rear screen. Among them, the cockpit audio services in different cockpit systems can communicate with each other. In scenarios involving audio streaming, such as screen migration and screen sharing, the cockpit audio services in different cockpit systems can communicate with each other to migrate audio focus information, audio routing, and media sessions, thereby achieving seamless switching of audio between sound devices connected to different cockpit systems.

[0214] In order to facilitate understanding of the multi-sound zone management solution of the present application, the subsequent embodiments of the present application are specifically described by taking the vehicle 100 configured with a cockpit system as an example.

[0215] Connect external sound devices to dynamically create sound zones

[0216] In some embodiments, when the cockpit system has no external sound device, the cockpit system may include a sound zone associated with the in-car speakers. All driving and passenger zones in the cockpit system can be associated with the sound zone associated with the in-car speakers. When the cockpit system is connected to an external sound device, the cockpit system can create a new sound zone and associate the driving and passenger zone where the screen bound to the external sound device is located with the newly created sound zone. In this way, the sound zone corresponding to the application in the driving and passenger zone can be dynamically adjusted, so that the audio of applications in different screens can be separated and played to different sound devices.

[0217] Figure 5 A schematic diagram of a cockpit audio partition architecture provided by the present application when no external sound device is connected is exemplified.

[0218] like Figure 5 As shown, the cockpit system includes driving area 0, driving area 1, and driving area 2. The screens in driving area 0 include a central control screen, an instrument screen, and a HUD. The screens in driving area 1 include a co-pilot screen, and the screens in driving area 2 include a rear screen. The applications corresponding to driving area 0 can be determined based on the applications running in the foreground or background in the central control screen, the instrument screen, and the HUD. For example, the applications corresponding to driving area 0 may include application 1, application 2, application 3, and so on. The applications corresponding to driving area 1 can be determined based on the applications running in the foreground or background in the co-pilot screen. For example, the applications corresponding to driving area 1 may include application 4, application 5, and so on. The applications corresponding to driving area 2 can be determined based on the applications running in the foreground or background in the rear screen. For example, the applications corresponding to driving area 2 may include application 6, and so on.

[0219] In the case that the cockpit system does not have an external sound device, the above-mentioned central control screen, instrument screen, HUD, co-pilot screen, and rear screen can all be bound to the in-car speakers. Therefore, driving zone 0 to driving zone 2 can all be associated with the whole vehicle sound zone. The whole vehicle sound zone is the sound zone associated with the in-car speakers.

[0220] Depend on Figure 5 It can be seen that all applications corresponding to driving zone 0 to driving zone 2 are set in the whole vehicle sound zone and are played through the in-car speakers. Among them, the applications corresponding to driving zone 0 to driving zone 2 all compete for audio focus in the whole vehicle sound zone. Applications that compete for audio focus in the whole vehicle sound zone can use the in-car speakers to play audio. For example, when application 1 on the central control screen holds the audio focus, the in-car speakers can obtain the audio playback data of application 1, and play the audio of application 1 according to the audio playback data of application 1. When application 4 on the co-pilot screen seizes the audio focus, the in-car speakers can pause the audio of application 1, obtain the audio playback data of application 4, and play the audio of application 4. Furthermore, when application 6 on the rear screen seizes the audio focus of application 4, the in-car speakers can pause the audio of application 4, obtain the audio playback data of application 6, and play the audio of application 6.

[0221] In addition, the media sessions of the applications corresponding to riding zone 0 to riding zone 2 can all be placed in a media session stack. The audio centers of the screens in riding zone 0 to riding zone 2 can obtain the information of the media sessions in this media session stack. The audio centers of these screens can display the multimedia playlist based on the information of the media session. In this way, any screen in riding zone 1 to riding zone 2 can obtain the information of the media sessions of the applications corresponding to all riding zones. The multimedia playlists displayed on the screens with audio centers in riding zone 1 to riding zone 2 can be the same.

[0222] In some embodiments, the multimedia playlist may include controls for controlling the playback of multimedia content. Since the multimedia playlist displayed in the audio center of all screens is the same, the user can view and control the multimedia content played on other screens (such as the co-pilot screen and the rear screen) on the central control screen, view and control the multimedia content played on other screens (such as the central control screen and the rear screen) on the co-pilot screen, and view and control the multimedia content played on other screens (such as the central control screen and the co-pilot screen) on the rear screen.

[0223] It can be seen from the above embodiments that when the cockpit system has no external sound device, all screens in the cockpit share audio resources. Only one application on all screens can obtain audio focus in the same time period, so that the audio can be played using the in-car speakers. The co-pilot screen and the rear screen can be equivalent to the extended screen of the central control screen.

[0224] Figure 6 The schematic diagram of the cockpit audio partition architecture provided by the present application when external earphones 1 and 2 are connected is exemplarily shown.

[0225] like Figure 6 As shown, the cockpit system includes driving zone 0, driving zone 1, and driving zone 2. The screens in driving zone 0 to driving zone 2 and the corresponding applications can refer to the aforementioned Figure 5 Introduction of embodiments.

[0226] In some embodiments, the headset 1 is connected to the cockpit system through the co-pilot screen. The co-pilot screen receives a user operation for connecting the headset 1. After the headset 1 is connected to the cockpit system, the cockpit system can bind the co-pilot screen to the headset 1. Since the co-pilot screen is a screen in the driving area 1, the cockpit system can create a headset audio zone 1, associate the headset audio zone 1 with the driving area 1, and assign the audio channel of the headset 1 to the headset audio zone 1. That is, the sound device associated with the audio zone 1 is the headset 1.

[0227] Headphones 2 are connected to the cockpit system through the rear screen. The rear screen receives a user operation for connecting headphones 2. After headphones 2 are connected to the cockpit system, the cockpit system can bind the rear screen to headphones 2. Since the rear screen is a screen in the driving zone 2, the cockpit system can create a headphone audio zone 2, associate the headphone audio zone 2 with the driving zone 2, and assign the audio channel of headphone 2 to the headphone audio zone 2. That is, the sound device associated with audio zone 2 is headphone 2.

[0228] The sound zone associated with driving zone 0 is still the whole vehicle sound zone. The whole vehicle sound zone is the sound zone associated with the speakers in the car.

[0229] Depend on Figure 6It can be seen that the applications corresponding to driving zone 0 (such as application 1, application 2, application 3, etc.) are set in the whole vehicle sound zone, and audio can be played through the in-car speakers. Applications corresponding to driving zone 1 (such as application 4, application 5, etc.) are set in headphone sound zone 1, and the headphones play audio through headphone 1. Applications corresponding to driving zone 2 (such as application 6, etc.) are set in headphone sound zone 2, and audio can be played through headphone 2. The applications corresponding to driving zone 0 to driving zone 2 can compete for audio focus in the sound zone associated with each driving zone, and use the sound device associated with the sound zone to play audio after competing for audio focus in the sound zone. For example, when application 1 on the central control screen holds the audio focus in the whole vehicle sound zone, the in-car speakers can obtain the audio playback data of application 1 and play the audio of application 1. When application 4 on the co-pilot screen holds the audio focus in the headphone sound zone 1, headphone 1 can obtain the audio playback data of application 4 and play the audio of application 4. The audio played by the in-car speakers and the audio played by headphone 1 can be independent of each other and do not interfere with each other. In this way, the driver can listen to the audio of application 1 through the in-car speaker and the user sitting in the front passenger seat can listen to the audio of application 4 through earphone 1 without affecting each other. Furthermore, when application 2 on the central control screen seizes the audio focus of application 1 in the whole vehicle sound zone, the in-car speaker can pause playing the audio of application 1, obtain the audio playback data of application 2, and play the audio of application 2. The above changes in the audio played by the in-car speaker do not affect the audio played by earphone 1. Earphone 1 can still play the audio of application 4.

[0230] The volume of the above-mentioned in-car speakers, earphones 1, and earphones 2 can also be controlled independently. For example, in response to an operation to adjust the volume on the central control screen, the cockpit system can adjust the volume of the in-car speakers. In response to an operation to adjust the volume on the co-pilot screen, the cockpit system can adjust the volume of earphones 1. In response to an operation to adjust the volume on the rear screen, the cockpit system can adjust the volume of earphones 2.

[0231] In addition, the media sessions of the applications corresponding to driving zones 0 to 2 may be placed in different media session stacks, respectively. The media session stack associated with driving zone 0 may only include the media sessions of the applications corresponding to driving zone 0. The media session stack associated with driving zone 1 may only include the media sessions of the applications corresponding to driving zone 1. The media session stack associated with driving zone 2 may only include the media sessions of the applications corresponding to driving zone 2. The multimedia playlists displayed by the audio center of the screens in different driving zones may be different according to the media session stacks associated with the corresponding driving zones. For example, the multimedia playlist displayed by the audio center of the central control screen includes the multimedia content information of the applications corresponding to driving zone 0, but does not include the multimedia content information of the applications corresponding to driving zone 1 and driving zone 2. The multimedia playlist displayed by the audio center of the co-pilot screen includes the multimedia content information of the applications corresponding to driving zone 1, but does not include the multimedia content information of the applications corresponding to driving zone 0 and driving zone 2. The user can view and control the multimedia content of the applications corresponding to each driving zone on the screens in each driving zone.

[0232] It can be seen from the above embodiments that the sound zone in the cockpit system can change with the dynamic changes of the external sound device of the cockpit system. As the sound zone associated with the driving zone changes, the application corresponding to the driving zone can also be dynamically set to the sound zone associated with the driving zone after the sound zone changes. Users can separate the audio played by different applications by connecting external sound devices in the cockpit system. In this way, different users can connect different sound devices to the screens in different driving zones, and then use different sound devices to listen to the audio they want to listen to without affecting each other.

[0233] The following describes a method for creating a sound zone when an external sound device is connected, as provided in an embodiment of the application.

[0234] Figure 7 The flowchart of a method for creating a sound zone provided by the present application is exemplified.

[0235] Here, the cockpit system is connected to the headset 1 through the co-pilot screen as an example. The headset 1 can be a Bluetooth headset.

[0236] like Figure 7 As shown, the method may include steps S711 to S713.

[0237] S711. The co-pilot screen receives a user operation for connecting earphone 1.

[0238] The user operation of connecting the headset 1 may include selecting the headset 1 on the co-pilot screen to establish a Bluetooth connection. The embodiment of the present application does not limit the user operation of connecting the headset 1.

[0239] S712: After earphone 1 is connected to the cockpit system, the cockpit system creates Bluetooth audio channel 1.

[0240] The headset 1 being connected to the cockpit system may indicate that the headset 1 and the cockpit system have completed Bluetooth pairing connection. The embodiment of the present application does not specifically limit the method of Bluetooth pairing connection.

[0241] The Bluetooth audio channel 1 may be used to transmit the audio stream to be played to the earphone 1, so that the earphone 1 plays the audio. The Bluetooth audio channel 1 may be an audio channel associated with the earphone 1.

[0242] S713. After the cockpit audio service in the cockpit system senses that the audio channel of earphone 1 is successfully created, earphone 1 is bound to the co-pilot screen, audio zone 1 is created, Bluetooth audio channel 1 is assigned to audio zone 1, and audio zone 1 is associated with driving zone 1.

[0243] For cockpit audio services, please refer to the above Figure 4 Introduction to the embodiment. Since the headset 1 is connected to the cockpit system through the co-pilot screen, the cockpit audio service can bind the headset 1 to the co-pilot screen. The cockpit audio service can call the audio zone management module to create audio zone 1, assign Bluetooth audio channel 1 to audio zone 1, and associate audio zone 1 with driving zone 1 where the co-pilot screen is located. In this way, the application corresponding to driving zone 1 associated with audio zone 1 can be routed to the audio channel corresponding to audio zone 1.

[0244] In some embodiments, if the headset 1 has been connected to the cockpit system through the co-pilot screen, the headset 1 can actively establish a Bluetooth connection with the cockpit system when it is connected to the cockpit system again, without the user having to perform a user operation to connect the headset 1. After the headset 1 is connected to the cockpit system again, the cockpit system can still bind the headset 1 to the co-pilot screen. In other words, if the headset 1 first accesses the cockpit system through the co-pilot screen, the subsequent active access of the headset 1 to the cockpit system can be assumed to be still accessed through the co-pilot screen.

[0245] Figure 8 The schematic diagram of the external sound device of the cockpit system is shown as an example.

[0246] Here, the cockpit system is described as follows: the cockpit system is connected to earphone 1 through the co-pilot screen, and earphone 2 is connected through the rear screen. Both earphone 1 and earphone 2 can be Bluetooth earphones.

[0247] like Figure 8As shown, when earphone 1 is connected to the cockpit system, the Bluetooth protocol module in the audio policy management module can search for a Bluetooth audio protocol that is compatible with earphone 1 and can transmit audio data to earphone 1. The Bluetooth protocol module can open output port 1 based on the found Bluetooth audio protocol. Output port 1 can be used to send data such as parameters and processes for establishing and transmitting audio streams to the audio channel management module. The audio policy management module can instruct the audio channel management module to create an audio channel. The audio channel management module can call the Bluetooth hardware abstraction module to open the port of Bluetooth audio channel 1 to complete the creation of Bluetooth audio channel 1. In this way, Bluetooth audio channel 1 can lead to earphone 1 through the Bluetooth hardware abstraction module.

[0248] Similarly, when earphone 2 is connected to the cockpit system, the Bluetooth protocol module can search for a Bluetooth audio protocol that is compatible with earphone 2 and can transmit audio data to earphone 2. The Bluetooth protocol module can open output port 2 based on the found Bluetooth audio protocol, and instruct the audio channel management module to create an audio channel through output port 2. The audio channel management module can call the Bluetooth hardware abstract module to open the port of Bluetooth audio channel 2 to complete the creation of Bluetooth audio channel 2. In this way, Bluetooth audio channel 2 can lead to earphone 2 through the Bluetooth hardware abstract module.

[0249] After earphone 1 is bound to the co-pilot screen and the Bluetooth audio channel 1 of earphone 1 is created, the audio management module of the driving zone 1 corresponding to the co-pilot screen can instruct the audio zone management module to create an audio zone 1 management module corresponding to audio zone 1, and assign Bluetooth audio channel 1 to audio zone 1. Then, the audio zone 1 management module can instruct the audio policy management module to create an audio routing table 1, and store the information of the application corresponding to the driving zone 1 and the information of the Bluetooth audio channel 1 corresponding to audio zone 1 in the audio routing table 1 according to the application corresponding to the driving zone 1. Since Bluetooth audio channel 1 is assigned to audio zone 1, audio routing table 1 can point to Bluetooth audio channel 1. This can mean that the audio of the application recorded in audio routing table 1 can be routed to Bluetooth application channel 1 and played by earphone 1.

[0250] Similarly, after earphone 2 is bound to the co-pilot screen and the Bluetooth audio channel 2 of earphone 2 is created, the audio management module of driving zone 2 corresponding to the co-pilot screen can instruct the audio zone management module to create an audio zone 2 management module corresponding to audio zone 2, and assign Bluetooth audio channel 2 to audio zone 2. Then, the audio zone 2 management module can instruct the audio policy management module to create an audio routing table 2, and store the information of the application corresponding to driving zone 2 and the information of Bluetooth audio channel 2 corresponding to audio zone 2 in the audio routing table 2 according to the application corresponding to driving zone 2. Since Bluetooth audio channel 2 is assigned to audio zone 2, audio routing table 2 can point to Bluetooth audio channel 2. This can mean that the audio of the application recorded in audio routing table 2 can be routed to Bluetooth application channel 2 and played by earphone 2.

[0251] The process of connecting other sound-generating devices to the cockpit system can refer to the process of connecting earphones 1 and 2 as described above.

[0252] Fig. 9 The flowchart of the method for creating an audio channel is exemplarily shown.

[0253] like Fig. 9 As shown, the method may include steps S911 to S920.

[0254] S911. The Bluetooth APP in the co-pilot screen can notify the audio service headset 1 to connect.

[0255] In some embodiments, the headset 1 can complete the Bluetooth pairing connection through the Bluetooth APP in the co-pilot screen and access the cockpit system. When the headset 1 is connected to the cockpit system, the Bluetooth APP in the co-pilot screen can send a message that the headset 1 is connected to the audio service.

[0256] The Bluetooth APP in the co-pilot screen can display a Bluetooth setting interface. The Bluetooth setting interface can display a prompt message that the headset 1 has been connected to the cockpit system.

[0257] S912: The audio service may send a message to the audio policy management module indicating that earphone 1 is connected.

[0258] S913: The audio policy management module may search for an available Bluetooth audio protocol and open the Bluetooth output port 1 through the Bluetooth audio protocol.

[0259] S914. The audio policy management module sends an instruction to create a Bluetooth audio channel to the audio channel management module.

[0260] S915. The audio channel management module sends an instruction to open the Bluetooth audio channel port to the Bluetooth hardware abstraction module.

[0261] S916. The Bluetooth hardware abstraction module opens the port of Bluetooth audio channel 1.

[0262] S917. The audio channel management module completes creation of Bluetooth audio channel 1.

[0263] Steps S912 to S917 may refer to the aforementioned Figure 8 Introduction of embodiments.

[0264] S918. The audio channel management module sends a message to the audio service indicating that the creation of Bluetooth audio channel 1 is complete.

[0265] S919. The audio service sends a message to the cockpit audio service indicating that creation of the audio channel of earphone 1 is complete.

[0266] The message sent by the audio service to the cockpit audio service may include: information about the audio channel of earphone 1 and information about the screen used by earphone 1 to access the cockpit system. The above information about the audio channel of earphone 1 can be used to indicate that the audio channel of earphone 1 is a Bluetooth audio channel. The above information about the screen used by earphone 1 to access the cockpit system can be used to indicate that earphone 1 accesses the cockpit system through the co-pilot screen.

[0267] S920. The cockpit audio service binds earphone 1 to the co-pilot screen, calls the audio zone management module to create audio zone 1, assigns Bluetooth audio channel 1 to audio zone 1, and associates audio zone 1 with driving zone 1.

[0268] Based on the message sent by the audio service in step S919, the cockpit audio service can bind earphone 1 to the co-pilot screen, call the audio zone management module to create audio zone 1, assign Bluetooth audio channel 1 to audio zone 1, and associate audio zone 1 with driving zone 1.

[0269] The specific implementation of step S920 can refer to the following Fig.10 and Fig.11 Introduction.

[0270] Fig. 9 The method for creating an audio channel for a sound device when an external sound device is connected to the cockpit system is merely an exemplary description of the present application and should not constitute a limitation to the present application.

[0271] Fig.10 and Fig.11 The schematic diagram exemplarily shows the specific process of the cockpit system creating a sound zone.

[0272] like Fig.10 As shown, the driving zone 0 audio management module is the driving zone audio management module corresponding to the central control screen, which can be used to manage the audio of the corresponding application of the driving zone 0. The driving zone 0 is bound to the central control screen. The driving zone 1 audio management module is the driving zone audio management module corresponding to the co-pilot screen, which can be used to manage the audio of the corresponding application of the driving zone 1. The driving zone 1 is bound to the co-pilot screen. The sound device bound to the central control screen can be an in-car speaker. The sound zone associated with the in-car speaker can be sound zone 0. Therefore, the driving zone 0 is associated with sound zone 0.

[0273] Since the sound device bound to the central control screen is the in-car speaker, the active device information in the driving zone 0 audio management module may include the in-car speaker information. Since the driving zone 0 is associated with the audio zone 0, the active audio zone information in the driving zone 0 audio management module may include the audio zone 0 information, and the active audio zone information may point to the audio zone 0 management module to indicate that the driving zone 0 is associated with the audio zone 0. The audio zone 0 management module may point to the audio routing table 0, which may indicate that the audio routing policy of the corresponding application of the driving zone 0 is stored in the audio routing table 0. The audio routing table 0 points to the audio channel of the in-car speaker, which may indicate that the audio of the corresponding application of the driving zone 0 is played by the in-car speaker. In addition, the active media session stack in the driving zone 0 media session management module points to the media session stack 0, which may indicate that the media session of the corresponding application of the driving zone 0 can be stored in the media session stack 0.

[0274] By the aforementioned Figure 5 It can be seen that when the co-pilot screen is not bound to an external sound device, the co-pilot screen is bound to the in-car speakers, and the driving zone 1 is also associated with the audio zone 0. At this time, the driving zone 1 is also associated with the audio zone 0. The active device information in the audio management module of the driving zone 1 may include the information of the in-car speakers. The active audio zone information in the audio management module of the driving zone 1 may include the information of the audio zone 0, and the active audio zone information may point to the audio zone 0 management module to indicate that the driving zone 1 is associated with the audio zone 0. The active media session stack in the media session management module of the driving zone 1 points to the media session stack 0, which may indicate that the media session of the corresponding application of the driving zone 1 can be stored in the media session stack 0.

[0275] Among them, the media sessions of the corresponding applications of driving area 0 and driving area 1 are both stored in media session stack 0, and the active media session stacks in the driving area 0 media session management module and the driving area 1 media session management module both point to media session stack 0. Therefore, the multimedia playlists displayed in the audio center of the central control screen and the co-pilot screen can be the same, both containing the multimedia content of the corresponding applications of driving area 0 and driving area 1.

[0276] When headset 1 is connected to the cockpit system through the co-pilot screen, the cockpit system can Fig.11 The method shown creates sound zone 1, and adjusts the sound zone associated with driving zone 1 to associate driving zone 1 with sound zone 1. Fig.11 As shown:

[0277] S1111. When earphone 1 is connected to the cockpit system and bound to the co-pilot screen, the driving zone 1 audio management module calls the audio zone management module to create audio zone 1 and generates an audio zone 1 management module.

[0278] After earphone 1 is connected to the cockpit system, the audio channel management module can create Bluetooth audio channel 1 for earphone 1. Since the sound device bound to the co-pilot screen is switched from the in-car speaker to earphone 1, the active device information in the audio management module of the driving area 1 can be updated to the information of earphone 1. For example, the above-mentioned information of earphone 1 may include the identification of earphone 1 and the identification of Bluetooth audio channel 1. According to the active device information, the audio management module of the driving area 1 can assign Bluetooth audio channel 1 to audio zone 1. In some embodiments, the method of assigning Bluetooth audio channel 1 to audio zone 1 may include: the audio management module of the driving area 1 sends the identification of Bluetooth audio channel 1 to the audio zone 1 management module.

[0279] S1112. Create an audio routing table 1 associated with audio zone 1, configure information of Bluetooth audio channel 1 in audio routing table 1, and configure application information of applications corresponding to driving zone 1.

[0280] The audio management module of driving zone 1 can send the information of applications (i.e., application list) corresponding to driving zone 1 to the audio zone 1 management module. The audio routing module of the audio zone 1 management module can call the audio policy management module to create an audio routing table 1 associated with audio zone 1. The audio routing module of the audio zone 1 management module can configure the information of Bluetooth audio channel 1 (such as the identifier of Bluetooth audio channel 1) from the audio management module of driving zone 1 and the application information of applications corresponding to driving zone 1 in the audio routing table 1. This can mean that the audio of the applications included in the audio routing table 1 are all routed to the Bluetooth audio channel 1 and played through the earphone 1.

[0281] S1113 , filtering out audio focus information requested by the application corresponding to driving zone 1 from the audio focus stack associated with audio zone 0 , and migrating the filtered audio focus information to the audio focus stack associated with audio zone 1 .

[0282] Before audio zone 1 is created, the audio zone associated with driving zone 1 is audio zone 0. The audio focus information requested by the application corresponding to driving zone 1 is stored in the audio focus stack associated with audio zone 0, and is managed by the focus management module in the audio zone 0 management module. When the audio zone associated with driving zone 1 needs to be switched from audio zone 0 to audio zone 1, the focus management module in the audio zone 0 management module can filter out the audio focus information of the application corresponding to driving zone 1 from the audio focus stack associated with audio zone 0, and send the filtered focus information to the focus management module in the audio zone 1 management module. The focus management module in the audio zone 1 management module can store the audio focus information requested by the application corresponding to driving zone 1 in the audio focus stack associated with audio zone 1. Among them, the focus management module in the audio zone 0 management module can delete the audio focus information requested by the application corresponding to driving zone 1 from the audio focus stack associated with audio zone 0.

[0283] In some embodiments, in the scenario where the audio focus information requested by the application corresponding to the driving zone 1 is migrated from the audio focus stack associated with the audio zone 0 to the audio focus stack associated with the audio zone 1, if the audio focus information at the top of the audio focus stack associated with the audio zone 0 is requested by the application corresponding to the driving zone 1, the audio focus information still retains its state of holding the audio focus after the migration. That is to say, an application corresponding to the driving zone 1 holds the audio focus in the audio zone 0 and is at the top of the audio focus stack associated with the audio zone 0. After the audio focus information is migrated, the application corresponding to the driving zone 1 still holds the audio focus, and the audio focus it holds changes from the audio focus in the audio 0 to the audio focus in the audio zone 1. This application is at the top of the audio focus stack associated with the audio zone 1. Exemplarily, when the driving zone 0 and the driving zone 1 are both associated with the audio zone 0, the music APP in the co-pilot screen applies for the audio focus in the audio zone 0 and plays the music through the in-car speakers associated with the audio zone 0. During the process of playing the above-mentioned music APP by the in-car speakers, if earphone 1 is connected to the cockpit system through the co-pilot screen, the music of the music APP can be switched to be played by earphone 1. Among them, the in-car speakers can stop playing the music of the music APP. Earphone 1 can continue playing the music of the music APP from the position where the in-car speakers stopped playing. The seamless switching of the above-mentioned audio between sound devices can be achieved with the dynamic changes of the sound zones in the cockpit system (such as creation or deletion), without the need for multimedia playback application perception and adaptation.

[0284] S1114 . The volume management module in the audio zone 1 management module sets the volume data of earphone 1 .

[0285] In some embodiments, earphone 1 is a sound-emitting device that has been connected to the cockpit system through the co-pilot screen. The list of historically connected devices in the audio management module of the driving area 1 may include information about earphone 1. The information about earphone 1 in the list of historically connected devices may include, but is not limited to: the identification of earphone 1, the volume of earphone 1 when it was last connected to the cockpit system, and so on. The audio management module of the driving area 1 may send the volume of the above-mentioned earphone 1 when it was last connected to the cockpit system to the audio zone 1 management module. The volume management module in the audio zone 1 management module may set the volume data of earphone 1 according to the volume of earphone 1 when it was last connected to the cockpit system. In this way, the volume of the audio played when earphone 1 is connected again can be kept consistent with the volume of the audio played when it was last connected to the cockpit system, reducing the operation of the user to readjust the volume to the user's personal accustomed volume, thereby improving the user's audio listening experience.

[0286] In some embodiments, the volume management module in the audio zone 1 management module may also set the volume data of the earphone 1 to the default volume data. The embodiment of the present application does not limit the method for setting the volume data of the earphone 1.

[0287] S1115 . Update the active audio zone information in the audio management module of driving zone 1 to driving zone 1 associated audio zone 1 .

[0288] The active audio zone information in the driving zone 1 audio management module can be changed from pointing to the audio zone 0 management module to Fig.10 The figure shows the direction to the audio zone 1 management module. This may indicate that the audio zone associated with the driving zone 1 is changed to the audio zone 1.

[0289] S1116: Adjust the audio playback route of the application corresponding to the driving area 1, and adjust the sound device for playing the audio of the application corresponding to the driving area 1 from the in-vehicle speaker to the earphone 1.

[0290] In some embodiments, when the audio zone associated with driving zone 1 is switched from audio zone 0 to audio zone 1, the audio zone 0 management module may instruct the audio policy management module to delete the information of the application corresponding to driving zone 1 in audio routing table 0. Fig.10 As shown, audio routing table 0 can still point to the audio channel of the in-car speaker. Audio routing table 1 created with the creation of audio zone 1 can point to Bluetooth audio channel 1. This means that the audio of the application recorded in audio routing table 1 (i.e., the application corresponding to driving zone 1) can be routed to Bluetooth application channel 1 and played by earphone 1.

[0291] S1117 . Create media session stack 1 , filter out media sessions of applications corresponding to driving zone 1 from media session stack 0 , and migrate the filtered media sessions to media session stack 1 .

[0292] In some embodiments, the riding zone 1 audio management module may synchronize the audio zone change associated with the riding zone 1 to the riding zone 1 media session management module. When it is determined that the audio zone associated with the riding zone 1 changes from audio zone 0 to audio zone 1, the riding zone 1 media session management module may call the media session service instantiation to create a media session stack 1, and migrate the media session of the application corresponding to the riding zone 1 in the media session stack 0 to the media session stack 1. The media session service instantiation may delete the media session of the application corresponding to the riding zone 1 in the media session stack 0.

[0293] S1118: Adjust the active media session stack in the media session management module of driving area 1 from pointing to media session stack 0 to pointing to media session stack 1.

[0294] like Fig.10 As shown, the active media session stack in the media session management module of driving zone 1 points to media session stack 1, which may indicate that the media session of the application corresponding to driving zone 1 may be stored in media session stack 1.

[0295] S1119. The driving zone 0 media session management module and the driving zone 1 media session management module report the change of the media session stack to one or more applications in the audio center, Bluetooth, and desktop service.

[0296] The driving area 0 media session management module can report changes in the media session stack 0 to system applications such as the audio center, Bluetooth, and desktop services on the screen bound to the driving area 0 (such as the central control screen).

[0297] The media session stack of driving area 1 can report changes of the media session stack associated with driving area 1 to system applications such as the audio center, Bluetooth, and desktop services on the screen bound to driving area 1 (such as the co-pilot screen).

[0298] S1120: The audio management module of driving zone 1 reports completion of audio zone switching associated with driving zone 1 to one or more applications in the audio center, Bluetooth, and desktop service.

[0299] The audio management module of driving zone 1 can report the completion of audio zone switching associated with driving zone 1 to system applications such as the audio center, Bluetooth, and desktop services on the screen bound to the driving zone.

[0300] In some embodiments, system applications such as the audio center, Bluetooth, and desktop services may provide corresponding feedback to the user based on changes in the media session stack and changes in the audio zones associated with the driving zone, so that the user can better use each audio zone to listen to audio.

[0301] Exemplarily, when the audio center of the central control screen receives the message that the media session stack 0 has changed, the audio center of the central control screen can update the multimedia playlist according to the changed media session stack 0. Since the changed media session stack 0 does not include the media session of the application corresponding to the driving area 1, the multimedia playlist displayed by the audio center of the central control screen does not include the multimedia content of the application corresponding to the driving area 1.

[0302] When the audio center of the co-pilot screen receives the message that the media session stack associated with the driving zone 1 changes from media session stack 0 to media session stack 1, the audio center of the co-pilot screen can update the multimedia playlist according to media session stack 1. Since media session stack 1 does not include the media session of the application corresponding to the driving zone 0, the multimedia playlist displayed in the audio center of the co-pilot screen does not include the multimedia content of the application corresponding to the driving zone 0.

[0303] In some embodiments, a sound zone may be associated with multiple sound-emitting devices. Multiple sound-emitting devices associated with a sound zone may share the same audio channel and play the same audio. For example, the central control screen and the co-pilot screen are in the same driving area. The central control screen may be bound to an in-car speaker. The co-pilot screen may be bound to earphone 1. The in-car speakers and earphone 1 are associated with the same sound zone. The audio used in the central control screen may be played in the in-car speakers and earphone 1 at the same time. The audio used in the co-pilot screen may also be played in the in-car speakers and earphone 1 at the same time.

[0304] Fig.11The order of the steps in the method shown does not limit the order in which the steps are executed. For example, the above steps S1112, S1113, S1114, and S1117 may be performed simultaneously. That is, when the audio zone associated with driving zone 1 changes from audio zone 0 to audio zone 1, the cockpit system may migrate the audio focus, audio routing, volume, and media session simultaneously. For another example, the above step S1120 may be performed before step S1119, or step S1120 and step S1119 may be performed simultaneously.

[0305] In some embodiments, the vehicle 100 includes a first screen and a second screen. The first screen and the second screen may correspond to different driving zones. For example, the first screen may be a central control screen. The second screen may be a co-pilot screen. The first screen may correspond to driving zone 0. The second screen may correspond to driving zone 1. In the case where the cockpit system does not have an external sound device through the second screen, the first screen and the second screen may both be associated with the first sound zone corresponding to the in-vehicle speaker. The first sound zone may be sound zone 0 in the present application. The fact that both the first screen and the second screen are associated with the first sound zone may mean that the driving zone 0 corresponding to the first screen is associated with the first sound zone, and the driving zone 1 corresponding to the second screen is also associated with the first sound zone.

[0306] In the case where both the first screen and the second screen are associated with the first audio zone, if the cockpit system detects that the first application corresponding to the second screen requests to play the first audio, the cockpit system can play the first audio through the in-car speakers. Then, when it is detected that the second application in the first screen requests to play the second audio, the cockpit system can play the second audio through the in-car speakers and pause the first audio or reduce the playback volume of the first audio. The above-mentioned first application and second application can both be multimedia playback applications. It can be seen that the application in the second screen and the application in the first screen both apply for audio focus in the first audio zone. When the first application in the second screen holds the audio focus in the first audio zone, the second application in the first screen can seize the audio focus of the first application.

[0307] When an input operation acting on the second screen to access the first sound device is detected, the cockpit system can connect to the first sound device and create a second sound zone corresponding to the first sound device. For example, the first sound device can be earphone 1. The second sound zone can be sound zone 1 of the present application. The cockpit system can change the sound zone associated with the second screen from the above-mentioned first sound zone to the second sound zone. In the case where the above-mentioned in-car speaker plays the second audio and reduces the playback volume of the first audio to play the first audio at a lower volume, the in-car speaker can stop playing the first audio after the sound zone associated with the second screen changes. When the cockpit system detects again that the first application requests to play the first audio, the cockpit system can play the first audio through the first sound device.

[0308] The above-mentioned first sound zone may correspond to a first audio focus stack. The first audio focus stack may include audio focus information of applications running in the foreground or background on the screen associated with the first sound zone. That is, the first audio focus stack may include audio focus information of applications corresponding to the driving zone (such as driving zone 0, driving zone) associated with the first sound zone. When creating a second sound zone, the cockpit system can create a second audio focus stack corresponding to the second sound zone, and migrate the audio focus information of applications running in the foreground or background on the second screen from the first audio focus stack to the second audio focus stack. Among them, in addition to the audio focus information of applications running in the foreground or background on the second screen, the cockpit system can also migrate the audio focus information of applications running in the foreground or background on other screens in the same driving zone as the second screen from the first audio focus stack to the second audio focus stack. For example, the second screen corresponds to driving zone 1. When creating a second sound zone, the cockpit system can migrate the audio focus information of applications corresponding to driving zone 1 from the first audio focus stack to the second audio focus stack.

[0309] The first audio zone may correspond to a first audio routing table. The first audio routing table includes application information of applications running in the foreground or background on the screen associated with the first audio zone, and the first audio routing table is configured with a first audio channel of the in-vehicle speaker, which is used to indicate that the audio of the application corresponding to the application information in the first audio routing table is routed to the first audio channel. For example, the first audio routing table may be the audio routing table 0 in this application. When creating a second audio zone, the cockpit system may create a second audio routing table corresponding to the second audio zone, and migrate the application information of the application running in the foreground or background on the second screen from the first audio routing table to the second audio routing table. Among them, in addition to the application information of the application running in the foreground or background on the second screen, the cockpit system may also migrate the application information of the application running in the foreground or background on other screens in the same driving zone as the second screen from the first audio routing table to the second audio routing table. For example, the second screen corresponds to driving zone 1. When creating a second audio zone, the cockpit system may migrate the application information of the application corresponding to driving zone 1 from the first audio routing table to the second audio routing table. The second audio routing table may be the audio routing table 1 of this application. A second audio channel may be configured in the second audio routing table. The second audio channel may be Bluetooth audio channel 1. That is, the second audio routing table may point to Bluetooth audio channel 1.

[0310] The above-mentioned first audio zone may correspond to a first media session stack. The first media session stack may be the media session stack 0 of the present application. When creating a second audio zone, the cockpit system may create a second media session stack corresponding to the second audio zone. The second media session stack may be the media session stack 1 of the present application. The cockpit system may migrate the media session of the application corresponding to the second audio zone associated with the driving zone (such as the driving zone) from the first media session stack to the second media session stack. For details, please refer to the above Fig.10The introduction of migrating a media session from media session stack 0 to media session stack 1.

[0311] Move the sound device out and delete the sound zone dynamically

[0312] In some embodiments, when an external sound device is disconnected from the cockpit system, the cockpit system can delete the audio zone associated with the sound device, and re-associate the driving zone associated with the deleted audio zone with the audio zone associated with the in-vehicle speakers. In this way, when the external sound device is removed, the cockpit system can also dynamically adjust the audio zone corresponding to the driving zone, thereby achieving seamless switching of audio between sound devices.

[0313] Fig.12 The flowchart of the method for deleting an audio channel when a sound-emitting device is removed provided by the present application is exemplified.

[0314] like Fig.12 As shown, the method may include steps S1211 to S1222.

[0315] S1211. The Bluetooth APP in the co-pilot screen can notify the audio service that earphone 1 is disconnected.

[0316] In some embodiments, the Bluetooth APP in the co-pilot screen can display a Bluetooth setting interface. The Bluetooth setting interface can display a prompt message that the headset 1 has been disconnected from the cockpit system.

[0317] S1212: The audio service may indicate to the cockpit audio service that earphone 1 is disconnected.

[0318] S1213. The cockpit audio service searches for the audio zone associated with earphone 1, and pauses the audio currently being played in the audio zone associated with earphone 1.

[0319] In some embodiments, when the above-mentioned message indicating that earphone 1 is disconnected is received, the driving zone 1 management in the cockpit audio service can search for the audio zone associated with earphone 1. In a possible implementation, the driving zone 1 audio management module can determine that earphone 1 is bound to the co-pilot screen in the driving zone 1 according to the active device information, and then determine that the audio zone associated with earphone 1 is audio zone 1 according to the active audio zone information. The driving zone 1 audio management module can instruct the driving zone 1 media session management module to control the media session of the application to which the audio being played in audio zone 1 belongs, so that earphone 1 pauses playing audio.

[0320] Step S1213 is optional. It is understandable that if the headset 1 is not playing audio before disconnection, the cockpit audio service may not need to perform step S1213.

[0321] S1214: The audio service may send a message to the audio policy management module to instruct the deletion of the audio channel of earphone 1.

[0322] S1215: The audio policy management module may close the Bluetooth output port 1.

[0323] Combined with the above Figure 8 It can be seen that the Bluetooth protocol module in the audio policy management module can close the output port 1.

[0324] S1216: The audio policy management module may send a message to the audio channel management module instructing to delete the Bluetooth audio channel 1.

[0325] S1217: The audio channel management module may send an instruction to the Bluetooth hardware abstraction module to close the port of Bluetooth channel 1.

[0326] S1218. The Bluetooth hardware abstraction module closes the port of Bluetooth audio channel 1.

[0327] S1219. The audio channel management module completes deleting Bluetooth audio channel 1.

[0328] S1220: The audio channel management module may send a message to the audio service indicating that the deletion of the Bluetooth audio channel 1 is complete.

[0329] S1221: The audio service may indicate to the cockpit audio service a message that the audio channel deletion of earphone 1 is complete.

[0330] S1222. The cockpit audio service may call the audio zone management module to delete audio zone 1, bind the co-pilot screen to the in-car speakers, and associate the driving zone 1 with the audio zone 0 of the in-car speakers.

[0331] The specific implementation of step S1222 can refer to the following Fig.13 and Fig.14 Introduction.

[0332] The above method of deleting the audio channel of the sound device when the sound device external to the cockpit system is removed is merely an exemplary description of the present application and should not constitute a limitation to the present application.

[0333] Fig.13 and Fig.14 The specific process diagram of deleting a sound zone in the cockpit system is exemplified.

[0334] When the cockpit system is connected to the headset 1 and the headset 1 is bound to the co-pilot screen, the relationship between the driving zone 1 and the audio zone 1 is as mentioned above. Fig.10 When the headset 1 is disconnected from the cockpit system, the cockpit system can refer to Fig.14 The method shown in the figure deletes the audio zone 1, adjusts the audio zone associated with the driving zone 1, and re-associates the driving zone 1 with the audio zone 0. Fig.14 As shown:

[0335] S1411. When earphone 1 bound to the co-pilot screen is disconnected, the audio management module of driving area 1 instructs the audio policy management module to delete audio routing table 1 and configure application information of the application corresponding to driving area 1 in audio routing table 0.

[0336] By the aforementioned Fig.12 It can be seen from the step S1212 shown that when the earphone 1 is disconnected, the cockpit audio service can receive a message from the audio service indicating that the earphone 1 is disconnected. Based on the message that the earphone 1 is disconnected, the cockpit audio service can determine that the earphone 1 is bound to the screen in the driving area 1 according to the active device information in the driving area audio management module. Therefore, the driving area 1 audio management module instructs the audio policy management module to delete the audio routing table 1, and configures the application information of the application corresponding to the driving area 1 in the audio routing table 0. Specifically, the driving area 1 audio management module can instruct the audio zone 1 management module to migrate the audio routing policy of the application corresponding to the driving area 1. The audio zone 1 management module can call the audio policy management module to migrate the application information of the application recorded in the audio routing table 1 to the audio routing table 0, and delete the audio routing table 1. Among them, in the present application, migrating the audio routing policy of an application from one audio routing table to another audio routing table can include migrating the application information (such as the application identifier) ​​of this application in this audio routing table to another audio routing table.

[0337] Since the earphone 1 is disconnected, the screen in the driving area 1 needs to be re-bound to the in-car speaker, so the active device information in the audio management module of the driving area 1 can be updated to the in-car speaker information. For example, the in-car speaker information can include the in-car speaker identifier and the in-car speaker audio channel identifier.

[0338] S1412: Migrate the audio focus information in the audio focus stack associated with audio zone 1 to the audio focus stack associated with audio zone 0, and send an audio focus preemption event to the applicant of the migrated audio focus information.

[0339] The sound device bound to the screen in driving zone 0 is changed to the in-car speaker, and the audio zone associated with driving zone 0 also needs to be changed to audio zone 0 associated with the in-car speaker. The driving zone 1 audio management module can instruct the audio zone 1 management module to migrate the audio focus information in the audio focus stack associated with audio zone 1 to the audio focus stack associated with audio zone 0. The audio focus information in the above-mentioned audio focus stack associated with audio zone 1 is the audio focus information applied for by the corresponding application of driving zone 1. Among them, the focus management module in the audio zone 1 management module can send the audio focus information in the audio focus stack associated with audio zone 1 to the audio zone 0 management module. The audio zone 0 management module can save the audio focus information in the above-mentioned audio focus stack associated with audio zone 1 to the audio focus stack associated with audio zone 0.

[0340] In some embodiments, in the scenario where the audio focus information requested by the application corresponding to the driving zone 1 is migrated from the audio focus stack associated with the audio zone 1 to the audio focus stack associated with the audio zone 0, if the application holding the audio focus in the audio zone 1 is playing audio, and the application holding the audio focus in the audio zone 0 is also playing audio, then after the audio focus information is migrated, the focus management module in the audio zone 0 management module can send an audio focus preemption event to the application holding the audio focus in the audio zone 1. Therefore, after the audio focus information requested by the application corresponding to the driving zone 1 is migrated to the audio focus stack associated with the audio zone 0, the application that originally held the audio focus in the audio zone 0 can continue to play the audio, while the application that originally held the audio focus in the audio zone 1 loses the audio focus and stops playing the audio.

[0341] S1413. The driving zone 1 audio management module calls the audio zone management module to delete the audio zone 1, and deletes the audio zone 1 management module.

[0342] S1414: Update the active audio zone information in the audio management module of driving zone 1 to audio zone 0 associated with driving zone 1.

[0343] The active audio zone information in the driving zone 1 audio management module can be obtained from the aforementioned Fig.10 The direction shown is the audio zone 1 management module, which changes to Fig.13 The figure shows the direction to the audio zone 0 management module. This may indicate that the audio zone associated with the driving zone 1 is changed to the audio zone 0.

[0344] S1415: Adjust the audio playback route of the application corresponding to the driving area 1, and adjust the sound device for playing the audio of the application corresponding to the driving area 1 from the earphone 1 to the in-vehicle speaker.

[0345] S1416: Migrate the media session in media session stack 1 to media session stack 0, and delete media session stack 1.

[0346] In some embodiments, the riding zone 1 audio management module may synchronize the audio zone change associated with the riding zone 1 to the riding zone 1 media session management module. When it is determined that the audio zone associated with the riding zone 1 changes from audio zone 1 to audio zone 0, the riding zone 1 media session management module may call the media session service instantiation to migrate the media session in the media session stack 1 to the media session stack 0, and delete the media session stack 1.

[0347] S1417: Adjust the active media session stack in the media session management module of driving area 1 from pointing to media session stack 1 to pointing to media session stack 0.

[0348] like Fig.13As shown, the active media session stack in the media session management module of driving zone 1 points to media session stack 0, which may indicate that the media session of the corresponding application of driving zone 1 may be stored in media session stack 0.

[0349] S1418. The driving zone 0 media session management module and the driving zone 1 media session management module report the change of the media session stack to one or more applications in the audio center, Bluetooth, and desktop service.

[0350] S1419. The audio management module of driving zone 1 reports to one or more applications in the audio center, Bluetooth, and desktop service that the switching of the audio zone associated with driving zone 1 is completed.

[0351] Step S1418 and step S1419 can refer to the aforementioned Fig.11 Steps S1119 and S1120 are shown.

[0352] The present application embodiment is for the above Fig.14 The execution order of the steps in the method shown is not limited.

[0353] like Fig.13 As shown, the execution of media session stack 0 by media session stack 1 may indicate that the media session in media session stack 1 is migrated to media session stack 0. The focus management module in audio zone 1 management module points to the focus management module in audio zone 0 management module, which may indicate that the audio focus information in the audio focus stack associated with audio zone 0 is migrated to the audio focus stack associated with audio zone 1. Media session stack 1, audio zone 1 management module, audio routing table 1 and Bluetooth audio channel 1 are represented by dotted lines, which may indicate that these modules are deleted from the cockpit system when earphone 1 is disconnected.

[0354] Figures 15A to 15D A schematic diagram of an audio playback scenario in which some sound-emitting devices are removed is exemplified.

[0355] like Fig.15A As shown, the central control screen is bound to the in-car speakers. The central control screen can display the navigation interface of the navigation application. The in-car speakers can play the navigation prompt sound of the navigation application. The co-pilot screen is bound to earphone 1, and the co-pilot screen can display the audio playback interface 1510 of the music APP. The audio playback interface 1510 may include a pause control 1511. The pause control 1511 can be used to pause the playback of music, such as music A. Earphone 1 can play music A of the music APP. It can be seen that when different screens are each bound to a sound device, the applications in different screens can play audio through the sound devices bound to the screens. In this way, users using different screens can listen to the audio they want to listen to without interfering with each other.

[0356] like Fig. 15BAs shown, when earphone 1 is disconnected, the cockpit system can bind the co-pilot screen to the central control screen. Earphone 1 can stop playing music A. The central control screen still displays the navigation interface of the navigation application. The navigation application in the central control screen still holds the audio focus in the sound zone associated with the in-car speakers, and uses the in-car speakers to play the navigation prompt sound. Since the sound device bound to the co-pilot screen has changed, the sound zone associated with the driving area where the co-pilot screen is located has also changed accordingly. The music APP in the co-pilot screen receives a message that the audio focus has been preempted. Therefore, the music APP pauses playing music. As shown Fig. 15B As shown, the co-pilot screen displays an audio playback interface 1510 of the music APP. The audio playback interface 1510 may include a playback control 1512. The playback control 1512 may indicate that music A is currently paused, and may be used to trigger the music APP to continue playing music A.

[0357] like Fig. 15C As shown, the central control screen can respond to the user operation of opening the audio center and display the user interface 1520 of the audio center. As can be seen from the above embodiments, when the sound device bound to the central control screen and the co-pilot screen is the same, the sound zones associated with the driving and passenger areas where the central control screen and the co-pilot screen are located are the same, and the driving and passenger areas where the central control screen and the co-pilot screen are located share a media session stack. Therefore, the audio center in the central control screen can obtain the media session applied in the co-pilot screen, so that the multimedia content in the co-pilot screen can be displayed. Fig. 15C It can be seen that the user interface 1520 may include multimedia content of the music APP in the co-pilot screen, such as a play control 1521. The play control 1521 can be used to trigger the music APP in the co-pilot screen to play music A.

[0358] In response to Fig. 15C The operation of the play control 1521 is shown, and the in-car speaker can start playing the music A of the music APP in the co-pilot screen. Fig. 15C The playback control 1521 shown can be switched to display Fig.15D The pause control 1522 shown. The pause control 1522 can be used to trigger the music A to pause the music. Fig. 15C The playback control 1521 shown also switches to display Fig.15D The pause control 1511 shown. Among them, the in-car speaker can continue to play music A.

[0359] In some embodiments, in response to Fig. 15CAs shown in the operation of the play control 1521, the audio center can send an instruction to the cockpit audio service to play the audio of the music APP. The cockpit audio service can find the media session of the music APP from the media session stack in the instantiation of the media session service, and then control the media session of the music APP to play the audio of the music APP. In addition, the cockpit audio service can call the audio zone management module to manage the audio focus of the audio zone associated with the in-car speaker, so that the music APP obtains the audio focus.

[0360] In some embodiments, the audio focus obtained by the navigation application in the central control screen is the ducking focus. After the music APP obtains the audio focus, the in-car speakers can still play the navigation prompt sound. Among them, when the in-car speakers play the navigation prompt sound, the in-car speakers can reduce the volume of the music in the music APP. When the navigation prompt sound is finished, the in-car speakers can restore the volume of the music in the music APP.

[0361] In some embodiments, when the music APP on the co-pilot screen holds the audio focus and earphone 1 is used to play music A, if earphone 1 is disconnected, the cockpit system can bind the co-pilot screen to the in-car speaker and assign the audio focus in the associated audio zone of the in-car speaker to the music APP on the co-pilot screen. In this way, the audio of the music APP can continue to play without pausing, and the in-car speaker can continue to play the above music A without the user having to Fig. 15C That is, when an application on a screen holds the audio focus in the audio zone associated with the driving zone where the screen is located, if the sound device bound to the screen is disconnected, the application on the screen can switch to hold the focus in the audio zone where the in-car speaker is located. The application can seamlessly switch to playing audio using the in-car speaker.

[0362] When the first screen (such as the central control screen) and the second screen (such as the co-pilot screen) are both associated with the first sound zone of the in-car speaker, the cockpit system detects an input operation for viewing audio playback information on the first screen, and can display a first list on the first screen according to the first media session stack. The first list includes the audio playback information indicated by the first media session stack. The above-mentioned input operation for viewing audio playback information can be a user operation to open the audio center in the first screen. The first list can refer to the aforementioned Fig. 15C In the case where the in-car speaker plays the second audio, when an input operation on the first audio playback information in the first list is detected on the first screen, the cockpit system can play the first audio through the in-car speaker and pause the second audio. Fig. 15C The operation of the playback control 1521 is shown. For example, the second audio may be Fig. 15BThe navigation prompt sound of the navigation application in the central control screen shown. The first audio can be Fig.15D Music A from the music app on the co-pilot screen is shown.

[0363] When the first screen (such as the central control screen) is associated with the first audio zone of the in-vehicle speaker, and the second screen (such as the co-pilot screen) is associated with the first sound device (such as earphone 1), the cockpit system detects an input operation for viewing audio playback information on the first screen, and can display a second list on the first screen according to the first media session stack. The second list includes the audio playback information indicated by the first media session stack. Since the media session of the application corresponding to the driving zone associated with the second audio zone in the first media session stack has been migrated to the second media session stack, the second list does not include the audio playback information of the application corresponding to the driving zone associated with the second audio zone (for example, the above Fig. 15C The cockpit system detects an input operation on the second screen for viewing the audio playback information, and can display a third list on the second screen according to the second media session stack. The third list includes the audio playback information indicated by the second media session stack. The third list can be the user interface of the audio center on the second screen. Since the first media session stack and the second media session stack are different, the second list and the third list are different.

[0364] Fig.16 The schematic diagram of audio stream transmission in different audio zones provided by the present application is exemplified.

[0365] When there are multiple audio zones in the cockpit system, the audio streams of applications corresponding to the driving and riding areas associated with different audio zones can be transmitted to different sound devices through different channels and played by different sound devices.

[0366] like Fig.16 As shown, the driving zone audio management module includes: the driving zone 0 audio management module corresponding to the central control screen, the driving zone 1 audio management module corresponding to the co-pilot screen, and the driving zone 2 audio management module corresponding to the rear screen. Among them, the driving zone 0 audio management module points to the audio zone 0 management module, which can indicate that the driving zone 0 is associated with the audio zone 0. Similarly, the driving zone 1 is associated with the audio zone 1. The driving zone 2 is associated with the audio zone 2. The audio zone 0 can be the audio zone associated with the in-vehicle speaker. The audio zone 1 can be the audio zone associated with the earphone 1. The audio zone 2 can be the audio zone associated with the earphone 2.

[0367] It can be seen from the above embodiments that the audio management module of driving zone 0 can instruct the audio zone 0 management module to manage the audio routing strategy of the corresponding application of driving zone 0. The audio zone 0 management module can store the audio routing strategy of the corresponding application of driving zone 0 in audio routing table 0. Audio routing table 0 points to the audio channel of the in-car speaker, which can indicate that the audio of the application recorded in audio routing table 0 can be routed to the audio channel of the in-car speaker. Similarly, the audio management module of driving zone 1 can instruct the audio zone 1 management module to manage the audio routing strategy of the corresponding application of driving zone 1. Audio routing table 1 points to the audio channel of the in-car speaker, which can indicate that the audio of the application recorded in audio routing table 1 can be routed to Bluetooth audio channel 1. The audio management module of driving zone 2 can instruct the audio zone 2 management module to manage the audio routing strategy of the corresponding application of driving zone 2. Audio routing table 2 points to the audio channel of the in-car speaker, which can indicate that the audio of the application recorded in audio routing table 2 can be routed to Bluetooth audio channel 2.

[0368] When an application needs to play audio, it can create a corresponding audio track (AudioTrack). The audio track can be a class used for audio playback, which can transmit audio data to the audio channel of the sound device.

[0369] Exemplarily, there may be track 0 in the central control screen. Track 0 can query the application to which track 0 belongs in the audio routing table 0 in the audio policy management module, and then transmit the audio stream of the application to which track 0 belongs to the audio channel of the in-car speaker (for example, audio channel 0, or audio channel 1, or audio channel 2, etc.) according to the audio routing table 0. The audio channel management module can transmit the audio stream in the audio channel of the in-car speaker to the in-car speaker through the in-car speaker hardware abstraction module. Then, the in-car speaker can play the above audio stream from track 0.

[0370] There may be audio track 1 in the co-pilot screen. Audio track 1 can query the application to which audio track 1 belongs in audio routing table 1 in the audio policy management module, and then transmit the audio stream of the application to which audio track 1 belongs to Bluetooth audio channel 1 according to audio routing table 1. The audio channel management module can transmit the audio stream in Bluetooth audio channel 1 to earphone 1 through the Bluetooth hardware abstraction module. Then, earphone 1 can play the audio stream from audio track 1.

[0371] There may be audio track 2 in the rear screen. Audio track 2 can query the application to which audio track 2 belongs in audio routing table 2 in the audio policy management module, and then transmit the audio stream of the application to which audio track 2 belongs to Bluetooth audio channel 2 according to audio routing table 2. The audio channel management module can transmit the audio stream in Bluetooth audio channel 2 to earphone 2 through the Bluetooth hardware abstraction module. Then, earphone 2 can play the audio stream from audio track 2.

[0372] In some embodiments, when it is detected that the first sound device is disconnected, the cockpit system can change the audio zone associated with the second screen from the second audio zone to the first audio zone, and adjust the audio focus stack, audio routing table, and media session stack corresponding to the first audio zone and the second audio zone. Figure 12 to Figure 14 The figure shows an introduction to the cockpit system adjusting the audio routing table 0, audio routing table 1, media session stack 0, media session stack 1, and the audio focus stacks corresponding to audio zones 0 and 1 after detecting that earphone 1 is disconnected.

[0373] Audio Focus Management

[0374] The cockpit system provided by the present application can manage the audio focus in multiple audio zones separately. The audio focus in different audio zones can be independent of each other and do not affect each other.

[0375] Fig.17 A schematic diagram of applying for audio focus provided by the present application is exemplified.

[0376] like Fig.17 As shown, when multimedia playback application 1 needs to play audio, it can request audio focus from the audio service through the audio management API. Multimedia playback application 1 can be a music APP, or a navigation APP, or a video APP, or a telephone APP and other applications that have audio playback requirements. The embodiment of the present application does not limit the type of multimedia playback application 1. The scenarios in which multimedia playback application 1 needs to play audio may include but are not limited to: multimedia playback application 1 detects a user operation for triggering audio playback, multimedia playback application 1 receives an incoming call and needs to play an incoming call prompt tone, multimedia playback application 1 plays a navigation prompt tone during navigation, etc.

[0377] The request for the multimedia playback application 1 to apply for the audio focus may include, but is not limited to: the application identifier of the multimedia playback application 1 and the type of the audio focus applied for.

[0378] The audio focus management module in the audio service can send a request for audio focus from multimedia playback application 1 to the cockpit audio service. The cockpit audio service can determine which driving zone corresponds to the multimedia playback application. Among them, the cockpit audio service can determine which driving zone audio management module's application list the multimedia playback application 1 is in based on the application identifier of the multimedia playback application 1. For example, if the application list of the driving zone 0 audio management module includes multimedia playback application 1, then multimedia playback application 1 is the application corresponding to driving zone 0.

[0379] In the case where the multimedia playback application 1 corresponds to the driving zone 0, the driving zone 0 audio management module can determine that the driving zone 0 is associated with the audio zone 0 according to the active audio zone information in the driving zone 0 audio management module, and then transmit the message of the multimedia playback application 1 requesting to apply for audio focus to the audio zone 0 management module corresponding to the audio zone 0. In this way, the multimedia playback application 1 can apply for audio focus in the audio zone 0. The focus management module in the audio zone 0 management module can determine whether the multimedia playback application 1 can obtain the focus according to the type of audio focus applied by the multimedia playback application 1 and the audio focus stack associated with the audio zone 0.

[0380] Exemplarily, if the audio focus stack associated with audio zone 0 indicates that the application holding the audio focus in audio zone 0 holds the exclusive focus, the audio zone 0 management module may reject the request of multimedia playback application 1 to apply for the audio focus. The request to reject the application for the audio focus may be passed to the multimedia playback application 1 through the cockpit audio service, the audio service, and the audio management API in sequence. In some embodiments, upon receiving the above-mentioned request to reject the application for the audio focus, the multimedia playback application 1 may provide the user with a prompt that the audio playback fails or the audio cannot be played temporarily.

[0381] If the audio focus stack associated with audio zone 0 indicates that the application holding the audio focus in audio zone 0 holds the long focus, the audio zone 0 management module may agree to the request of multimedia playback application 1 to apply for audio focus. Among them, the audio zone 0 management module may delete or move back one position the audio focus information at the top of the audio focus stack associated with audio zone 0, and place the audio focus information of multimedia playback application 1 in the audio focus stack associated with audio zone 0. The audio focus information of the multimedia playback application 1 may include information such as the application identifier of the multimedia playback application 1 and the type of audio focus requested. In this way, multimedia playback application 1 can hold the audio focus in audio zone 0 and use the sound device associated with audio zone 0 to play audio.

[0382] Similarly, when the multimedia player application 1 corresponds to the driving zone 1, the driving zone 1 audio management module can transmit a message of the multimedia player application 1 requesting to apply for audio focus to the sound zone 1 management module. In this way, the multimedia player application 1 can apply for audio focus in the sound zone 1. The focus management module in the sound zone 1 management module can determine whether the multimedia player application 1 can obtain the focus according to the type of audio focus applied by the multimedia player application 1 and the audio focus stack associated with the sound zone 1.

[0383] In the case where the multimedia playback application 1 corresponds to the driving zone 2, the driving zone 1 audio management module can transmit a message of the multimedia playback application 1 requesting to apply for audio focus to the audio zone 2 management module. In this way, the multimedia playback application 1 can apply for audio focus in the audio zone 2. The focus management module in the audio zone 2 management module can determine whether the multimedia playback application 1 can obtain the focus according to the type of audio focus applied by the multimedia playback application 1 and the audio focus stack associated with the audio zone 2.

[0384] In some embodiments, when the audio zone associated with the driving zone changes, the driving zone audio management module can instruct the audio zone management module to update the audio focus stack associated with the corresponding audio zone. For example, driving zone 1 changes from being associated with audio zone 1 to being associated with audio zone 0. The driving zone 1 audio management module can instruct the audio zone 1 management module to migrate the audio focus information in the audio focus stack associated with audio zone 1 to the audio focus stack associated with audio zone 0. If multimedia playback application 1 corresponds to driving zone 1, the audio focus information of multimedia playback application 1 is migrated to the audio focus stack associated with audio zone 0. The audio zone for which multimedia playback application 1 competes for audio focus changes from audio zone 1 to audio zone 0. In the process of the above-mentioned audio focus information migration, multimedia playback application 1 does not need to perceive the changes in the audio zone for which it competes for audio focus. Multimedia playback application 1 only needs to apply for audio focus or give up audio focus to the audio service through the audio management API.

[0385] As can be seen from the above embodiments, different audio zones can independently support applications to apply for audio focus. Applications holding audio focus in different audio zones can use different sound devices to play audio. The cockpit system can manage the audio focus in a sound zone to ensure that the appropriate application can hold the audio focus when multiple applications compete for audio focus in the same audio zone, and use the sound device associated with the audio zone to play audio.

[0386] Volume Management

[0387] The cockpit system provided by the present application can manage the volume in multiple sound zones separately. The volume in different sound zones can be adjusted independently without affecting each other. That is, adjusting the volume in one sound zone will not affect the volume in other sound zones.

[0388] Fig.18 It is a schematic diagram of adjusting the volume of a multimedia playback application provided in an embodiment of the present application.

[0389] like Fig.18As shown, the multimedia playback application 1 can request the audio service to adjust the volume through the audio management API. In some embodiments, after detecting the user operation of adjusting the volume, the multimedia playback application 1 can request to adjust the volume. For example, the user operation of adjusting the volume detected by the multimedia playback application 1 may include: an operation of sliding up or sliding down on a preset area on the video playback interface displayed by the multimedia playback application 1. The embodiment of the present application does not limit the user operation used to trigger the multimedia playback application 1 to adjust the volume.

[0390] The request for the multimedia playback application 1 to adjust the volume may include, but is not limited to: the application identifier of the multimedia playback application 1 and the volume level.

[0391] The volume management module in the audio service can send a request for adjusting the volume of the multimedia playback application 1 to the cockpit audio service. The cockpit audio service can determine which driving zone the multimedia playback application corresponds to.

[0392] If multimedia playback application 1 corresponds to driving zone 0, the driving zone 0 audio management module can transmit a message requesting the multimedia playback application 1 to adjust the volume to the audio zone 0 management module. Then, the audio zone 0 management module can adjust the volume of the audio played by the in-vehicle speaker through the in-vehicle speaker hardware abstraction module.

[0393] If the multimedia playback application 1 corresponds to the driving zone 1, the driving zone 1 audio management module can transmit a message to the audio zone 1 management module requesting the multimedia playback application 1 to adjust the volume. Then, the audio zone 1 management module can adjust the volume of the audio played by the earphone 1 through the Bluetooth hardware abstraction module. In some embodiments, when adjusting the volume of the earphone 1, the Bluetooth hardware abstraction module can select a corresponding volume adjustment method to adjust the volume according to whether the earphone 1 supports absolute volume. Among them, if the earphone 1 supports absolute volume, the Bluetooth hardware abstraction module can adjust the volume of the earphone 1 by adjusting the absolute volume. If the earphone 1 does not support absolute audio, the Bluetooth hardware abstraction module can adjust the volume of the earphone 1 by adjusting the non-absolute volume. The above-mentioned absolute volume may refer to sending the volume size to be adjusted to the earphone, and the earphone performs volume processing on the audio stream to achieve volume adjustment. The above-mentioned non-absolute volume may refer to the Bluetooth hardware abstraction module performing volume processing on the audio stream according to the volume size to be adjusted, and handing over the audio stream after volume processing to the earphone for playback. That is, in the above-mentioned method of adjusting the non-absolute volume, the earphone does not need to perform volume processing. The non-absolute volume can also be called relative volume. The embodiment of the present application does not limit the above-mentioned method of adjusting the volume.

[0394] Similarly, if the multimedia playback application 1 corresponds to the driving zone 2, the driving zone 2 audio management module can transmit a message from the multimedia playback application 1 to the audio zone 2 management module requesting to adjust the volume. Then, the audio zone 2 management module can adjust the volume of the audio played by the earphone 2 through the Bluetooth hardware abstraction module. When adjusting the volume of the earphone 2, the Bluetooth hardware abstraction module can select a corresponding volume adjustment method to adjust the volume according to whether the earphone 2 supports absolute volume. For details, please refer to the above method for adjusting the volume of the earphone 1. I will not go into details here.

[0395] Fig.19 It is a schematic diagram of a system application for adjusting volume provided in an embodiment of the present application.

[0396] like Fig.19 The system application 1 shown can request the cockpit audio service to adjust the volume through the cockpit audio management API. System application 1 can be a system application such as a cockpit voice assistant, an audio center, or a desktop service. After detecting the user operation of adjusting the volume, system application 1 can request to adjust the volume. For example, if system application 1 is a cockpit voice assistant, the user operation of adjusting the volume detected by system application 1 may include: a voice instruction to adjust the volume. If system application 1 is an audio center, the user operation of adjusting the volume detected by system application 1 may include: a user operation of the volume control (such as a volume bar, etc.) in the multimedia playlist displayed in the audio center. If the system application is a desktop service, the user operation of adjusting the volume detected by system application 1 may include: a user operation of the volume control (such as a volume bar, etc.) displayed on the desktop, and a user operation of the physical button on the screen for adjusting the volume. The embodiment of the present application does not limit the user operation used to trigger the system application 1 to adjust the volume.

[0397] In some embodiments, the system application 1 can determine which screen it belongs to and the driving zone bound to the screen to which it belongs. The request for adjusting the volume by the system application 1 may include but is not limited to: the driving zone identifier and volume level of the driving zone bound to the screen to which the system application 1 belongs.

[0398] The cockpit audio service can determine the driving zone according to the driving zone identifier in the request, and use the driving zone audio management module corresponding to the driving zone to adjust the volume.

[0399] If the driving zone identifier indicates driving zone 0, the driving zone 0 audio management module can transmit a message requesting the system application 1 to adjust the volume to the audio zone 0 management module. In this way, the volume in audio zone 0, that is, the volume of the in-vehicle speaker, can be adjusted.

[0400] If the riding zone identifier indicates riding zone 1, the riding zone 1 audio management module can transmit a message requesting the system application 1 to adjust the volume to the audio zone 1 management module. In this way, the volume in audio zone 1, that is, the volume of earphone 1, can be adjusted.

[0401] If the riding zone identifier indicates riding zone 2, the riding zone 2 audio management module can transmit a message requesting the system application 1 to adjust the volume to the audio zone 2 management module. In this way, the volume in audio zone 2, that is, the volume of earphone 2, can be adjusted.

[0402] The specific method for adjusting the volume of the corresponding sound zone by the driving zone audio management module corresponding to the driving zone 0, driving zone 1 or driving zone 2 can refer to the aforementioned Fig.18 The description of the embodiments will not be repeated here.

[0403] Figures 20A to 20D The following are some schematic diagrams of volume adjustment scenarios provided by this application.

[0404] like Fig. 20A As shown, the central control screen is bound to the in-car speaker. The central control screen can display the navigation interface of the navigation application. The in-car speaker can play the navigation prompt tone of the navigation application. The co-pilot screen is bound to the earphone 1, and the co-pilot screen can display the audio playback interface 2010 of the music APP. The audio playback interface 2010 may include a volume bar 2011. The volume bar 2011 can be used to trigger the music APP to adjust the volume. The earphone 1 can play the music A of the music APP.

[0405] In response to Fig. 20A The operation shown is to slide right on the volume bar 2011 to increase the volume, and the music APP can request to increase the volume.

[0406] like Fig. 20B As shown, the cockpit system can adjust the volume of the audio played by the earphone 1 based on the music APP's request to increase the volume, so that the earphone 1 increases the volume to play music A. The specific method of adjusting the volume of the audio played by the earphone 1 by the cockpit system can refer to the above Fig.18 The method described in the examples.

[0407] Depend on Fig. 20A and Fig. 20B It can be seen that when the application in the co-pilot screen requests to adjust the volume, the cockpit system can adjust the volume of the earphone 1 bound to the co-pilot screen, while keeping the volume of the in-car speaker unchanged. In this way, users who use the central control screen and listen to audio through the in-car speaker and users who use the co-pilot screen and listen to audio through earphone 1 can listen to audio and adjust the volume independently without interfering with each other.

[0408] like Fig. 20CAs shown, when earphone 1 plays music (such as music A) in a music APP, the co-pilot screen can display desktop 2020 in response to the user's operation of calling up the desktop. In some embodiments, desktop 2020 may include a volume control. In response to a user operation of increasing the volume on volume control 2021, the desktop service in the co-pilot screen can request to increase the volume.

[0409] like Fig.20D As shown, the cockpit system can adjust the volume of the audio played by the headset 1 based on the request of the desktop service to increase the volume in the co-pilot screen, so that the headset 1 increases the volume to play music A. The specific method of adjusting the volume of the audio played by the headset 1 by the desktop service can refer to the above Fig.19 The method described in the examples.

[0410] In some embodiments, when the first screen (such as the central control screen) is associated with the first sound zone (such as sound zone 0) of the in-car speaker, and the second screen (such as the co-pilot screen) is associated with the second sound zone (such as sound zone 1) of the first sound device (such as earphone 1), the user can adjust the volume of the in-car speaker through the first screen, and adjust the volume of the first sound device through the second screen. For example, when an input operation for adjusting the volume to the first volume is detected on the second screen, the cockpit system can adjust the playback volume of the first sound device to the first volume. The above process of adjusting the volume of the first sound device can refer to the above Figures 20A to 20D Embodiment of the invention.

[0411] Media Session Management

[0412] The cockpit system provided in the present application can manage the media sessions of the applications corresponding to each driving zone. Among them, when the audio zones associated with multiple driving zones are the same, the cockpit system can uniformly manage the media sessions of the applications corresponding to these multiple driving zones. The media sessions of the applications corresponding to these multiple driving zones can be placed in the same media session stack. When the audio zones associated with two driving zones are different, the cockpit system can manage the media sessions of the applications corresponding to these two driving zones separately. The media sessions of the applications corresponding to these two driving zones can be placed in two different media session stacks respectively. In other words, the media session stacks in the cockpit system can correspond to the audio zones one by one. The media session stack corresponding to a audio zone may include the application corresponding to the driving zone associated with this audio zone. The media session stacks corresponding to different audio zones can be independent of each other and do not affect each other.

[0413] Fig.21 This is a schematic diagram of a media session management provided by this application.

[0414] Here, an example is taken for explanation that the cockpit system includes driving zone 0, driving zone 1, and driving zone 2, and driving zone 0 and driving zone are associated with the same sound zone, and the sound zone associated with driving zone 2 is different from the sound zone associated with driving zone 0 and driving zone 1.

[0415] like Fig.21 As shown, the cockpit media session management module includes the driving zone 0 media session management module, the driving zone 1 media session management module, and the driving zone 2 media session management module. Among them, the screen in the driving zone 0 is bound to the in-vehicle speaker. The driving zone 0 is associated with the sound zone of the in-vehicle speaker. The driving zone 0 media session management module can instruct the media session service to instantiate and create a media session stack 0, and store the media session of the corresponding application of the driving zone 0 in the media session stack 0.

[0416] There is no headset bound to the screen in driving zone 1, that is, no external sound device is connected to the cockpit system through the screen in driving zone 1. Therefore, the screen in driving zone 1 is bound to the in-car speaker. Driving zone 1 and driving zone 0 are associated with the same audio zone. The driving zone 1 media session management module can store the media session of the corresponding application of driving zone 1 in media session stack 0.

[0417] The driving zone 2 has a screen bound to earphone 2. Therefore, the driving zone 2 can be associated with the audio zone of earphone 2. The driving zone 2 media session management module can instruct the media session service to instantiate and create a media session stack 2, and store the media session of the corresponding application of the driving zone 2 in the media session stack 2.

[0418] The audio center on the screen in the driving zone 0 can call the driving zone 0 media session management module through the cockpit audio management API to obtain the media session. The audio center on the screen in the driving zone 0 can obtain the media session in the media session stack 0.

[0419] The audio center on the screen in the driving area 1 can call the driving area 1 media session management module through the cockpit audio management API to obtain the media session. The audio center on the screen in the driving area 1 can obtain the media session in the media session stack 0.

[0420] The audio center on the screen in the driving area 2 can call the driving area 2 media session management module through the cockpit audio management API to obtain the media session. The audio center on the screen in the driving area 2 can obtain the media session in the media session stack 2.

[0421] Media session stack 0 includes media sessions for the corresponding applications in riding zone 0 and riding zone 1. Therefore, the multimedia playlists displayed in the audio center on the screen in riding zone 0 and riding zone 1 are the same, and both include multimedia content information for the corresponding applications in riding zone 0 and riding zone 1. Media session stack 0 includes media sessions for the corresponding applications in riding zone 2. The multimedia playlist displayed in the audio center on the screen in riding zone 2 includes multimedia content information for the corresponding applications in riding zone 2, and is different from the multimedia playlists displayed in the audio center on the screen in riding zone 0 and riding zone 1.

[0422] When an external sound device is connected to the cockpit system through the screen in the driving area 1, the sound device bound to the screen in the driving area 1 can be switched from the in-car speaker to the external sound device. The driving area 2 media session management module can instruct the media session service to instantiate a new media session stack, such as media session stack 1, and migrate the media session of the corresponding application in the driving area 1 from media session stack 0 to media session stack 1.

[0423] It can be seen that the media session stack and the media sessions in the media session stack can change dynamically with the connection and removal of external sound devices.

[0424] In some embodiments, a multimedia playback application can request a media session service to create a media session corresponding to the multimedia playback application through a media session management API. The media session service can send a message to the cockpit media session management module that the multimedia application requests to create a media session. Then, the cockpit media session management module can determine which driving zone the multimedia playback application corresponds to, and use the driving zone media session management module corresponding to the corresponding driving zone to create a media session. For example, if the multimedia playback application corresponding to driving zone 0 requests to create a media session, the driving zone 0 media session management module can create a media session corresponding to the multimedia playback application and store the media session in media session stack 0.

[0425] The multimedia player application can request the media session service to access its own media session through the media session management API. Upon receiving the request from the multimedia player application to access the media session, the media session service can obtain the media session information of the multimedia player application from the media session service instantiation, and then send the media session information to the multimedia player application.

[0426] It can be seen that the multimedia playback application accesses the media session based on the media session service, and does not need to perceive the changes in the media session stack in the cockpit system.

[0427] Screen Migration

[0428] The cockpit system provided in the present application can support screen migration. Screen migration can refer to migrating multimedia content played on one screen to another screen. Among them, the original screen that performs screen migration can cancel the display of the playback interface of the migrated multimedia content, and the sound device bound to the original screen can stop playing the audio in the migrated multimedia content. After the screen migration, the screen that receives the screen migration can display the playback interface of the migrated multimedia content, and the sound device bound to the screen can continue to play the audio from the position where the multimedia content was played when the screen migration was performed. This can ensure the continuity of audio playback in the screen migration scenario.

[0429] Fig. 22 This is a schematic diagram of a screen migration method provided by the present application.

[0430] Here we take the example of migrating the multimedia content being played on the central control screen to the co-driver screen, and the sound devices bound to the central control screen and the co-driver screen are different.

[0431] like Fig. 22 As shown, the desktop service can generate an APP migration event for application 1 to migrate from the central control screen to the co-driver screen. Application 1 can be the multimedia playback application corresponding to the central control screen. The multimedia playback interface of application 1 can be displayed on the central control screen, and the sound device bound to the central control screen, such as the in-car speaker, is playing the audio of application 1. In response to the user operation of migrating the multimedia playback interface of application 1 from the central control screen to the co-driver screen, the desktop service in the central control screen can generate the above-mentioned APP migration event of application 1. That is, the APP migration event may refer to the display screen change of the multimedia playback interface of application 1. The APP migration event may also be called a screen migration event.

[0432] In some embodiments, the APP migration event may include but is not limited to: the application identifier of the migrated application (i.e., application 1), the screen identifier of the original screen (i.e., the central control screen), and the screen identifier of the screen receiving the screen migration (i.e., the co-pilot screen). The desktop service can send the above APP migration event to the driving area audio management module. Driving area 0 is bound to the central control screen. Driving area 1 is bound to the co-pilot screen. Before the screen migration, the application list in the driving area 0 audio management module may include application 1, and the application list in the driving area 1 audio management module may not include application 1.

[0433] Based on the above APP migration event, the audio management module of driving zone 0 can migrate application 1 in the application list to the audio management module of driving zone 1. Therefore, after the screen migration, the application list in the audio management module of driving zone 0 may not include application 1, and the application list in the audio management module of driving zone 1 may include application 1. In other words, application 1 changes from the application corresponding to driving zone 0 to the application corresponding to driving zone 1.

[0434] Driving zone 0 is associated with audio zone 0. Driving zone 1 is associated with audio zone 1. Based on the above-mentioned APP migration event, the driving zone 0 audio management module can also instruct the audio zone 0 management module to migrate the audio routing strategy and audio focus information of application 1 to the audio zone 1 management module. The audio zone 1 management module can store the audio routing strategy of the above-mentioned application 1 in the audio routing table associated with audio zone 1, and store the audio focus information of application 1 in the audio focus stack associated with audio zone 1. Based on the migration of the above-mentioned audio routing strategy and audio focus information, the sound device that plays the audio of application 1 can be changed from the in-car speaker to the sound device bound to the co-pilot screen, such as earphone 1.

[0435] In some embodiments, the audio zone 1 management module may place the audio focus information of application 1 at the top of the audio focus stack associated with audio zone 1. Then, after the screen migration, application 1 may continue to hold the audio focus, but the audio focus it holds changes from the audio focus of audio zone 0 to the audio focus of audio zone 1.

[0436] When the riding zone corresponding to application 1 changes, the riding zone 1 audio management module can synchronize the riding zone corresponding to application 1 with the riding zone 1 media session management module. Then, the riding zone 1 media session management module can instruct the media session service instantiation to migrate the media session of application 1 from media session stack 0 to media session stack 1.

[0437] In some embodiments, when the multimedia playback interface of the above-mentioned application 1 is migrated to the co-pilot screen, the desktop service of the co-pilot screen can display the multimedia playback interface of application 1 according to the size of the co-pilot screen, so that the multimedia playback interface of application 1 is adapted to the co-pilot screen.

[0438] It should be noted that in the above screen migration scenario, although application 1 changes from the application corresponding to driving zone 0 to the application corresponding to driving zone 1, the storage location of the application installation package of application 1 can remain unchanged. The application icon of application 1 can still be displayed on the central control screen. In response to the operation of opening application 1, the cockpit system can still display the user interface of application 1 on the central control screen.

[0439] By the above Fig. 22 It can be seen that during the screen migration process, the migrated application does not need to adapt the multimedia display interface and audio stream according to the screen after the migration. The migration of the multimedia playback interface display screen and the migration of the audio sound device can be completed by the driving area management layer and the audio framework layer in the cockpit system. Fig.16It can be seen from the embodiment of audio stream transmission that since the audio routing strategy of the above-mentioned application 1 is migrated to the audio routing table associated with audio zone 1, the audio track of application 1 can find application 1 in the audio routing table associated with audio zone 1, and then the audio of application 1 is transmitted to the audio channel of earphone 1 according to the audio routing table associated with audio zone 1. This can achieve seamless switching of the audio of application 1 from the in-car speaker to earphone 1, maintaining the continuity of audio playback. The above-mentioned seamless switching of the audio of application 1 from the in-car speaker to earphone 1 can mean that when the audio of application 1 is switched to earphone 1, earphone 1 can continue to play the audio from the position where the in-car speaker stops playing the audio.

[0440] Fig.23 and Fig.24 It is a schematic diagram of the screen migration scenario provided by this application.

[0441] like Fig.23 As shown, the central control screen displays the multimedia playback interface of application 1. The driving zone 0 bound to the central control screen contains application 1. That is, application 1 is currently the application corresponding to driving zone 0. Driving zone 0 is associated with the entire vehicle sound zone. The media session stack corresponding to the entire vehicle sound zone includes the media session of application 1. Application 1 can determine that the audio of application 1 needs to be routed to the audio channel of the in-car speakers based on the audio routing table associated with the entire vehicle sound zone. Therefore, application 1 can transmit the audio playback data to the audio channel of the in-car speakers. The in-car speakers can play the audio of application 1. In response to the operation of migrating the multimedia playback interface of application 1 to the co-pilot screen, the cockpit system can migrate the media session, audio routing strategy, and audio focus of application 1. For the specific process, please refer to the aforementioned Fig. 22 Introduction to the embodiment. The embodiment of the present application does not limit the above-mentioned operation of migrating the multimedia playback interface of application 1 to the co-pilot screen. For example, the operation can be an operation of sliding three fingers in a specified direction on the multimedia playback interface of application 1.

[0442] like Fig.24As shown, when the multimedia playback interface of application 1 is migrated from the central control screen to the co-driver screen, the co-driver screen can display the multimedia playback interface of application 1. The central control screen can cancel the display of the multimedia playback interface of application 1. Application 1 migrates from driving zone 0 to driving zone 1. That is, application 1 changes to the application corresponding to driving zone 1. The media session of application 1 is migrated to the media session stack corresponding to headphone audio zone 1. The media session stack corresponding to the whole vehicle audio zone no longer contains the media session of application 1. In addition, since the audio routing strategy of application 1 is migrated from the audio routing table associated with the whole vehicle audio zone to the audio routing table associated with headphone audio zone 1, application 1 can determine that the audio of application 1 needs to be routed to the audio channel of headphone 1 according to the audio routing table associated with headphone audio zone 1. Therefore, application 1 can transmit audio playback data to the audio channel of headphone 1. Headphone 1 can play the audio of application 1. After the in-vehicle speaker fails to receive the audio playback data of application 1, the audio of application 1 can be stopped.

[0443] After the above application 1 completes the screen migration, the central control screen can open other multimedia playback applications and play the corresponding audio through the in-car speaker. At this time, the in-car speaker and the headset 1 can play audio separately without interfering with each other.

[0444] It can be seen from the above screen migration scenario that even if application 1 is not installed on the co-pilot screen, the multimedia playback interface of application 1 can be displayed through the above screen migration method, and the audio of application 1 can be played through the earphone 1 bound to the co-pilot screen.

[0445] In some embodiments, in response to the operation of closing application 1 on the co-pilot screen, the cockpit system can delete application 1 from the application list corresponding to driving zone 1, and delete the audio focus information and audio routing strategy of application 1 in audio zone 1.

[0446] In some embodiments, the multimedia playback interface of the above application 1 can also be migrated from the co-pilot screen back to the central control screen. For details, please refer to the introduction of the migration of the multimedia playback interface of application 1 from the central control screen to the co-pilot screen in the above embodiment.

[0447] In some embodiments, when the first screen (such as the central control screen) is associated with the first sound zone (such as sound zone 0) of the in-vehicle speaker, and the second screen (such as the co-pilot screen) is associated with the second sound zone (such as sound zone 1) of the first sound device (such as earphone 1), the audio played by the in-vehicle speaker can be migrated from the first screen to the second screen as the multimedia playback interface corresponding to the audio is migrated to the first sound device for playback. For example, the first screen displays the first interface of the second application. The first interface can be Fig.23The multimedia playback interface of application 1 is shown. The first interface is the playback interface of the second audio. The second audio can be the audio of application 1. When an input operation to migrate the first interface from the first screen to the second screen is detected, the cockpit system can display the first interface on the second screen, cancel the display of the first interface on the first screen, and switch the second audio from the in-car speaker to the first sound device to continue playing. The in-car speaker can pause playing the second audio. For details, please refer to Fig.23 and Fig.24 The multimedia playback interface of application 1 is migrated from the central control screen to the co-pilot screen.

[0448] In the above scenario where the first interface of the second application is migrated from the first screen to the second screen, the cockpit system can migrate the audio focus of the second application in the audio focus stack, migrate the application information of the second application in the audio routing table, and migrate the media session of the second application in the media session stack. Fig. 22 The example shown is an introduction to an embodiment of migrating application 1.

[0449] Screen Sharing

[0450] The cockpit system provided in this application can support screen sharing. Screen sharing can refer to sharing the multimedia content played on one screen to one or more other screens. Among them, the original screen for screen sharing can continue to display the playback interface of the shared multimedia content.

[0451] Fig.25 This is a schematic diagram of a screen sharing method provided by this application.

[0452] Here we take the example of sharing the multimedia content being played on the co-pilot screen to the central control screen and the rear screen, and the sound devices bound to the central control screen, the co-pilot screen, and the rear screen are different.

[0453] like Fig.25 As shown, in the cockpit system, the cockpit audio service may also include a sharing management module. The sharing management module may be used to manage screen sharing. Among them, the content managed by the sharing management module may include: the riding area information that receives sharing, the shared application information, the emitting riding area information, and the shared riding area information. The riding area information that receives sharing can be used to indicate the riding area to which the screen that receives multimedia content sharing belongs. The shared application information can be used to indicate which application the shared multimedia content belongs to. The emitting riding area information can be used to indicate which riding area, after screen sharing, has the sound device bound to the screen play the audio in the shared multimedia content. The shared riding area information can be used to indicate the riding area to which the screen that initiates screen sharing belongs, that is, the riding area to which the original screen that played the shared content before screen sharing belongs.

[0454] like Fig.25 As shown, the desktop service can generate an APP sharing event for sharing application 4 from the co-pilot screen to the central control screen and the rear screen. Application 4 can be the multimedia playback application corresponding to the co-pilot screen. The multimedia playback interface of application 4 can be displayed on the co-pilot screen, and the sound device bound to the co-pilot screen, such as earphone 1, is playing the audio of application 4. In response to the user operation of sharing the multimedia playback interface of application 4 from the co-pilot screen to the central control screen and the rear screen, the desktop service of the co-pilot screen can generate the above-mentioned APP sharing event of application 4. That is, the APP sharing event can mean that the multimedia playback interface of application 4 will be displayed on more screens. The APP sharing event can also be called a screen sharing event.

[0455] In some embodiments, the APP sharing event may include but is not limited to: the application identifier of the shared application (i.e., application 4), the screen identifier of the original screen (i.e., the co-pilot screen), and the screen identifier of the screen receiving the sharing (i.e., the central control screen and the rear screen). The desktop service can send the above-mentioned APP sharing event to the sharing management module. The sharing management module can create a sharing area 0 management module based on the APP sharing event. The sharing area 0 management module can be used to manage the APP sharing events of the above-mentioned application 4. Among them, the sharing management module may include one or more sharing area management modules. A sharing area management module can be used to manage an APP sharing event. When other APP sharing events are received, the sharing management module can create a new sharing area management module.

[0456] Based on the APP sharing event of application 4, the sharing area management module can determine the following information in the APP sharing event: the driving area information that accepts the sharing, the shared application information, the driving area information that is voiced, and the driving area information that is shared. Among them, in the APP sharing event of application 4, the driving area information that accepts the sharing can be used to indicate the driving area 0 where the central control screen is located and the driving area 2 where the rear screen is located. The shared application information can be used to indicate application 4. The voiced driving area information can be used to indicate the driving area 0 where the central control screen bound to the in-car speaker is located. The shared driving area information can be used to indicate the driving area where the co-pilot screen is located. That is to say, in the scenario where the co-pilot screen shares the multimedia content of application 4 with the central control screen and the rear screen, the co-pilot screen, the central control screen and the rear screen can all display the multimedia playback interface of application 4, and the audio playing application 4 is switched from earphone 1 to the in-car speaker. In this way, the driver can watch the multimedia playback interface of application 4 through the central control screen, and the user sitting in the co-pilot seat can watch the multimedia playback interface of application 4 through the co-pilot screen. Users sitting in the back row can watch the multimedia playback interface of application 4 through the back screen. All users in the car can listen to the audio of application 4 through the in-car speakers.

[0457] The embodiments of the present application do not limit the method of selecting the sound device after screen sharing. In some embodiments, if the screen to be shared includes the central control screen or the original screen for screen sharing is the central control screen, the sound device after screen sharing can be the in-car speaker bound to the central control screen. Other external sound devices in the cockpit system do not need to play the audio of the shared application. If the screen to be shared does not include the central control screen, and the original screen for screen sharing is not the central control screen, the sound device after screen sharing can include the sound device bound to the screen to be shared, and the sound device bound to the original screen for screen sharing. In some embodiments, in any APP sharing event, the sound device after screen sharing can be switched to the in-car speaker. Other external sound devices in the cockpit system do not need to play the audio of the shared application.

[0458] The sharing zone 0 management module can indicate to the riding zone 1 audio management module that riding zone 1 is the shared riding zone, and the shared application is application 4, based on the shared riding zone information and the shared application information. The sharing zone 0 management module can instruct the riding zone 0 audio management module and the riding zone 2 audio management module to accept screen sharing based on the received shared riding zone information. The sharing zone 0 management module can indicate that riding zone 0 is the riding zone where the sound device is located after screen sharing, based on the sounding riding zone information. Then, the riding zone 1 audio management module can migrate application 4 in the application list to the riding zone 0 audio management module. Therefore, after screen sharing, the application list in the riding zone 1 audio management module may not include application 4, and the application list in the riding zone 0 audio management module may include application 4. That is to say, application 4 changes from the application corresponding to riding zone 1 to the application corresponding to riding zone 0.

[0459] Driving zone 0 is associated with audio zone 0. Driving zone 1 is associated with audio zone 1. Based on the instructions of the above-mentioned sharing zone 0 management module, the driving zone 1 audio management module can instruct the audio zone 1 management module to migrate the audio routing strategy and audio focus information of application 4 to the audio zone 0 management module. The audio zone 0 management module can store the audio routing strategy of the above-mentioned application 4 in the audio routing table associated with audio zone 0, and store the audio focus information of application 4 in the audio focus stack associated with audio zone 0. Based on the migration of the above-mentioned audio routing strategy and audio focus information, the sound device that plays the audio of application 4 can be changed from earphone 1 to the in-car speaker bound to the central control screen.

[0460] In some embodiments, the audio zone 0 management module may place the audio focus information of application 4 at the top of the audio focus stack associated with audio zone 0. Then, after the screen migration, application 4 may continue to hold the audio focus, but the audio focus it holds changes from the audio focus of audio zone 1 to the audio focus of audio zone 0.

[0461] In some embodiments, when the riding zone corresponding to application 4 changes, the riding zone 0 audio management module may synchronize the riding zone corresponding to application 4 with the riding zone 0 media session management module. Then, the riding zone 0 media session management module may instruct the media session service instantiation to migrate the media session of application 4 from media session stack 1 to media session stack 0. Optionally, the media session of application 4 may not be migrated and is still stored in media session stack 1.

[0462] In some embodiments, when the multimedia playback interface of the above-mentioned application 4 is shared to the central control screen and the rear screens, the desktop service of the central control screen can display the multimedia playback interface of application 4 according to the size of the central control screen, so that the multimedia playback interface of application 4 is adapted to the central control screen; the desktop service of the rear screen can display the multimedia playback interface of application 4 according to the size of the rear screen, so that the multimedia playback interface of application 4 is adapted to the rear screen.

[0463] It should be noted that in the above screen sharing scenario, although application 4 is changed from the application corresponding to driving zone 1 to the application corresponding to driving zone 0, the storage location of the application installation package of application 4 can remain unchanged. The application icon of application 4 can still be displayed on the co-pilot screen.

[0464] By the above Fig.25 It can be seen that during the screen sharing process, the shared application does not need to adapt the multimedia display interface and audio stream according to the screen receiving the screen sharing. The migration of the multimedia playback interface display screen and the migration of the audio sound device can be completed by the driving area management layer and the audio framework layer in the cockpit system. Fig.16 As can be seen from the embodiment of audio stream transmission, since the audio routing strategy of the above-mentioned application 4 is migrated to the audio routing table associated with audio zone 0, the audio track of application 4 can find application 4 in the audio routing table associated with audio zone 0, and then the audio of application 4 is transmitted to the audio channel of the in-car speaker according to the audio routing table associated with audio zone 0. This can achieve seamless switching of the audio of application 4 from earphone 1 to the in-car speaker, maintaining the continuity of audio playback.

[0465] Fig.26 and Fig. 27 This is a schematic diagram of the screen sharing scenario provided by this application.

[0466] like Fig.26As shown, the co-pilot screen displays the multimedia playback interface of application 4. The driving zone 1 bound to the co-pilot screen contains application 4. That is, application 4 is currently the application corresponding to driving zone 1. Driving zone 1 is associated with headphone audio zone 1. The media session stack corresponding to headphone audio zone 1 includes the media session of application 4. Application 4 can determine that the audio of application 4 needs to be routed to the audio channel of headphone 1 based on the audio routing table associated with headphone audio zone 1. Therefore, application 4 can transmit the audio playback data to the audio channel of headphone 1. Headphone 1 can play the audio of application 4. In response to the operation of sharing the multimedia playback interface of application 4 to the central control screen and the rear screen, the cockpit system can migrate the media session, audio routing strategy, and audio focus of application 4. Please refer to the aforementioned for the specific process. Fig.25 Introduction to the embodiment. The embodiment of the present application does not limit the above-mentioned operation of migrating the multimedia playback interface of application 1 to the co-pilot screen.

[0467] like Fig. 27 As shown, when the multimedia playback interface of application 4 is shared from the co-pilot screen to the central control screen and the rear screen, the co-pilot screen, the central control screen and the rear screen can all display the multimedia playback interface of application 4. Application 4 is migrated from driving zone 1 to driving zone 0. That is, application 4 is changed to the application corresponding to driving zone 0. The media session of application 4 is migrated to the media session stack corresponding to the whole vehicle audio zone. The media session stack corresponding to headphone audio zone 1 no longer contains the media session of application 4. In addition, since the audio routing strategy of application 4 is migrated from the audio routing table associated with headphone audio zone 1 to the audio routing table associated with the whole vehicle audio zone, application 4 can determine that the audio of application 4 needs to be routed to the audio channel of the in-vehicle speaker according to the audio routing table associated with the whole vehicle audio zone. Therefore, application 4 can transmit the audio playback data to the audio channel of the in-vehicle speaker. The in-vehicle speaker can play the audio of application 4. After earphone 1 cannot receive the audio playback data of application 4, it can stop playing the audio of application 4. Earphone 2 also cannot receive the audio playback data of application 4, and there is no need to play the audio of application 4.

[0468] In some embodiments, after the above-mentioned application 4 completes screen sharing, in response to the operation of closing application 4 on the co-pilot screen, the co-pilot screen can cancel the display of the multimedia playback interface of application 4. The central control screen and the rear screen can still continue to display the multimedia playback interface of application 4, and the in-car speakers can continue to play the audio of application 4. After closing application 4, the co-pilot screen can open other multimedia playback applications, and play the corresponding audio through headphones 1. Similarly, after the above-mentioned application 4 completes screen sharing, in response to the operation of closing application 4 on the rear screen, the rear screen can cancel the display of the multimedia playback interface of application 4. The central control screen and the co-pilot screen can still continue to display the multimedia playback interface of application 4, and the in-car speakers can continue to play the audio of application 4. After closing application 4, the rear screen can open other multimedia playback applications, and play the corresponding audio through headphones 2.

[0469] In some embodiments, after the above-mentioned application 4 completes screen sharing, in response to the operation of closing application 4 on the central control screen, the co-pilot screen can cancel the multimedia playback interface of application 4. The co-pilot screen and the rear screen can still continue to display the multimedia playback interface of application 4, and the in-car speakers can continue to play the audio of application 4. Among them, application 4 still holds the audio focus of the entire vehicle sound zone. If other applications in the central control screen apply for audio focus and seize the audio focus held by application 4, the in-car speakers can pause playing the audio of application 4 and instead play the audio of the application that seizes the audio focus of application 4.

[0470] In some embodiments, in a screen sharing scenario, if a portion of the screens in a riding area are screens that accept sharing, and another portion of the screens are screens that do not accept sharing, then the screens in this riding area that accept sharing can display the shared multimedia content, and the screens that do not accept sharing do not display the shared multimedia content. Furthermore, in the case where this riding area is a riding area that sounds after screen sharing, when an application occupies the audio focus on the screen that does not receive sharing in this riding area, the shared multimedia content can be paused, and the sound-emitting device in the associated sound zone in this riding area can play the audio of the application that occupies the audio focus. In the case where this riding area is not a riding area that sounds after screen sharing, when an application applies for audio focus on the screen that does not receive sharing in this riding area, the sound-emitting device in the associated sound zone in this riding area can play the audio of the application that applies for audio focus, and the audio in the shared multimedia content can continue to be emitted by other riding areas.

[0471] Figures 28A to 28F These are some other screen sharing scene diagrams provided by this application.

[0472] like Fig.28A As shown, the central control screen is bound to the in-car speakers. The central control screen can display the user interface of the radio. The in-car speakers can play the audio of the radio. The co-pilot screen is bound to earphone 1. The co-pilot screen can display the video playback interface 2810 and sharing control 2811 of video APP1. Sharing control 2811 can be used to trigger screen sharing. Earphone 1 can play the audio of video A in video APP1. The rear screen is bound to earphone 2. The rear screen can display the video playback interface of video APP2. Earphone 2 can play the audio of video B in video APP2.

[0473] like Fig.28BAs shown, in response to the operation of the sharing control 2811, the co-pilot screen may display a device option box 2812. The device option box 2812 may include one or more device options for screen sharing. For example, the all screens option, the central control screen option, the co-pilot screen option, etc. The all screens option can be used to share multimedia content to all screens in the cockpit. The central control screen option can be used to share multimedia content to the central control screen. The rear screen option can be used to share multimedia content to the rear screen. In response to Fig.28B As shown in the figure, by selecting the operation of all screens, the co-pilot screen can share the multimedia content of video APP1 to all screens in the cockpit, such as the central control screen and the rear screens.

[0474] like Fig.28C As shown, the central control screen can accept screen sharing from the co-pilot screen and display the video playback interface of video APP1. The rear screen can also accept screen sharing from the co-pilot screen and display the video playback interface of video APP1. The co-pilot screen can still display the video playback interface of video APP1. Among them, after the co-pilot screen shares its screen, the sound device that plays the audio in the shared multimedia content can be switched from earphone 1 to the in-car speaker. The in-car speaker can pause the audio of the radio and continue playing the audio of video A. Earphone 1 can pause the audio of video A. Earphone 2 can pause the audio of video B.

[0475] like Fig.28D As shown, after the co-pilot screen is shared, the phone APP in the central control screen receives an incoming call. The central control screen can cancel the video playback interface of video APP1 and instead display the incoming call prompt interface of the phone APP. Among them, the phone APP can seize the audio focus of video APP1. The in-car speakers can pause the audio of video A and play the incoming call prompt tone of the phone APP instead. Since the audio focus of video APP1 is seized, the multimedia content in video APP1 can be paused. Fig.28D As shown, the co-pilot screen can display the video playback interface 2810 of the video APP1. The video playback interface 2810 may include a playback control 2813. The playback control 2813 may indicate that the video A in the video playback interface 2810 is currently in a paused state, and can be used to trigger the playback of video A. Similarly, the video playback interface of the video APP1 displayed on the rear screen may also indicate that the video A is paused. There is no audio playback in both earphones 1 and 2.

[0476] like Fig.28EAs shown in the figure, after the co-pilot screen shares the screen, the co-pilot screen can close the video APP1 and open the music APP. The co-pilot screen can display the music APP and play the music A in the audio APP through the earphone 1 bound to the co-pilot screen. The central control screen and the rear screen can continue to display the shared multimedia content, that is, the video playback interface of the video APP1. The in-car speakers can continue to play the audio of the video A in the video APP1. There is no audio playback in the earphone 2.

[0477] like Fig.28F As shown, after the co-pilot screen shares the screen, the rear screen can close video APP1 and open music APP1. The rear screen can display music APP1 and play music B in music APP1 through earphone 2 bound to the rear screen. The central control screen and the co-pilot screen can continue to display the shared multimedia content, that is, the video playback interface of video APP1. The in-car speakers can continue to play the audio of video A in video APP1. Earphone 1 has no audio playback.

[0478] In some embodiments, when the first screen (such as the central control screen) is associated with the first sound zone (such as sound zone 0) of the in-car speaker, and the second screen (such as the co-pilot screen) is associated with the second sound zone (such as sound zone 1) of the first sound device (such as earphone 1), the audio played by the first sound device can be shared from the second screen to the first screen along with the multimedia interface corresponding to the audio, and migrated to the in-car speaker for playback. For example, the second screen displays the second interface of the first application. The second interface can be the playback interface of the first audio. The second interface can refer to the present application Fig.26 The multimedia playback interface of application 4 is shown. The first audio may be, for example, the audio of application 4. When an input operation of sharing the second interface from the second screen to the first screen is detected, the cockpit system may display the second interface on the first screen and switch the first audio from the first sound device to the in-car speaker for continued playback. Fig.26 and Fig. 27 The illustrated embodiment of sharing the multimedia playback interface of application 4 from the co-pilot screen to the central control screen is introduced.

[0479] In the scenario where the second interface of the first application is migrated from the second screen to the first screen, the cockpit system can migrate the audio focus of the first application in the audio focus stack, migrate the application information of the first application in the audio routing table, and migrate the media session of the first application in the media session stack. Fig.25 The example shown is an introduction to an embodiment of migrating application 4.

[0480] In some embodiments, the vehicle further includes a third screen. For example, the third screen may be a rear screen. The third screen is associated with a different sound zone from the first screen and a different sound zone from the second screen. For example, the third screen is associated with a third sound zone of a second sound device. The second sound device may be earphone 2. The third sound zone may be sound zone 2. When an input operation to share the second interface from the second screen to the third screen is detected, the cockpit system may display the second interface on the third screen. For details, please refer to Fig.26 and Fig. 27 As shown, after the co-pilot screen shares the multimedia playback interface of Application 4 with the rear screen, the rear screen displays the multimedia playback interface of Application 4.

[0481] The above-mentioned input operation of sharing the second interface from the second screen to the first screen and the input operation of sharing the second interface from the second screen to the third screen may be the same input operation. Fig.28B The operations for all screen options are shown. Optionally, the input operation of sharing the second interface from the second screen to the first screen and the input operation of sharing the second interface from the second screen to the third screen can also be two separate operations.

[0482] In some embodiments, after the second screen shares the second interface with the first screen and / or the third screen, when detecting that the fourth application on the second screen requests to play the fourth audio, the cockpit system can play the fourth audio through the first sound device. Fig.28E The fourth audio may be music A of the music APP. When it is detected that the fifth application on the third screen requests to play the fifth audio, the cockpit system may play the fifth audio through the second sound device. For example, the fifth application may be the music A of the music APP. Fig.28F The fifth audio may be the music B of the music zone APP1. When the sixth application on the first screen requests to play the sixth audio, the cockpit system may play the sixth audio through the in-car speakers and pause the first audio. For example, the sixth application may be the music B of the music zone APP1. Fig.28D The sixth audio may be an incoming call reminder tone of the phone APP.

[0483] It is understandable that the various user interfaces described in the embodiments of the present application are only example interfaces and do not limit the present application scheme. In other embodiments, the user interface may adopt a different interface layout, may include more or fewer controls, and may increase or decrease other functional options, as long as they are based on the same inventive concept provided by the present application, they are all within the scope of protection of the present application.

[0484] It should be noted that, without causing any contradiction or conflict, any feature in any embodiment of the present application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of the present application.

[0485] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A cockpit multi-sound zone management method, It is characterized in that The method is applied to a vehicle, the vehicle includes an in-vehicle speaker and multiple screens, the multiple screens include a first screen and a second screen, the first screen and the second screen are both associated with a first sound zone corresponding to the in-vehicle speaker, and the method includes: detecting that a first application on the second screen requests to play a first audio, and playing the first audio through the in-vehicle speaker; Detecting that a second application on the first screen requests to play a second audio, playing the second audio through the in-vehicle speaker, and pausing the first audio or reducing the playing volume of the first audio; An input operation of connecting a first sound device to the second screen is detected, the vehicle is connected to the first sound device, and a second sound zone corresponding to the first sound device is created; wherein the sound zone associated with the second screen is changed from the first sound zone to the second sound zone; It is detected again that the first application requests to play the first audio, and the first audio is played through the first sound device.

2. The method according to claim 1, It is characterized in that A first configuration file is stored in the vehicle, and the first configuration file is used to indicate the correspondence between the multiple screens and the driving area, wherein any one of the multiple screens corresponds to a driving area, the sound zone associated with the screen corresponding to the same driving area remains consistent, and the sound zones associated with the screens corresponding to different driving areas are the same or different.

3. The method according to claim 1 or 2, It is characterized in that The vehicle includes a first audio focus stack corresponding to the first audio zone, the first audio focus stack including audio focus information of an application running in the foreground or background on a screen associated with the first audio zone; After detecting the input operation of connecting the first sound-emitting device to the second screen, the method further includes: A second audio focus stack corresponding to the second audio zone is created, and audio focus information of an application running in the foreground or background on the second screen is migrated from the first audio focus stack to the second audio focus stack.

4. The method according to any one of claims 1 to 3, It is characterized in that The vehicle includes a first audio routing table corresponding to the first audio zone, the first audio routing table includes application information of an application running in the foreground or the background on a screen associated with the first audio zone, and the first audio channel of the in-vehicle speaker is configured in the first audio routing table, for indicating that the audio of the application corresponding to the application information in the first audio routing table is routed to the first audio channel; After detecting the input operation of connecting the first sound-emitting device to the second screen, the method further includes: Creating a second audio routing table corresponding to the second audio zone, and migrating application information of applications running in the foreground or background on the second screen from the first audio routing table to the second audio routing table; A second audio channel of the first sound-emitting device is created, and the second audio channel is configured in the second audio routing table, wherein the second audio channel configured in the second audio routing table is used to indicate that the audio of the application corresponding to the application information in the second audio routing table is routed to the second audio channel.

5. The method according to any one of claims 1 to 4, It is characterized in that The vehicle includes a first media session stack corresponding to the first audio zone, the first media session stack including a media session of an application running in the foreground or the background on a screen associated with the first audio zone; After detecting the input operation of connecting the first sound-emitting device to the second screen, the method further includes: A second media session stack corresponding to the second audio zone is created, and a media session of an application running in the foreground or the background on the second screen is migrated from the first media session stack to the second media session stack.

6. The method according to claim 5, It is characterized in that Before detecting an input operation on the second screen connected to the first sound-emitting device, the method further includes: detecting an input operation for viewing audio playback information on the first screen, and displaying a first list on the first screen according to the first media session stack, wherein the first list includes the audio playback information indicated by the first media session stack; An input operation for viewing audio playback information on the second screen is detected, and the first list is displayed on the second screen according to the first media session stack.

7. The method according to claim 6, It is characterized in that The first list includes audio playback information of the first audio; and the method further includes: When the in-vehicle speaker plays the second audio and the first audio is paused, detecting an input operation on the first screen for the audio playback information of the first audio in the first list, wherein the input operation for the audio playback information of the first audio in the first list is used to play the first audio; The first audio is played through the in-vehicle speaker, and the second audio is paused.

8. The method according to any one of claims 5 to 7, It is characterized in that After detecting the input operation of connecting the first sound-emitting device to the second screen, the method further includes: An input operation for viewing audio playback information on the first screen is detected, and a second list is displayed on the first screen according to the first media session stack, wherein the second list includes the audio playback information indicated by the first media session stack, and the second list does not include the audio playback information of the first audio; An input operation for viewing audio playback information on the second screen is detected, and a third list is displayed on the second screen according to the second media session stack, wherein the third list includes the audio playback information indicated by the second media session stack, and the second list includes the audio playback information of the first audio.

9. The method according to any one of claims 1 to 8, It is characterized in that After the vehicle is connected to the first sound-generating device, the method further includes: An input operation on the second screen for adjusting the volume to a first volume is detected, and the volume of the audio played by the first sound device is adjusted to the first volume.

10. The method according to any one of claims 1 to 9, It is characterized in that After the vehicle is connected to the first sound-generating device, the method further includes: An input operation on the first screen for adjusting the volume to a second volume is detected, and the volume of the audio played by the in-vehicle speaker is adjusted to the second volume.

11. The method according to any one of claims 1 to 10, It is characterized in that The first screen displays a first interface of the second application, the first interface being a playback interface of the second audio, and after the vehicle is connected to the first sound device, the method further includes: An input operation of migrating the first interface from the first screen to the second screen is detected, the first interface is displayed on the second screen, the first interface is canceled from the first screen, and the second audio is switched from the in-vehicle speaker to the first sound device for continued playback.

12. The method according to claim 11, It is characterized in that After switching the second audio from the in-vehicle speaker to the first sound device for continued playback, the method further includes: It is detected that a third application on the first screen requests to play a third audio, and the third audio is played through the in-vehicle speaker.

13. The method according to claim 11 or 12, It is characterized in that After switching the second audio from the in-vehicle speaker to the first sound device for continued playback, the method further includes: It is detected that the first application on the second screen requests to play the first audio, plays the first audio through the first sound device, and pauses playing the second audio or reduces the playing volume of the second audio.

14. The method according to any one of claims 11 to 13, It is characterized in that The vehicle includes a first audio focus stack, a first audio routing table, and a first media session stack corresponding to the first audio zone, and a second audio focus stack, a second audio routing table, and a second media session stack corresponding to the second audio zone, the first audio focus stack including audio focus information of the second application, the first audio routing table including application information of the second application, and the first media session stack including a media session stack of the first application; After detecting the input operation of migrating the first interface from the first screen to the second screen, the method further includes: The audio focus information of the second application is migrated from the first audio focus stack to the second audio focus stack, the application information of the second application is migrated from the first audio routing table to the second audio routing table, and the media session of the second application is migrated from the first media session stack to the second media session stack.

15. The method according to any one of claims 1 to 14, It is characterized in that A second interface of the first application is displayed on the second screen, where the second interface is a playback interface of the first audio; After the first application is detected again to request to play the first audio, and the first audio is played through the first sound device, the method further includes: An input operation of sharing the second interface from the second screen to the first screen is detected, the second interface is displayed on the first screen, and the first audio is switched from the first sound device to the in-vehicle speaker for continued playback.

16. The method according to claim 15, It is characterized in that The vehicle further includes a third screen, the third screen is associated with a third sound zone of a second sound device, the second sound device is connected to the vehicle, and the method further includes: An input operation of sharing the second interface from the second screen to the third screen is detected, and the second interface is displayed on the third screen.

17. The method according to claim 15 or 16, It is characterized in that After switching the first audio from the first sound-emitting device to the in-vehicle speaker for continued playback, the method further includes: It is detected that the fourth application on the second screen requests to play the fourth audio, and the fourth audio is played through the first sound device.

18. The method according to claim 16, It is characterized in that The method further comprises: It is detected that the fifth application on the third screen requests to play the fifth audio, and the fifth audio is played through the second sound device.

19. The method according to any one of claims 15 to 18, It is characterized in that After switching the first audio from the first sound-emitting device to the in-vehicle speaker for continued playback, the method further includes: It is detected that the sixth application on the first screen requests to play the sixth audio, the sixth audio is played through the in-vehicle speaker, and the first audio is paused.

20. The method according to any one of claims 15 to 19, It is characterized in that The vehicle includes a first audio focus stack and a first audio routing table corresponding to the first audio zone, and a second audio focus stack and a second audio routing table corresponding to the second audio zone, wherein the second audio focus stack includes audio focus information of the first application, and the second audio routing table includes application information of the first application; After detecting the input operation of sharing the second interface from the second screen to the first screen, the method further includes: The audio focus information of the first application is migrated from the second audio focus stack to the first audio focus stack, and the application information of the first application is migrated from the second audio routing table to the first audio routing table.

21. The method according to any one of claims 1 to 20, It is characterized in that After the vehicle is connected to the first sound-generating device, the method further includes: detecting that the first sound device is disconnected, changing the sound zone associated with the second screen from the second sound zone to the first sound zone; When the in-car speaker is playing the second audio, it is detected that the first application on the second screen requests to play the first audio, the first audio is played through the in-car speaker, and the second audio is paused or the volume of the second audio is reduced.

22. The method according to claim 21, It is characterized in that The vehicle includes a first audio focus stack corresponding to the first sound zone and a second audio focus stack corresponding to the second sound zone. After detecting that the first sound device is disconnected, the method further includes: The audio focus information in the second audio focus stack is migrated to the first audio focus stack, and the second audio focus stack is deleted.

23. The method according to claim 21 or 22, It is characterized in that The vehicle includes a first audio routing table corresponding to the first audio zone and a second audio routing table corresponding to the second audio zone. After detecting that the first sound device is disconnected, the method further includes: The application information in the second audio routing table is migrated to the first audio routing table, and the second audio routing table is deleted.

24. The method according to any one of claims 21 to 23, It is characterized in that The vehicle includes a first media session stack corresponding to the first audio zone and a second media session stack corresponding to the second audio zone. After detecting that the first sound device is disconnected, the method further includes: Migrate the media session in the second media session stack to the first media session stack, and delete the second media session stack.

25. The method according to any one of claims 1 to 24, It is characterized in that The vehicle further includes a fourth screen, the fourth screen being consistent with a sound zone associated with the second screen, and after the first application requesting to play the first audio is detected again and the first audio is played through the first sound device, the method further includes: It is detected that the seventh application on the fourth screen requests to play the seventh audio, the seventh audio is played through the first sound device, and the first audio is paused or the playback volume of the seventh audio is reduced.

26. A vehicle, It is characterized in that The vehicle includes multiple screens, in-vehicle speakers, a memory and a processor, wherein the screen is used to display a user interface; the in-vehicle speakers are used to play audio; the processor is used to store a computer program; and the processor is used to call the computer program so that the vehicle executes the method described in any one of claims 1-25.

27. A computer-readable storage medium storing instructions, It is characterized in that When the instructions are executed on a vehicle, the vehicle executes the method according to any one of claims 1-25.

28. A computer program product, It is characterized in that The computer program product comprises computer instructions, and when the computer instructions are run on a vehicle, the vehicle is caused to execute the method according to any one of claims 1 to 25.

Images