Audio arbitration interaction method and device, vehicle and storage medium

Abstract

The invention relates to the technical field of vehicles, in particular to an audio arbitration interaction method and device, a vehicle and a storage medium, and the method comprises the steps: receiving an audio focus request of at least one target audio system and an audio focus of at least one other audio system in a use state; and identifying audio priorities of each target audio system and at least one other audio system in the use state, generating an audio priority identification result, if the highest audio priority exists in the at least one target audio system, allocating a target audio focus to the target audio system corresponding to the highest audio priority, and if the highest audio priority exists in the at least one target audio system, allocating a target audio focus to the target audio system corresponding to the highest audio priority. And adding the target audio focus into a focus stack, and removing the preempted audio focus of the at least one other audio system at the same time, so as to control the target audio system to perform audio playing based on the target audio focus. Therefore, the problems of audio focus conflict, asynchronous audio focus distribution, incapability of determining recovery of the audio focus after the audio focus is preempted and the like in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to an audio arbitration interaction method, device, vehicle and storage medium. Background Art

[0002] With the rapid development of smart cockpit technology, more and more electronic devices and operating systems are integrated into vehicles. These systems include not only traditional dashboards and infotainment systems, but also various assisted driving systems, communication modules, environmental perception systems, etc. Each system has its own audio requirements. For example, the dashboard is used to display vehicle status information (such as speed, fuel consumption) and issue corresponding prompts, and the assisted driving system is used to issue safety warnings or voice prompts related to autonomous driving. Therefore, a flexible and reliable audio management system is needed to realize audio source arbitration interaction between multiple operating systems.

[0003] In the related art, a centralized audio focus management method is usually adopted, that is, a main audio system is set to be responsible for the arbitration of all audio requests. When other systems need to play audio, they must send a request to the main system and wait for approval before playing.

[0004] However, the above method has the following problems: (1) Single point failure: If the main audio system fails or crashes, the entire audio management system may fail, resulting in all audio requests being unable to be processed normally; (2) Lack of mandatory nature: Some applications may ignore the arbitration results and insist on audio output, resulting in sound confusion; (3) Simple interaction: There is a lack of complex interaction logic. When a certain audio source is preempted, the system cannot intelligently decide whether to lower the volume, pause, or completely stop playing, nor can it cope with situations such as system crashes or network interruptions. This needs to be solved urgently. Summary of the invention

[0005] The present application provides an audio arbitration interaction method, device, vehicle and storage medium to solve the problems existing in the related art, such as audio focus conflict, asynchronous audio focus allocation and inability to determine the recovery of audio focus after preemption.

[0006] The first aspect of the present application provides an audio arbitration interaction method, comprising the following steps:

[0007] determining whether an audio focus request of at least one target audio system is received;

[0008] If an audio focus request of the at least one target audio system is received, determining whether the audio focus of at least one other audio system is in use; if the audio focus of the at least one other audio system is in use, identifying the audio priority of each target audio system and the audio priority of the at least one other audio system in use, and generating audio priority identification results for each target audio system and the at least one other audio system;

[0009] Based on the audio priority identification result, if there is a highest audio priority in at least one target audio system, a target audio focus is directly allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that has been preempted is removed, so as to control the target audio system to perform audio playback based on the target audio focus.

[0010] According to one embodiment of the present application, after allocating a target audio focus to the target audio system corresponding to the highest audio priority, the method further includes:

[0011] Based on the target audio focus of the target audio system, sending the target audio focus to the at least one other audio system, and determining whether a feedback result from the at least one other audio system is received;

[0012] If feedback results from at least one other audio system are received, the target audio focus is added to the focus stack, and the target audio system is controlled to perform audio playback based on the target audio focus; otherwise, it is determined that the target audio focus has failed to be added to the focus stack, and the control of the target audio system to perform audio playback based on the target audio focus is stopped.

[0013] According to an embodiment of the present application, the identifying the audio priority of each target audio system and the audio priority of the at least one other audio system in use includes:

[0014] A current driving scene of the vehicle is identified based on the current driving state and current driving environment of the vehicle, and an audio priority of each target audio system and an audio priority of the at least one other audio system in use are identified based on the current driving scene.

[0015] According to one embodiment of the present application, the identifying the audio priority of each target audio system and the audio priority of the at least one other audio system in use further includes:

[0016] Obtain the user's vehicle driving habits;

[0017] A priority definition rule for the user is generated based on the vehicle driving habits of the user, and the audio priority of each target audio system and the audio priority of the at least one other audio system in use are identified based on the priority definition rule.

[0018] According to one embodiment of the present application, before controlling the target audio system to play audio based on the target audio focus, the method further includes:

[0019] Creating an audio buffer corresponding to the target audio system, and determining whether the target audio system has a target audio focus;

[0020] If the target audio system has the target audio focus, the audio data of the target audio system is written to the audio buffer using the application interface corresponding to the target audio system; otherwise, the audio data is refused to be written and an error code is returned to control the target audio system to stop audio playback.

[0021] According to an embodiment of the present application, the above-mentioned audio arbitration interaction method further includes:

[0022] Determine whether a heartbeat packet of each target audio system and each other audio system is received within a preset time;

[0023] If the heartbeat packet of the at least one target audio system or the at least one other audio system is not received within the preset time, it is determined that the at least one target audio system or the at least one other audio system is in an abnormal state, and the focus stack is traversed to remove the target audio focus corresponding to the at least one target audio system in the abnormal state or the audio focus corresponding to the at least one other audio system from the focus stack, and the target audio focus of the at least one target audio system or the audio focus of the at least one other audio system is updated.

[0024] According to the audio arbitration interaction method of the embodiment of the present application, when an audio focus request of at least one target audio system is received and the audio focus of at least one other audio system is in use, the audio priority of each target audio system and at least one other audio system in use is identified, and an audio priority identification result is generated. If there is a highest audio priority in at least one target audio system, a target audio focus is allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that has been preempted is removed, so as to control the target audio system to play audio based on the target audio focus. Thus, the problems of audio focus conflict, asynchronous audio focus allocation, and inability to determine the recovery of audio focus after preemption that exist in the related art are solved. By implementing distributed arbitration logic between various audio systems, the audio focus is closely associated with the writing of the audio data stream, so that audio output can only be performed when the system obtains the audio focus, thereby realizing a more robust, flexible and reliable audio management system.

[0025] A second aspect of the present application provides an audio arbitration interaction device, including:

[0026] A determination module, used to determine whether an audio focus request of at least one target audio system is received;

[0027] a generating module, configured to, if receiving an audio focus request of the at least one target audio system, determine whether the audio focus of at least one other audio system is in use, and if the audio focus of the at least one other audio system is in use, identify the audio priority of each target audio system and the audio priority of the at least one other audio system in use, and generate audio priority identification results of each target audio system and the at least one other audio system;

[0028] A control module is used to allocate a target audio focus directly to the target audio system corresponding to the highest audio priority based on the audio priority identification result, if there is a highest audio priority in the at least one target audio system, and add the target audio focus to the focus stack, and remove the audio focus of the at least one other audio system that has been preempted, so as to control the target audio system to perform audio playback based on the target audio focus.

[0029] According to one embodiment of the present application, after allocating the target audio focus to the target audio system corresponding to the highest audio priority, the control module is further configured to:

[0030] Based on the target audio focus of the target audio system, sending the target audio focus to the at least one other audio system, and determining whether a feedback result from the at least one other audio system is received;

[0031] If feedback results from at least one other audio system are received, the target audio focus is added to the focus stack, and the target audio system is controlled to perform audio playback based on the target audio focus; otherwise, it is determined that the target audio focus has failed to be added to the focus stack, and the control of the target audio system to perform audio playback based on the target audio focus is stopped.

[0032] According to one embodiment of the present application, the generating module is specifically used for:

[0033] A current driving scene of the vehicle is identified based on the current driving state and current driving environment of the vehicle, and an audio priority of each target audio system and an audio priority of the at least one other audio system in use are identified based on the current driving scene.

[0034] According to one embodiment of the present application, the generating module is specifically used for:

[0035] Obtain the user's vehicle driving habits;

[0036] A priority definition rule for the user is generated based on the vehicle driving habits of the user, and the audio priority of each target audio system and the audio priority of the at least one other audio system in use are identified based on the priority definition rule.

[0037] According to an embodiment of the present application, before controlling the target audio system to play audio based on the target audio focus, the control module is further configured to:

[0038] Creating an audio buffer corresponding to the target audio system, and determining whether the target audio system has a target audio focus;

[0039] If the target audio system has the target audio focus, the audio data of the target audio system is written to the audio buffer using the application interface corresponding to the target audio system; otherwise, the audio data is refused to be written and an error code is returned to control the target audio system to stop audio playback.

[0040] According to one embodiment of the present application, the above-mentioned audio arbitration interaction device is further used for:

[0041] Determine whether a heartbeat packet of each target audio system and each other audio system is received within a preset time;

[0042] If the heartbeat packet of the at least one target audio system or the at least one other audio system is not received within the preset time, it is determined that the at least one target audio system or the at least one other audio system is in an abnormal state, and the focus stack is traversed to remove the target audio focus corresponding to the at least one target audio system in the abnormal state or the audio focus corresponding to the at least one other audio system from the focus stack, and the target audio focus of the at least one target audio system or the audio focus of the at least one other audio system is updated.

[0043] According to the audio arbitration interaction device of the embodiment of the present application, when receiving an audio focus request of at least one target audio system and the audio focus of at least one other audio system is in use, the audio priority of each target audio system and at least one other audio system in use is identified, and an audio priority identification result is generated. If there is a highest audio priority in at least one target audio system, a target audio focus is allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that has been preempted is removed, so as to control the target audio system to play audio based on the target audio focus. Thus, the problems of audio focus conflict, asynchronous audio focus allocation, and inability to determine the recovery of audio focus after preemption that exist in the related art are solved. By implementing distributed arbitration logic between various audio systems, the audio focus is closely associated with the writing of the audio data stream, so that audio output can only be performed when the system obtains the audio focus, thereby realizing a more robust, flexible and reliable audio management system.

[0044] A third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the audio arbitration interaction method as described in the above embodiment.

[0045] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the audio arbitration interaction method as described in the above embodiment.

[0046] A fifth aspect of the present application provides a computer program product, including a computer program, which is executed to implement the audio arbitration interaction method described in the above embodiment.

[0047] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0049] Figure 1 A flowchart of an audio arbitration interaction method provided according to an embodiment of the present application;

[0050] Figure 2 is a schematic diagram of a system module according to an embodiment of the present application;

[0051] Figure 3 A schematic diagram of a process of applying for focus in a multi-system situation according to an embodiment of the present application;

[0052] Figure 4 A schematic diagram of a data structure used for multi-system communication according to an embodiment of the present application;

[0053] Figure 5 A schematic diagram of a flow chart of focus release in a multi-system situation according to an embodiment of the present application;

[0054] Figure 6 A schematic diagram of combining result data stream channels of audio arbitration according to an embodiment of the present application;

[0055] Figure 7 A schematic diagram of a process flow for processing an abnormal system crash according to an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the data structure of a multi-system heartbeat packet according to an embodiment of the present application;

[0057] Fig. 9 is an exemplary diagram of an audio arbitration interaction device according to an embodiment of the present application;

[0058] Fig.10 It is a schematic diagram of the structure of a vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0060] The following describes the audio arbitration interaction method, device, vehicle and storage medium of the embodiment of the present application with reference to the accompanying drawings. In view of the problems of audio focus conflict, asynchronous audio focus allocation and inability to determine the recovery of audio focus after preemption in the related art mentioned in the above background technology, the present application provides an audio arbitration interaction method, in which, when receiving an audio focus request of at least one target audio system and the audio focus of at least one other audio system is in use, the audio priority of each target audio system and at least one other audio system in use is identified, and an audio priority identification result is generated. If there is a highest audio priority in at least one target audio system, the target audio focus is allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that is preempted is removed, so as to control the target audio system to play audio based on the target audio focus. Thus, the problems of audio focus conflict, asynchronous audio focus allocation and inability to determine the recovery of audio focus after preemption in the related art are solved. By implementing distributed arbitration logic between each audio system, the audio focus is closely associated with the writing of the audio data stream, so that audio output can only be performed when the system obtains the audio focus, thereby realizing a more robust, flexible and reliable audio management system.

[0061] Specifically, Figure 1 A flowchart of an audio arbitration interaction method provided in an embodiment of the present application is provided.

[0062] like Figure 1 As shown, the audio arbitration interaction method includes the following steps:

[0063] In step S101 , it is determined whether an audio focus request of at least one target audio system is received.

[0064] Specifically, since there are multiple operating systems in the vehicle, and these operating systems have their own audio systems and need to make sounds, but there is only one audio system, if it is not controlled, each audio system can play audio at will, which will cause sound confusion. Therefore, it is necessary to uniformly manage the audio system of the entire vehicle system to avoid sound confusion and ensure that important audio information is played first. To this end, the embodiment of the present application is based on a distributed audio focus management mechanism to solve the above problem.

[0065] Specifically, if Figure 2As shown, the embodiment of the present application mainly includes three modules, namely, an adaptation module, an arbitration module and a communication module, wherein the adaptation module provides a unified access interface and a control interface, so that the audio systems of different operating systems can be connected to the arbitration system. In some operating systems, if there is already an operation interface related to audio source arbitration, it can be directly reused to connect the system through the adaptation module. If there is no operation interface related to audio source arbitration, the fixed arbitration interface provided by the system can be used, and this module is not required; the arbitration module is the core module of the system, which arbitrates according to the policy in the audio policy table to determine whether the currently applied application can be played. If it can be played, the focus is obtained, and the focus is added to the focus stack for memory for the next arbitration, policy management The audio policy table will be loaded when the system starts. This audio policy table can be managed in the cloud or pre-fabricated in the system. If it is pre-fabricated in the system, as the system is upgraded and iterated, the audio policy table version in each system may be different. Therefore, at startup, each audio system needs to handshake. If the audio policy table of its own system is the latest, the audio policy table will be distributed to all systems. The main task of the communication module is to synchronize data during system initialization and arbitration. It uses Ethernet-based communications such as SomeIP, FDBus, and FastDDS for communication. The serialization method can use the cross-language protobuf protocol to ensure that all systems use the same audio strategy and perform necessary data exchange during the arbitration process.

[0066] Therefore, when managing audio in a multi-system environment, Figure 3 As shown, first, after the system is initialized, multiple systems need to synchronize audio policy tables with each other, with the latest version of the audio policy table taking precedence. Then, it is determined whether an audio focus request from at least one target audio system is received. That is, when a target audio system application module needs to play audio, an audio focus request is applied for, and then audio priority is determined based on the audio focus request.

[0067] Among them, the audio strategy table can be directly embedded in the firmware or software of each audio system when the system leaves the factory, that is, common audio usage scenarios (such as navigation prompts, phone calls, entertainment systems, etc.) are taken into account. The pre-made strategy table can be loaded when the system starts and take effect immediately, or it can be obtained from the cloud, and the vehicle can download the latest audio strategy table regularly through a network connection, so that the audio strategy can be dynamically adjusted according to different markets, models, user preferences and other factors to ensure that the system always uses the most optimized configuration. Alternatively, it can be combined with the local and cloud. When the system starts, the system first loads the local pre-made audio strategy table, and then tries to download the latest version from the cloud. If the cloud version is newer than the local version, the local audio strategy table is replaced. If it cannot connect to the cloud, the local audio strategy table continues to be used. Alternatively, it can be customized based on the user's preferences. For example, the user can choose to play navigation prompts first when driving, and entertainment content first when parking. This personalized configuration can be set through the in-vehicle application or mobile phone application and saved locally or in the cloud. It can be selected according to the actual use needs of the vehicle, and is not specifically limited here.

[0068] In step S102, if an audio focus request of at least one target audio system is received, it is determined whether the audio focus of at least one other audio system is in use. If the audio focus of at least one other audio system is in use, the audio priority of each target audio system and the audio priority of at least one other audio system in use are identified, and audio priority identification results are generated for each target audio system and at least one other audio system.

[0069] According to one embodiment of the present application, identifying the audio priority of each target audio system and the audio priority of at least one other audio system in use includes: identifying a current driving scene of the vehicle based on the current driving state and the current driving environment of the vehicle, and identifying the audio priority of each target audio system and the audio priority of at least one other audio system in use based on the current driving scene.

[0070] According to one embodiment of the present application, identifying the audio priority of each target audio system and the audio priority of at least one other audio system in use also includes: obtaining the user's vehicle driving habits; generating the user's priority definition rules based on the user's vehicle driving habits, and identifying the audio priority of each target audio system and the audio priority of at least one other audio system in use based on the priority definition rules.

[0071] Specifically, after at least one target audio system application module needs to play audio and applies for an audio focus request, if an audio focus request of at least one target audio system is received, it is determined whether the audio focus of at least one other audio system is in use. If the audio focus of at least one other audio system is in use, directly playing the audio focus of the target audio system may cause sound confusion, affect the user experience, and may even cause important safety information to be ignored. Therefore, the system needs a mechanism to coordinate these audio requests to ensure that important information is played first while reasonably handling other audio sources. Therefore, it is necessary to identify the audio priority of each received target audio system and the audio priority of at least one other audio system in use to determine whether to grant audio focus. If audio focus is granted, the application can play audio normally. If audio focus is not obtained, the system may refuse to play, or require the application to lower the volume or pause playback.

[0072] Specifically, first, it is necessary to determine the audio priority of each target audio system and the audio priority of at least one other audio system in use. Since the priority of the audio focus determines the preemption relationship between different audio sources, the system usually sets the priority according to the type and importance of the audio content. The current driving scene of the vehicle can be identified based on the current driving state and current driving environment of the vehicle. For example, safety-related audio (such as collision warnings, emergency braking prompts, navigation key prompts, etc.) must be played first to ensure driving safety and can be set to the highest priority; communication-related audio (such as phone calls, text message notifications, voice assistant responses, etc.) usually needs to be processed in a timely manner to avoid missing important information and can be set to high priority; navigation and assisted driving-related audio (such as route prompts, lane departure warnings, etc.) are not as urgent as safety prompts, but are still very important during driving and can be set to medium priority; entertainment-related audio (such as music, video, broadcast, etc.) can be paused or lowered when necessary to make way for more important audio and can be set to low priority.

[0073] For example, when driving on a highway, navigation prompts may have a higher priority than entertainment music because the driver needs to focus more on the road at this time. The system may automatically lower the volume of background music to ensure that the navigation prompts are clear and audible; when the vehicle detects potential dangers (such as obstacles ahead, lane departure, etc.), the priority of safety prompts will be immediately increased, and all other audio will be paused or lowered to ensure that the driver can focus on safety prompts.

[0074] Optionally, an embodiment of the present application may also generate a user's priority definition rules based on the user's vehicle driving habits, and identify the audio priority of each target audio system and the audio priority of at least one other audio system in use based on the priority definition rules. When multiple target audio systems request audio focus at the same time, active identification can be performed based on the user-defined priority definition rules to determine the highest priority audio focus request, wherein the priority definition rules can be dynamically updated according to the user's settings.

[0075] In step S103, based on the audio priority recognition result, if there is a highest audio priority in at least one target audio system, a target audio focus is directly allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that has been preempted is removed, so as to control the target audio system to perform audio playback based on the target audio focus.

[0076] According to one embodiment of the present application, before controlling the target audio system to perform audio playback based on the target audio focus, it also includes: creating an audio buffer corresponding to the target audio system, and determining whether the target audio system has a target audio focus; if the target audio system has a target audio focus, using the application interface corresponding to the target audio system to write the audio data of the target audio system to the audio buffer, otherwise, refusing to write the audio data, and returning an error code to control the target audio system to stop audio playback.

[0077] According to one embodiment of the present application, after allocating a target audio focus to the target audio system corresponding to the highest audio priority, it also includes: based on the target audio focus of the target audio system, sending the target audio focus to at least one other audio system, and determining whether feedback results from at least one other audio system are received; if feedback results from at least one other audio system are received, adding the target audio focus to the focus stack, and controlling the target audio system to play audio based on the target audio focus; otherwise, determining that the target audio focus has failed to be added to the focus stack, and stopping controlling the target audio system to play audio based on the target audio focus.

[0078] Specifically, Figure 3 and Figure 4As shown, if there is a highest audio priority in at least one target audio system, that is, after receiving an audio focus request from at least one target audio system and there is no audio focus of at least one other audio system in use, the arbitration module will identify the audio priority of each received target audio system and identify the audio focus request with the highest priority, and the audio focus request with the highest priority will be approved for priority use. For example, if there are collision warning, SMS notification or broadcast audio focus requests at the same time, since the collision warning is the highest safety-related audio priority, the audio focus request of the collision warning will be approved first, and the target audio focus will be directly allocated to the target audio system corresponding to the highest audio priority, and the target audio focus will be added to the focus stack; if an audio focus request from at least one target audio system is received and there is at least one other audio system in use, the arbitration module will identify the audio priority of each received target audio system and the audio priority of at least one other audio system in use. If the audio focus request with the highest priority is identified as a newly applied target audio system, the audio arbitration result is not rejected, and a notification is sent to other audio systems (the message content is as shown in FIG. 1 ). Figure 4 As shown in the figure, request_type is request focus). After receiving the notification, other audio systems will add the target audio focus to the focus stack at the same time, remove other audio focuses occupied by the target audio focus, and pause the sound sources in other audios. Then the target audio system will collect the feedback results of other audio systems. If there is a failure, the whole process will end with failure, and the control of the target audio system for audio playback based on the target audio focus will be stopped. Otherwise, the application is considered successful, and the control of the target audio system for audio playback is allowed.

[0079] Further, if Figure 5 As shown, if the application module of a target audio system no longer needs to play a certain audio, first, it needs to apply to release the audio focus. This operation is initiated by the target audio system where the application module is located. The target audio system removes the target audio focus in the focus stack and sends a notification to other audio systems (the message content is as follows Figure 4 As shown, request_type is release focus), this target audio focus needs to be removed; secondly, start traversing the list of temporarily lost target audio focus, if there is a target audio focus that needs to be restored, restore it to the foreground, this operation is still performed in the target audio system; finally, send a notification to other audio systems that this focus is activated, and notify the application that applied to restore the audio focus that the audio focus has been regained and the audio can be played.

[0080] Among them, the focus stack can be divided into the following types: activated focus stack, temporarily lost focus stack, delayed acquisition focus stack, and permanently lost focus stack. According to the result of audio arbitration, the operations on the focus stack are as follows: Focus that does not allow preemption: do not operate the focus stack and keep the original data structure; Long-term focus that allows preemption: remove the audio focus that does not coexist with the newly applied audio focus in the activated focus stack and the temporarily lost focus stack, and add the newly applied audio focus to the activated focus stack; Short-term focus that allows preemption: remove the focus that does not coexist with the newly applied audio focus in the activated focus stack. If these focuses are long focuses, they also need to be added to the temporarily lost stack. After the newly applied audio focus is released, the temporarily lost audio focus is restored; Concurrent interaction: directly add it to the activated focus stack, and then perform logical judgment on the volume reduction, whether the former reduces the volume, the latter reduces the volume, or does not reduce the volume; Delayed focus: add it to the delayed focus stack, wait for the release of the activated focus stack to change, and then determine whether to activate this focus.

[0081] Further, such as Figure 6 As shown, in order to avoid the situation where the application does not follow the result of the audio source arbitration and insists on audio output, which will still cause sound confusion, the embodiment of the present application combines the result of the audio source arbitration with the data stream channel, and then creates an audio buffer corresponding to the target audio system, and determines whether the target audio system has a target audio focus. If the target audio system has a target audio focus, the audio data of the target audio system is written to the audio buffer using the application interface corresponding to the target audio system. Otherwise, the audio data is refused to be written, and an error code is returned to control the target audio system to stop audio playback.

[0082] That is to say, when the target audio system needs to play a certain audio, first, call its system playback API (Application Programming Interface), and the application requests to create an audio buffer inside AudioFlinger, namely AudioTrack. Other types of audio systems can select an entry point in their audio link to add this logic; secondly, after AudioTrack is successfully created, write data. Each time data is written, check the holding of the target audio focus. If the playback process does not have the corresponding target audio focus, the target audio focus is rejected. Otherwise, the normal data writing process is followed. After refusing to write for 5000ms, if the target audio focus is still not available, an exception is thrown and an error code is returned to control the target audio system to stop audio playback. Therefore, after the above steps, if the target audio focus is not applied for, the sound will definitely not be played. However, for a small number of strong mixing applications (sound sources that can be mixed with any sound source on demand), it may not be necessary to apply for focus. This application system also respects its needs and defines special USAGE types for it. When it uses these USAGEs for playback, it will not judge whether the focus is held.

[0083] According to an embodiment of the present application, the above-mentioned audio arbitration interaction method further includes: determining whether a heartbeat packet of each target audio system and each other audio system is received within a preset time; if a heartbeat packet of at least one target audio system or at least one other audio system is not received within the preset time, determining that at least one target audio system or at least one other audio system is in an abnormal state, and traversing the focus stack, removing the target audio focus corresponding to the at least one target audio system in the abnormal state or the audio focus corresponding to the at least one other audio system from the focus stack, and updating the target audio focus of the at least one target audio system or the audio focus of the at least one other audio system.

[0084] The preset time may be set by those skilled in the art according to actual test requirements, or may be obtained through a limited number of computer simulations, and is not specifically limited here.

[0085] Specifically, in order to ensure that the target audio system can maintain high availability and fault tolerance when it is in a crash state, the embodiment of the present application uses a heartbeat packet mechanism and focus stack consistency check to enable a rapid response when a target audio system crashes, ensuring that other audio systems continue to operate normally and maintaining consistency in audio management.

[0086] Specifically, if Figure 7 and Figure 8As shown, when the target audio system operates normally, a heartbeat packet is sent periodically, for example, a heartbeat packet is sent to other audio systems once every 2s, mainly including system status information, the current target audio focus number (foscus_number) and the status of the focus stack (foscus_stack). The target audio focus number indicates the audio focus number currently held by the target audio system, which is used to identify whether the target audio system has a target audio focus. The status of the focus stack indicates the focus stack status of the target audio system, which records the history of the target audio focus currently held or previously held by the target audio system, which helps other audio systems verify the consistency of the focus stack, so that the status of each audio system can be monitored in real time by sending heartbeat packets periodically, and abnormal situations can be discovered in time.

[0087] Furthermore, when the target audio system crashes and enters the restart process, it will stop sending heartbeat packets. Other audio systems will detect their abnormal state because they cannot receive the heartbeat packets of the target audio system. If other audio systems still have not received the heartbeat packets after a preset time (for example, 5 seconds), the system will determine that the target audio system may have crashed, and remove the target audio system from the system list, and simultaneously remove the target audio focus of the target audio system in the focus stack. This ensures that other audio systems will not continue to rely on the crashed target audio system, thereby avoiding audio management confusion. At this time, other audio systems will continue to apply for and release audio focus according to the normal process, and the system's audio management logic will not be affected, thereby ensuring the continuity of the user experience.

[0088] Therefore, through the above steps, after a target audio system crashes, other audio systems can still cooperate with each other and will not crash due to the crash of a target audio system. At the same time, the foscus_number and foscus_stack fields in the heartbeat packet can also be used to verify whether the target audio system is consistent with the focus stack of other audio systems. If not, the corresponding fault-tolerant mechanism is triggered to automatically repair the error, which not only improves the robustness of the system, but also enhances the collaboration ability between multiple audio systems, thereby ensuring the stability and reliability of audio management in complex environments.

[0089] According to the audio arbitration interaction method of the embodiment of the present application, when an audio focus request of at least one target audio system is received and the audio focus of at least one other audio system is in use, the audio priority of each target audio system and at least one other audio system in use is identified, and an audio priority identification result is generated. If there is a highest audio priority in at least one target audio system, a target audio focus is allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that has been preempted is removed, so as to control the target audio system to play audio based on the target audio focus. Thus, the problems of audio focus conflict, asynchronous audio focus allocation, and inability to determine the recovery of audio focus after preemption that exist in the related art are solved. By implementing distributed arbitration logic between various audio systems, the audio focus is closely associated with the writing of the audio data stream, so that audio output can only be performed when the system obtains the audio focus, thereby realizing a more robust, flexible and reliable audio management system.

[0090] Next, the audio arbitration interaction device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0091] Fig. 9 4 is a block diagram of an audio arbitration interaction device according to an embodiment of the present application.

[0092] like Fig. 9 As shown, the audio arbitration interaction device 10 includes: a determination module 100 , a generation module 200 and a control module 300 .

[0093] The determination module 100 is used to determine whether an audio focus request of at least one target audio system is received;

[0094] A generating module 200, configured to, if an audio focus request of at least one target audio system is received, determine whether the audio focus of at least one other audio system is in use, and if the audio focus of at least one other audio system is in use, identify the audio priority of each target audio system and the audio priority of at least one other audio system in use, and generate audio priority identification results for each target audio system and at least one other audio system;

[0095] The control module 300 is used to directly allocate a target audio focus to the target audio system corresponding to the highest audio priority based on the audio priority recognition result, if there is a highest audio priority in at least one target audio system, and add the target audio focus to the focus stack, and at the same time remove the audio focus of at least one other audio system that has been preempted, so as to control the target audio system to play audio based on the target audio focus.

[0096] According to one embodiment of the present application, after allocating the target audio focus to the target audio system corresponding to the highest audio priority, the control module 300 is further configured to:

[0097] Based on the target audio focus of the target audio system, sending the target audio focus to at least one other audio system, and determining whether a feedback result of the at least one other audio system is received;

[0098] If feedback results from at least one other audio system are received, the target audio focus is added to the focus stack, and the target audio system is controlled to play audio based on the target audio focus; otherwise, it is determined that the target audio focus has failed to be added to the focus stack, and the control of the target audio system to play audio based on the target audio focus is stopped.

[0099] According to one embodiment of the present application, the generating module 200 is specifically used for:

[0100] A current driving scene of the vehicle is identified based on the current driving state and current driving environment of the vehicle, and an audio priority of each target audio system and an audio priority of at least one other audio system in use are identified based on the current driving scene.

[0101] According to one embodiment of the present application, the generating module 200 is specifically used for:

[0102] Obtain the user's vehicle driving habits;

[0103] A priority definition rule of the user is generated based on the vehicle driving habits of the user, and an audio priority of each target audio system and an audio priority of at least one other audio system in use are identified based on the priority definition rule.

[0104] According to one embodiment of the present application, before controlling the target audio system to play audio based on the target audio focus, the control module 300 is further configured to:

[0105] Create an audio buffer corresponding to the target audio system, and determine whether the target audio system has a target audio focus;

[0106] If the target audio system has a target audio focus, the audio data of the target audio system is written to the audio buffer using the application interface corresponding to the target audio system. Otherwise, the audio data is rejected and an error code is returned to control the target audio system to stop audio playback.

[0107] According to an embodiment of the present application, the above-mentioned audio arbitration interaction device 10 is further used for:

[0108] Determine whether a heartbeat packet of each target audio system and each other audio system is received within a preset time;

[0109] If the heartbeat packet of at least one target audio system or at least one other audio system is not received within the preset time, it is determined that the at least one target audio system or at least one other audio system is in an abnormal state, and the focus stack is traversed to remove the target audio focus corresponding to the at least one target audio system in the abnormal state or the audio focus corresponding to the at least one other audio system from the focus stack, and the target audio focus of the at least one target audio system or the audio focus of the at least one other audio system is updated.

[0110] According to the audio arbitration interaction device of the embodiment of the present application, when receiving an audio focus request of at least one target audio system and the audio focus of at least one other audio system is in use, the audio priority of each target audio system and at least one other audio system in use is identified, and an audio priority identification result is generated. If there is a highest audio priority in at least one target audio system, a target audio focus is allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that has been preempted is removed, so as to control the target audio system to play audio based on the target audio focus. Thus, the problems of audio focus conflict, asynchronous audio focus allocation, and inability to determine the recovery of audio focus after preemption that exist in the related art are solved. By implementing distributed arbitration logic between various audio systems, the audio focus is closely associated with the writing of the audio data stream, so that audio output can only be performed when the system obtains the audio focus, thereby realizing a more robust, flexible and reliable audio management system.

[0111] Fig.10 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:

[0112] A memory 1001 , a processor 1002 , and a computer program stored in the memory 1001 and executable on the processor 1002 .

[0113] When the processor 1002 executes the program, the audio arbitration interaction method provided in the above embodiment is implemented.

[0114] Furthermore, the vehicle also includes:

[0115] The communication interface 1003 is used for communication between the memory 1001 and the processor 1002 .

[0116] The memory 1001 is used to store computer programs that can be executed on the processor 1002 .

[0117] The memory 1001 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0118] If the memory 1001, the processor 1002 and the communication interface 1003 are implemented independently, the communication interface 1003, the memory 1001 and the processor 1002 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0119] Optionally, in a specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are integrated on a chip, the memory 1001, the processor 1002 and the communication interface 1003 can communicate with each other through an internal interface.

[0120] The processor 1002 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0121] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above audio arbitration interaction method is implemented.

[0122] This embodiment further provides a computer program product, including a computer program, which is executed to implement the audio arbitration interaction method of the above embodiment.

[0123] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0124] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0125] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0126] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0127] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0128] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0130] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An audio arbitration interaction method, characterized in that: The following steps are involved: determining whether an audio focus request of at least one target audio system is received; If an audio focus request of the at least one target audio system is received, determining whether the audio focus of at least one other audio system is in use; if the audio focus of the at least one other audio system is in use, identifying the audio priority of each target audio system and the audio priority of the at least one other audio system in use, and generating audio priority identification results for each target audio system and the at least one other audio system; Based on the audio priority identification result, if there is a highest audio priority in at least one target audio system, a target audio focus is directly allocated to the target audio system corresponding to the highest audio priority, and the target audio focus is added to the focus stack, and the audio focus of at least one other audio system that has been preempted is removed, so as to control the target audio system to perform audio playback based on the target audio focus.

2. The method according to claim 1, characterized in that After allocating a target audio focus to the target audio system corresponding to the highest audio priority, the method further includes: Based on the target audio focus of the target audio system, sending the target audio focus to the at least one other audio system, and determining whether a feedback result of the at least one other audio system is received; If feedback results from at least one other audio system are received, the target audio focus is added to the focus stack, and the target audio system is controlled to perform audio playback based on the target audio focus; otherwise, it is determined that the target audio focus has failed to be added to the focus stack, and the control of the target audio system to perform audio playback based on the target audio focus is stopped.

3. The method according to claim 1, characterized in that The identifying the audio priority of each target audio system and the audio priority of the at least one other audio system in use comprises: A current driving scene of the vehicle is identified based on the current driving state and current driving environment of the vehicle, and an audio priority of each target audio system and an audio priority of the at least one other audio system in use are identified based on the current driving scene.

4. The method according to claim 3, characterized in that The identifying the audio priority of each target audio system and the audio priority of the at least one other audio system in use further includes: Obtain the user's vehicle driving habits; A priority definition rule for the user is generated based on the vehicle driving habits of the user, and the audio priority of each target audio system and the audio priority of the at least one other audio system in use are identified based on the priority definition rule.

5. The method according to claim 1, characterized in that Before controlling the target audio system to play audio based on the target audio focus, the method further includes: Creating an audio buffer corresponding to the target audio system, and determining whether the target audio system has a target audio focus; If the target audio system has the target audio focus, the audio data of the target audio system is written to the audio buffer using the application interface corresponding to the target audio system; otherwise, the audio data is refused to be written and an error code is returned to control the target audio system to stop audio playback.

6. The method according to claim 1, characterized in that Also includes: Determine whether a heartbeat packet of each target audio system and each other audio system is received within a preset time; If the heartbeat packet of the at least one target audio system or the at least one other audio system is not received within the preset time, it is determined that the at least one target audio system or the at least one other audio system is in an abnormal state, and the focus stack is traversed to remove the target audio focus corresponding to the at least one target audio system in the abnormal state or the audio focus corresponding to the at least one other audio system from the focus stack, and the target audio focus of the at least one target audio system or the audio focus of the at least one other audio system is updated.

7. An audio arbitration interactive device, characterized in that: include: A determination module, used to determine whether an audio focus request of at least one target audio system is received; a generating module, configured to, if receiving an audio focus request of the at least one target audio system, determine whether the audio focus of at least one other audio system is in use, and if the audio focus of the at least one other audio system is in use, identify the audio priority of each target audio system and the audio priority of the at least one other audio system in use, and generate audio priority identification results of each target audio system and the at least one other audio system; A control module is used to allocate a target audio focus directly to the target audio system corresponding to the highest audio priority based on the audio priority identification result, if there is a highest audio priority in the at least one target audio system, and add the target audio focus to the focus stack, and remove the audio focus of the at least one other audio system that has been preempted, so as to control the target audio system to perform audio playback based on the target audio focus.

8. The device according to claim 7, characterized in that After allocating the target audio focus to the target audio system corresponding to the highest audio priority, the control module is further configured to: Based on the target audio focus of the target audio system, sending the target audio focus to the at least one other audio system, and determining whether a feedback result of the at least one other audio system is received; If feedback results from at least one other audio system are received, the target audio focus is added to the focus stack, and the target audio system is controlled to perform audio playback based on the target audio focus; otherwise, it is determined that the target audio focus has failed to be added to the focus stack, and the control of the target audio system to perform audio playback based on the target audio focus is stopped.

9. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the audio arbitration interaction method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the audio arbitration interaction method according to any one of claims 1 to 6.

Images