Vehicle audio focus management method driven by audio and video playing stream of virtual machine

Abstract

The invention is suitable for the technical field of vehicle machine control, and provides a virtual machine audio and video playing stream driven vehicle machine audio focus management method, which comprises the following steps: when a virtual machine monitors that an application program triggers a multimedia data playing behavior, generating playing stream state information; the virtual machine subsystem judges whether an effective audio and video playing stream exists in the virtual machine or not based on the playing stream state information, and transmits the playing stream state information to an outer layer agent process when the effective audio and video playing stream exists in the virtual machine; the outer layer agent process generates an audio focus request based on the playing stream state information when the vehicle machine system allows multimedia playing, and sends the audio focus request to the vehicle machine audio management module, so that the vehicle machine audio management module carries out audio focus arbitration, and sends an audio focus arbitration result to the virtual machine subsystem through the outer layer agent process; and controlling the audio and video playing behavior in the virtual machine by the virtual machine subsystem. According to the invention, the multi-sound-source switching experience in the vehicle-mounted terminal system can be improved.

Description

Technical Field

[0001] This application belongs to the field of vehicle control technology, and in particular relates to a method for managing vehicle audio focus driven by a virtual machine audio and video playback stream. Background Technology

[0002] With the development of in-vehicle infotainment (IVI) systems and in-vehicle virtualization technology, more and more vehicle systems are running third-party applications through virtual machines to achieve application isolation, system security, and unified resource management.

[0003] In existing technologies, in-vehicle infotainment systems typically use an audio focus mechanism to uniformly schedule multiple audio sources to avoid interference between audio sources such as navigation, voice calls, and multimedia playback. However, in virtual machine environments, existing audio focus management methods still suffer from poor multi-source switching experiences: in high-priority audio scenarios such as navigation announcements, voice interactions, and reversing warnings, third-party applications are forced to release audio focus, causing audio interruptions in these applications and impacting the user experience. Summary of the Invention

[0004] This application provides a method for managing in-vehicle audio focus driven by a virtual machine audio and video playback stream, which can solve the problem of poor experience when switching between multiple audio sources in the in-vehicle system.

[0005] This application provides a method for managing in-vehicle audio focus driven by a virtual machine audio / video playback stream. The method is applied to an in-vehicle infotainment system. The system includes a virtual machine subsystem, an outer agent process, and an in-vehicle audio management module. The virtual machine subsystem runs the virtual machine's internal operating system and applications. The outer agent process manages the virtual machine's audio resources. The in-vehicle audio management module schedules and arbitrates the audio focus of multiple audio streams within the system. The method includes:

[0006] When the virtual machine detects that an application running on the virtual machine has triggered multimedia data playback, it generates playback stream status information to characterize the multimedia data playback status of the application on the virtual machine; the multimedia data is audio or video.

[0007] The virtual machine subsystem determines whether there is a valid audio or video playback stream inside the virtual machine based on the playback stream status information, and if there is a valid audio or video playback stream, it passes the playback stream status information to the outer agent process.

[0008] When the vehicle system allows multimedia playback, the outer agent process generates an audio focus request based on the playback stream status information and sends the audio focus request to the vehicle audio management module so that the vehicle audio management module can perform audio focus arbitration on the audio focus request.

[0009] After receiving the audio focus arbitration result from the vehicle audio management module, the outer agent process sends the audio focus arbitration result to the virtual machine subsystem so that the virtual machine subsystem can control the audio and video playback behavior inside the virtual machine.

[0010] Optionally, the virtual machine can listen to the audio output port of the virtual machine through a virtual audio device to determine whether the application running on the virtual machine has triggered multimedia data playback behavior.

[0011] Optionally, the playback stream status information includes a playback start identifier, a playback pause identifier, a playback stop identifier, and a playback stream type identifier; the playback start identifier, playback pause identifier, playback stop identifier, and playback stream type identifier all carry the identity identifier of the corresponding application.

[0012] Optionally, the virtual machine subsystem determines whether a valid audio / video playback stream exists within the virtual machine based on the playback stream status information, including:

[0013] The virtual machine subsystem determines the playback start flag, playback pause flag, playback stop flag, and playback stream type flag. If the playback start flag indicates that there is an application outputting multimedia data inside the virtual machine, then it is determined that there is a valid audio and video playback stream inside the virtual machine.

[0014] Optionally, the virtual machine subsystem may pass through the playback stream status information to the outer agent process via the virtual device communication channel.

[0015] Optionally, the virtual device communication channel can be one or more of shared memory, virtual character devices, and inter-process communication mechanisms.

[0016] Optionally, the audio focus request carries: the identity of the application corresponding to the playback start identifier and the playback stream type identifier of the application.

[0017] Optionally, before the step of generating an audio focus request based on playback stream status information, the vehicle-mounted audio focus management method further includes:

[0018] The outer agent process determines whether the application corresponding to the playback start identifier has applied for audio focus when the vehicle system allows multimedia playback. If it has not applied for audio focus, it proceeds to the step of generating an audio focus request based on the playback stream status information; otherwise, the process ends.

[0019] The above-mentioned solution in this application has the following beneficial effects:

[0020] In this embodiment, when an application playing audio or video on a virtual machine is forced to release audio focus due to high-priority audio scenarios such as navigation announcements, voice interaction, and reversing prompts requiring audio focus, the virtual machine monitors the application's audio and video playback behavior. Upon detecting audio and video playback behavior, it generates playback stream status information representing the application's audio and video playback status. Then, the virtual machine subsystem determines whether a valid audio and video playback stream exists within the virtual machine based on this playback stream status information. If a valid audio and video playback stream exists within the virtual machine, the playback stream status information is passed through to the outer agent process. The outer agent process then sends an audio focus request to the vehicle's audio management module when the vehicle system allows multimedia playback, thereby enabling the application to resend the audio focus request, minimizing the application's audio interruption time, and improving the multi-source switching experience in the vehicle system.

[0021] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of a vehicle audio focus management method driven by a virtual machine audio / video playback stream, provided as an embodiment of this application. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0025] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0028] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] To address the poor audio source switching experience in current in-vehicle infotainment systems, this application provides a virtual machine-driven audio / video playback stream-based in-vehicle audio focus management method. This method addresses the issue of applications playing audio or video on a virtual machine being forced to release audio focus due to high-priority audio scenarios such as navigation announcements, voice interactions, and reversing prompts. The virtual machine monitors the application's audio / video playback behavior and generates playback stream status information representing the application's playback state. The virtual machine subsystem then uses this status information to determine if a valid audio / video playback stream exists within the virtual machine. If a valid playback stream exists, the status information is passed to an outer proxy process. This outer proxy process then sends an audio focus request to the in-vehicle audio management module when multimedia playback is enabled in the in-vehicle system. This allows the application to resend its audio focus request, minimizing audio interruption time and improving the multi-source switching experience in the in-vehicle infotainment system.

[0031] The following describes the in-vehicle audio focus management method driven by a virtual machine audio and video playback stream provided in this application by way of specific embodiments.

[0032] The virtual machine audio and video playback stream driven vehicle audio focus management method provided in this application embodiment is applied to the vehicle system. The vehicle system runs a host operating system, on which at least one virtual machine instance is created and runs. The virtual machine is used to run third-party applications or multimedia applications.

[0033] The aforementioned vehicle infotainment system deploys a virtual machine subsystem, an outer agent process, and a vehicle audio management module. The virtual machine subsystem runs the operating system and applications within the virtual machine; the outer agent process runs within the host operating system and manages the virtual machine's audio resources; the vehicle audio management module handles unified scheduling and audio focus arbitration for multiple audio streams within the vehicle infotainment system. The virtual machine subsystem and the outer agent process interact with each other via a virtual device communication channel.

[0034] It should be noted that the aforementioned vehicle audio management module is an existing audio management module in current vehicle systems. In this embodiment, the module arbitrates received audio focus requests according to a pre-defined audio scheduling and audio focus arbitration mechanism to achieve audio scheduling.

[0035] like Figure 1 As shown in the embodiments of this application, the in-vehicle audio focus management method driven by a virtual machine audio and video playback stream includes the following steps:

[0036] Step 11: When the virtual machine detects that an application running on the virtual machine has triggered a multimedia data playback behavior, it generates playback stream status information to characterize the multimedia data playback status of the application on the virtual machine. The multimedia data is audio or video.

[0037] In some embodiments of this application, the virtual machine can listen to its audio output port via a virtual audio device to determine whether an application running on the virtual machine has triggered multimedia data playback. A virtual audio device refers to a virtual device within the virtual machine used for playing audio.

[0038] In other words, if audio output is detected at the audio output port, it is assumed that the application on the virtual machine has triggered multimedia data playback.

[0039] When a virtual machine detects that an application on the virtual machine has triggered multimedia data playback, it generates corresponding playback stream status information (i.e., the playback stream status information of audio and video obtained through monitoring). This playback stream status information includes a playback start flag, a playback pause flag, a playback stop flag, and a playback stream type flag. The playback start flag, playback pause flag, playback stop flag, and playback stream type flag all carry the identity identifier of the corresponding application.

[0040] For example, when the playback start flag indicates that an application currently outputting multimedia data exists inside the virtual machine, the playback start flag carries the identity of the application currently outputting multimedia data, which can be a process identifier (PID). When the playback start flag indicates that no application is currently outputting multimedia data inside the virtual machine, the identity of the application carried by the playback start flag can be 0, to indicate that no application is currently outputting multimedia data.

[0041] Similarly, when the play-pause flag indicates that an application is currently paused within the virtual machine, the flag carries the identity of that application, which can be a process identifier (PID). When the play-pause flag indicates that no application is currently paused within the virtual machine, the identity of the application carried by the flag can be 0, indicating that no application is currently paused.

[0042] When the playback stop flag indicates that an application is currently playing but has stopped within the virtual machine, the playback stop flag carries the identity of the application that is currently playing but has stopped. This identity can be a process identifier (PID). When the playback stop flag indicates that no application is currently playing but has stopped within the virtual machine, the identity of the application carried by the playback stop flag can be 0, indicating that no application is currently playing but has stopped.

[0043] The playback stream type identifier indicates the type of multimedia data output by the application currently outputting multimedia data (e.g., 01 for audio, 10 for video), and carries the identity identifier of the application outputting the multimedia data. It is understood that when no application is currently outputting multimedia data within the virtual machine, both the playback stream type identifier and the carried application identity identifier can be 0, indicating that no application is currently outputting multimedia data.

[0044] It is understood that the specific forms of the playback start flag, playback pause flag, playback stop flag, and playback stream type flag can be set according to the code requirements of the actual operation process, and the embodiments of this application do not limit their specific forms.

[0045] Step 12: The virtual machine subsystem determines whether there is a valid audio or video playback stream inside the virtual machine based on the playback stream status information, and if there is a valid audio or video playback stream, it passes the playback stream status information to the outer agent process.

[0046] Specifically, the virtual machine subsystem can determine the playback start flag, playback pause flag, playback stop flag, and playback stream type flag. If the playback start flag indicates that an application is currently outputting multimedia data inside the virtual machine, then it is determined that a valid audio / video playback stream exists inside the virtual machine. Conversely, if it is determined that all audio / video playback streams inside the virtual machine are paused or stopped, then it is determined that no valid playback stream exists inside the virtual machine.

[0047] When the virtual machine subsystem determines that a valid audio or video playback stream exists within the virtual machine, it can transmit the playback stream status information to the outer agent process through the virtual device communication channel. This virtual device communication channel can be one or more of shared memory, virtual character devices, and inter-process communication mechanisms.

[0048] Step 13: When the vehicle system allows multimedia playback, the outer agent process generates an audio focus request based on the playback stream status information and sends the audio focus request to the vehicle audio management module so that the vehicle audio management module can perform audio focus arbitration on the audio focus request.

[0049] The aforementioned audio focus request carries the identity of the application corresponding to the playback start identifier and the playback stream type identifier of that application. That is, the audio focus request carries the identity of the application currently outputting multimedia data within the virtual machine, as well as the type of that multimedia data (audio or video). This allows the application to resend the audio focus request, preventing the application from being in a silent state even when it is actually outputting multimedia data after being forced to release audio focus, thus minimizing the duration of sound interruptions and improving the multi-source switching experience in the vehicle's infotainment system.

[0050] It's understandable that whether an application can successfully secure audio focus depends on the audio focus arbitration performed by the vehicle's audio management module after receiving the audio focus request. This application proposes resending the audio focus request through an outer proxy process; however, the actual decision still rests with the vehicle's audio management module's own mechanism.

[0051] It should be noted that, to avoid repeatedly submitting audio focus requests, the outer agent process, when determining that the vehicle system allows multimedia playback, will check whether the application currently outputting multimedia data has submitted an audio focus request after being forced to release audio focus. Specifically, the outer agent process will determine whether the application corresponding to the playback start identifier (the application currently outputting multimedia data) has applied for audio focus when the vehicle system allows multimedia playback. If it has not applied for audio focus, it will proceed to the step of generating an audio focus request based on the playback stream status information; otherwise, the process will end.

[0052] The outer proxy process can interact with the vehicle's audio management module to determine whether the vehicle system allows multimedia playback. The outer proxy process can generate and send an audio focus request to the vehicle's audio management module according to the audio focus request format agreed upon by the vehicle system.

[0053] Step 14: After receiving the audio focus arbitration result sent by the vehicle audio management module, the outer agent process sends the audio focus arbitration result to the virtual machine subsystem so that the virtual machine subsystem can control the audio and video playback behavior inside the virtual machine.

[0054] After completing audio focus arbitration, the vehicle audio management module sends the arbitration result (either the audio focus request result or preemption status feedback) to the outer agent process (specifically via the virtual device communication channel). This allows the outer agent process to forward the result to the virtual machine subsystem, enabling the subsystem to control audio and video playback within the virtual machine. Understandably, when audio focus is successfully requested, the virtual machine subsystem can control the currently outputting multimedia data application to continue outputting multimedia data (in practice, no operation is required as the application is currently playing). If audio focus request fails, the virtual machine subsystem displays a prompt message reminding the user that the application is currently playing silently and asking if they want to stop / pause the application's audio and video playback. This allows the user to perform relevant operations on the application to stop / pause its audio and video playback.

[0055] It should be noted that the above process describes how, when an application playing audio or video on the virtual machine is forced to release audio focus due to high-priority audio scenarios such as navigation announcements, voice interactions, or reversing prompts, the outer proxy process resubmits the audio focus request to the vehicle's audio management module, allowing the application to regain audio focus. It can be understood that, when needed, the outer proxy process can also resubmit the audio focus release request to the vehicle's audio management module to allow the application to release audio focus.

[0056] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for managing in-vehicle audio focus driven by a virtual machine audio / video playback stream, characterized in that, This technology is applied to in-vehicle infotainment systems, which deploy a virtual machine subsystem, an outer agent process, and an in-vehicle audio management module. The virtual machine subsystem is used to run the operating system and applications inside the virtual machine, the outer agent process is used to manage the virtual machine's audio resources, and the in-vehicle audio management module is used to schedule and arbitrate audio focus for multiple audio streams in the in-vehicle infotainment system. The in-vehicle audio focus management method includes: When the virtual machine detects that an application running on the virtual machine has triggered a multimedia data playback action, it generates playback stream status information to characterize the multimedia data playback status of the application on the virtual machine; the multimedia data is audio or video. The virtual machine subsystem determines whether there is a valid audio or video playback stream inside the virtual machine based on the playback stream status information, and if there is a valid audio or video playback stream, it passes the playback stream status information to the outer proxy process. When the vehicle system allows multimedia playback, the outer proxy process generates an audio focus request based on the playback stream status information and sends the audio focus request to the vehicle audio management module so that the vehicle audio management module can perform audio focus arbitration on the audio focus request. After receiving the audio focus arbitration result sent by the vehicle audio management module, the outer agent process sends the audio focus arbitration result to the virtual machine subsystem so that the virtual machine subsystem can control the audio and video playback behavior inside the virtual machine.

2. The vehicle audio focus management method according to claim 1, characterized in that, The virtual machine monitors its audio output port via a virtual audio device to determine whether an application running on the virtual machine has triggered multimedia data playback.

3. The vehicle audio focus management method according to claim 1, characterized in that, The playback stream status information includes a playback start identifier, a playback pause identifier, a playback stop identifier, and a playback stream type identifier; the playback start identifier, the playback pause identifier, the playback stop identifier, and the playback stream type identifier all carry the identity identifier of the corresponding application.

4. The vehicle audio focus management method according to claim 3, characterized in that, Based on the playback stream status information, the virtual machine subsystem determines whether a valid audio / video playback stream exists within the virtual machine, including: The virtual machine subsystem determines the playback start flag, the playback pause flag, the playback stop flag, and the playback stream type flag. If the playback start flag indicates that there is an application outputting multimedia data inside the virtual machine, then it is determined that there is a valid audio and video playback stream inside the virtual machine.

5. The vehicle audio focus management method according to claim 1, characterized in that, The virtual machine subsystem transmits the playback stream status information to the outer agent process through the virtual device communication channel.

6. The vehicle audio focus management method according to claim 5, characterized in that, The virtual device communication channel is one or more of the following: shared memory, virtual character devices, and inter-process communication mechanisms.

7. The vehicle audio focus management method according to claim 3, characterized in that, The audio focus request carries: the identity identifier of the application corresponding to the playback start identifier and the playback stream type identifier of the application.

8. The vehicle audio focus management method according to claim 1, characterized in that, Prior to the step of generating an audio focus request based on the playback stream status information, the vehicle-mounted audio focus management method further includes: The outer proxy process determines whether the application corresponding to the playback start identifier has applied for audio focus when the vehicle system allows multimedia playback. If it has not applied for audio focus, it proceeds to the step of generating an audio focus request based on the playback stream status information; otherwise, the process ends.

Images