Audio data processing methods, apparatus, equipment, storage media and software products

Abstract

This disclosure provides an audio data processing method, apparatus, device, storage medium, and program product, relating to the field of vehicle technology. The method, applied to an in-vehicle infotainment system, includes: in response to a successful connection with a mobile terminal, storing audio data to be played received from the mobile terminal into an intermediate buffer of the in-vehicle infotainment system via a preset thread; and retrieving the audio data to be played from the intermediate buffer for rendering and playback. This method, by retrieving audio data from the intermediate buffer of the in-vehicle infotainment system for rendering and playback, reduces the time spent retrieving audio data, increases data access efficiency, and thus effectively controls the sampling of audio data, ensuring playback quality.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle technology, and in particular to an audio data processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] For scenarios where mobile terminals and vehicle infotainment systems establish screen mirroring based on corresponding in-vehicle connectivity systems (such as CarPlay), and the mobile terminal plays audio through the in-vehicle infotainment system, the common approach is for the in-vehicle infotainment system to initialize the corresponding audio playback interface (API) and directly transmit the audio data read from the mobile terminal to the audio playback API to achieve playback functionality. In this implementation method, when the system load of the in-vehicle infotainment system is heavy, it will lead to a large sampling rate error in the audio data acquisition process, which will fail to meet the sampling rate requirements of the in-vehicle connectivity system and affect audio playback. Summary of the Invention

[0003] This disclosure provides an audio data processing method, an audio data processing device, an electronic device, a computer-readable storage medium, and a computer program product. By obtaining audio data from the intermediate buffer of the vehicle's infotainment system for rendering and playback, the time used to call audio data is reduced, data access efficiency is increased, thereby improving the processing efficiency of audio data and ensuring playback effect.

[0004] In a first aspect, embodiments of this disclosure propose an audio data processing method applied to an in-vehicle infotainment system, comprising: in response to a successful connection with a mobile terminal, storing audio data to be played received from the mobile terminal into an intermediate buffer of the in-vehicle infotainment system via a preset thread; and retrieving the audio data to be played from the intermediate buffer for rendering and playback.

[0005] Secondly, this disclosure provides an audio data processing device for use in a vehicle infotainment system, comprising: a first processing module and a playback module. The first processing module is configured to, in response to a successful connection with a mobile terminal, store audio data to be played received from the mobile terminal into an intermediate buffer of the vehicle infotainment system via a preset thread; the playback module is configured to retrieve the audio data to be played from the intermediate buffer and render and play it.

[0006] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the audio data processing method described in any implementation of the first aspect.

[0007] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the audio data processing method described in any implementation of the first aspect.

[0008] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the audio data processing method as described in any implementation of the first aspect.

[0009] The audio data processing scheme provided in this embodiment, in response to a successful connection with a mobile terminal, can first store the read audio data received from the mobile terminal in an intermediate buffer added to the vehicle's infotainment system through a preset thread dedicated to reading the audio data to be played. When there is a need for audio playback, the audio data to be played is retrieved from the intermediate buffer for rendering. This avoids direct interaction with the mobile terminal when retrieving the audio data to be played. By retrieving the audio data from the intermediate buffer of the vehicle's infotainment system for rendering and playback, the time used to call audio data is reduced, data access efficiency is increased, and the sampling of audio data can be effectively controlled to ensure playback effect.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0012] Figure 1 This is an exemplary system architecture to which this disclosure can be applied;

[0013] Figure 2 A flowchart of an audio data processing method provided in this disclosure embodiment;

[0014] Figure 3 A flowchart illustrating a method for storing audio data to be played into an intermediate buffer, as provided in this embodiment of the disclosure;

[0015] Figure 4 A flowchart illustrating an audio data processing method in an application scenario provided by an embodiment of this disclosure;

[0016] Figure 5 A structural block diagram of an audio data processing apparatus provided in an embodiment of this disclosure;

[0017] Figure 6 This is a schematic diagram of the structure of an electronic device suitable for performing an audio data processing method, provided as an embodiment of the present disclosure. Detailed Implementation

[0018] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0019] It should be noted that the collection, acquisition, storage, processing, transmission, provision, disclosure, and application of user personal information (such as identity verification information) involved in the technical solution disclosed herein are all carried out with the user's knowledge and explicit authorization, comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0020] For scenarios where mobile terminals and vehicle infotainment systems establish screen mirroring based on corresponding in-vehicle connectivity systems (such as CarPlay), and the mobile terminal plays audio through the in-vehicle system, the typical approach is for the in-vehicle system to initialize a corresponding audio playback interface (Application Programming Interface, API, such as Android's Oboe interface) and directly transmit the audio data read from the mobile terminal to this audio playback API to achieve playback functionality. The core plugin of the in-vehicle connectivity system periodically calculates the current audio sampling rate based on the frequency and size of the audio data read by the in-vehicle system. In this implementation, when the in-vehicle system is under heavy load, it can lead to large sampling rate errors during the audio data acquisition process, thus failing to meet the sampling rate requirements of the in-vehicle connectivity system and affecting audio playback.

[0021] Therefore, a solution is needed to improve audio data sampling in order to ensure that the sampling rate requirements of the aforementioned in-vehicle connectivity system and the audio playback effect are met.

[0022] Figure 1 An exemplary architecture of a vehicle 100 to which embodiments of the audio data processing methods, audio data processing apparatus, electronic devices, computer-readable storage media, and computer program products of this disclosure can be applied is illustrated. The vehicle 100 may include a first controller 110, a second controller 120, a screen 130, and peripherals 140. The first controller 110 and the second controller 120 are communicatively connected to enable data interaction.

[0023] In some optional implementations of the embodiments of this disclosure, the first controller 210 can be implemented as a microcontroller (MCU), mainly responsible for controlling and managing various devices and sensors of the vehicle. The first controller 210 can be used to acquire and process sensor data, control and schedule actuators, etc. It should be noted that, without departing from the teachings of this disclosure, the first controller 210 can also be implemented as other devices with data processing capabilities, and no specific limitations are made herein.

[0024] In some optional implementations of the embodiments of this disclosure, the second controller 120 can be implemented as a system-on-a-chip (SoC), which integrates a processor, memory, peripherals, and other functions. Due to its powerful computing and processing capabilities, the second controller 120 can be integrated into an in-vehicle infotainment system (referred to as an in-vehicle infotainment system) and used to process and analyze sensor data in real time, and make decisions. It should be noted that, without departing from the teachings of this disclosure, the second controller 120 can also be implemented as other devices with computing and processing capabilities, and no specific limitations are made herein.

[0025] In some optional implementations of the embodiments of this disclosure, the first controller 110 and the second controller 120 may be specifically located in the cockpit domain of the vehicle 100.

[0026] It should be noted that in some optional implementations of the embodiments of this disclosure, other controllers may also be provided in the vehicle 100 to better realize the functions of the vehicle system, etc., which are not specifically limited here.

[0027] In some optional implementations of the embodiments of this disclosure, the screen 130 may include, but is not limited to, at least one of an instrument cluster display, a central control display, and a rear-seat display. In some optional implementations, after the vehicle system mounted on the second controller 120 is started, it can output the instrument cluster interface, etc., to the instrument cluster display for display, and output the central control interface, etc., to the central control display for display.

[0028] In some optional implementations of the embodiments of this disclosure, the peripheral device 140 of the vehicle 100 may refer to the device of the vehicle system mounted on the second controller 120, which may include, but is not limited to, devices such as microphones and vehicle cameras, etc., which will not be listed here.

[0029] Please refer to Figure 2 , Figure 2 A flowchart of an audio data processing method provided in this disclosure embodiment, the audio data processing method being applied to an in-vehicle system, wherein process 200 includes the following steps:

[0030] Step 201: In response to the successful establishment of a connection with the mobile terminal, the audio data to be played received from the mobile terminal is stored in the intermediate buffer of the vehicle's infotainment system via a preset thread.

[0031] This step is intended for the entity executing the audio data processing method (e.g., Figure 1 After the vehicle's infotainment system (VIS) in the vehicle 105 shown successfully establishes a connection with a mobile terminal (such as a user's mobile phone, for example via Bluetooth), it can first store the read audio data (such as audio pulse code modulation (PCM) data) received from the mobile terminal in an intermediate buffer area added to the VIS to increase data access efficiency.

[0032] Step 202: Obtain the audio data to be played from the intermediate buffer and render and play it.

[0033] Based on step 201, this step aims to enable the aforementioned execution entity to conveniently retrieve the audio data to be played from the intermediate cache of the vehicle system when it needs to render and play audio data. This data is pre-stored in the preset thread.

[0034] The audio data processing method provided in this embodiment, in response to a successful connection with a mobile terminal, can first store the read audio data received from the mobile terminal in an intermediate buffer added to the vehicle's infotainment system through a preset thread dedicated to reading the audio data to be played. When there is a need for audio playback, the audio data to be played is retrieved from the intermediate buffer for rendering. This avoids direct interaction with the mobile terminal when retrieving the audio data to be played. By retrieving the audio data from the intermediate buffer of the vehicle's infotainment system for rendering and playback, the time used to call audio data is reduced, data access efficiency is increased, and the sampling of audio data can be effectively controlled to ensure playback effect.

[0035] In some optional implementations of the embodiments of this disclosure, in the above... Figure 2 Based on the corresponding audio data processing method embodiment, the above step 201 can be specifically implemented as follows: storing the audio data to be played into the intermediate buffer area through a preset thread according to a preset sampling interval duration.

[0036] In this embodiment, the frequency or number of times the audio data to be played received from the mobile terminal is stored in the intermediate buffer of the vehicle system can be controlled by a preset sampling interval duration, so as to ensure that the storage space utilization rate corresponding to the intermediate buffer is basically kept in dynamic balance, thereby ensuring the audio playback effect.

[0037] It should be noted that the value of the preset sampling interval can be set according to specific needs, and is not specifically limited here. In some optional implementations of the embodiments of this disclosure, the value of the preset sampling interval can be 20ms.

[0038] In some optional implementations of the embodiments of this disclosure, in any of the above embodiments, step 201 can be specifically implemented as: storing the audio data to be played from the mobile terminal and located in the vehicle's in-vehicle interconnection system into an intermediate buffer area via a preset thread.

[0039] In this embodiment, specifically in response to a successful connection with the mobile terminal, the audio data to be played received from the mobile terminal is first stored in the vehicle's preset in-vehicle connectivity system. Furthermore, the audio data located in the preset in-vehicle connectivity system can be transferred to the vehicle's intermediate buffer via the aforementioned preset thread. This effectively avoids direct interaction with the preset in-vehicle connectivity system when obtaining the audio data to be played, reducing the time used to call the audio data.

[0040] Furthermore, in some optional implementations of the embodiments of this disclosure, the audio data to be played received from the mobile terminal can be stored in the core plugin of the preset in-vehicle connectivity system. Furthermore, the preset in-vehicle connectivity system includes, but is not limited to, the CarPlay system, wherein the core plugin of the CarPlay system can be specifically implemented as a Software Development Kit (SDK) library.

[0041] In some optional implementations of the embodiments of this disclosure, in the above... Figure 2 Based on the corresponding audio data processing method embodiment, step 202 above can be specifically implemented as follows: Obtain the audio data to be played from the intermediate buffer through the vehicle's preset audio playback interface for rendering and playback. In this way, the audio data to be played, pre-stored in the vehicle's intermediate buffer via the preset thread can be conveniently obtained through the vehicle's corresponding audio playback interface, i.e., the preset audio playback interface, thereby improving data access efficiency. In some optional implementations of this disclosure embodiment, in the above... Figure 2 Based on the corresponding audio data processing method embodiment, the initial storage space size of the intermediate buffer corresponds to the amount of audio data for the first preset duration; and the audio data processing method may further include the following: in response to the data volume of the audio data to be played being greater than the data volume for the first preset duration, adjusting the storage space size of the intermediate buffer according to the audio type of the audio data to be played through a preset thread.

[0042] In this embodiment, the relationship between the real-time data volume of the received audio data to be played and the initial storage space size of the intermediate buffer in the vehicle's infotainment system can be dynamically monitored. When it is detected that the intermediate buffer is insufficient to accommodate the received data, the storage space size of the intermediate buffer is dynamically adjusted according to the audio type of the audio data to be played through the aforementioned preset thread, thereby adapting to the audio data load in different scenarios. The initial storage space size of the intermediate buffer corresponds to a certain preset duration of audio data, i.e., a first preset duration, to ensure that at least the storage requirements of this data volume are met, while also helping to improve the utilization rate of the storage resources corresponding to the intermediate buffer.

[0043] It should be noted that the value of the first preset duration can be set according to specific needs, and is not specifically limited here. In some optional implementations of the embodiments of this disclosure, the value of the first preset duration can be 1000ms.

[0044] Furthermore, in some optional implementations of the embodiments of this disclosure, the above-described method of adjusting the storage space size of the intermediate buffer according to the audio type of the audio data to be played via a preset thread can be specifically implemented as follows:

[0045] In response to the audio data to be played belonging to a first audio type, the initial storage space size is expanded to the first storage space size through a preset thread; wherein, the first storage space size corresponds to the amount of audio data of a second preset duration; wherein, the first audio type includes non-dialogue types; in response to the audio data to be played belonging to a second audio type, the initial storage space size is expanded to the second storage space size through a preset thread; wherein, the second audio type includes dialogue types, and the second storage space size corresponds to the amount of audio data of a third preset duration, wherein, the third preset duration is longer than the first preset duration and shorter than the second preset duration.

[0046] In this embodiment, if the initial storage space of the intermediate buffer in the vehicle's infotainment system is insufficient to accommodate the received real-time data, the initial storage space can be expanded to different sizes via a preset thread, depending on the audio type of the audio data to be played. This dynamically adjusts the storage space of the intermediate buffer in the vehicle's infotainment system, thereby improving the effective utilization of the system's storage resources. Specifically, the expanded first storage space size corresponding to a first audio type (such as music) is matched with the audio data volume of a second preset duration, and the expanded second storage space size corresponding to a second audio type (such as voice calls or human-computer dialogue based on a voice assistant in a mobile terminal, such as Siri) is matched with the audio data volume of a third preset duration. The sizes of the first, third, and second preset durations increase sequentially.

[0047] Furthermore, in some optional implementations of the embodiments of this disclosure, the second preset duration is less than or equal to the fourth preset duration. In this embodiment, in order to effectively control the upper limit of the storage space that the intermediate buffer of the vehicle system can occupy, the sampling of audio data can be effectively controlled. It should be noted that the value of the fourth preset interval duration can be set according to specific needs and is not specifically limited here. In some optional implementations of the embodiments of this disclosure, the value of the fourth preset duration can be 10s.

[0048] Furthermore, in some optional implementations of the embodiments of this disclosure, based on any of the above embodiments, the audio data to be played can be stored in an intermediate buffer through the following process to achieve effective control over audio data sampling. For details, please refer to... Figure 3 This process can be executed in the vehicle's infotainment system, and process 300 includes the following steps:

[0049] Step 301: Obtain the operation duration corresponding to the current time when the audio data to be played is stored in the intermediate buffer through the preset thread.

[0050] This step is intended for the entity executing the audio data processing method (e.g., Figure 1 The vehicle-mounted unit (V2U) in the vehicle 105 shown records the operation time (or actual operation time) experienced each time the audio data to be played is stored in the intermediate buffer through the aforementioned preset thread. In some implementations of this disclosure, the operation time can be determined by recording the timestamps before and after the operation of storing the audio data to be played in the intermediate buffer, and using the time difference between the timestamps before and after the operation as the operation time.

[0051] Step 302: Determine the difference between the preset sampling interval and the target duration as the current sleep duration; where the target duration is the sum of the operation duration and the historical sleep duration error.

[0052] Based on step 301, this step aims to have the aforementioned execution entity calculate the current sleep duration error. Specifically, the difference between the preset sampling interval duration and the target duration can be used as the current sleep duration, and the target duration is the sum of the operation duration and the historical sleep duration error. That is, the current sleep duration = preset sampling interval duration - operation duration - historical sleep duration error, where the historical sleep duration error can refer to the sleep duration error corresponding to the previous audio data sampling (according to the preset sampling time interval duration) adjacent to the current audio data sampling.

[0053] Step 303: In response to the current sleep duration being greater than a preset value, perform a sampling sleep operation within the current sleep duration; wherein, the sampling sleep operation includes suspending the operation of storing the audio data to be played into the intermediate buffer according to the preset sampling interval duration.

[0054] Step 304: In response to the current sleep duration being less than or equal to the preset value, set the value of the historical sleep duration error to 0, and continue to execute the operation of storing the audio data to be played into the intermediate buffer according to the preset sampling interval duration.

[0055] Building upon step 302, this step aims to have the aforementioned executing entity determine, based on the relationship between the current sleep duration and a preset value, whether a sampling sleep operation needs to be performed according to the current sleep duration, thereby effectively controlling the sampling of audio data. Specifically, when it is determined that the calculated current sleep duration is greater than the preset value, it indicates that the current requirement for audio data sampling according to the preset sampling interval is met, and the system resources and system load occupied are within a reasonable and controllable range. In this case, a sampling sleep operation can be performed within the current sleep duration, that is, the operation of storing the audio data to be played into the intermediate buffer according to the preset sampling interval can be suspended or temporarily stopped. If the calculated current sleep duration is less than or equal to the preset value, it means that the current system can meet the requirement of sampling audio data according to the preset sampling interval. However, the system resources occupied are increased and the system load is heavy. It is possible that the audio data can be read continuously without performing the sampling sleep operation, and the requirement of reading data according to the preset sampling interval cannot be met. In this case, there is no need to perform the sampling sleep operation. At this time, the value of the historical sleep duration error can be set from the current value to 0, and the operation of storing the audio data to be played into the intermediate buffer according to the preset sampling interval can continue.

[0056] In some optional implementations of the embodiments of this disclosure, in Figure 3 Based on the corresponding embodiments, the following may also be included:

[0057] Obtain the actual execution time corresponding to the sampling sleep operation; determine the difference between the actual execution time and the current sleep time as the current sleep time error; wherein, the current sleep time error is used as the new historical sleep time error.

[0058] In this embodiment, considering that the actual elapsed time during the actual execution of the sampling sleep operation may differ from the theoretical value of the current sleep duration calculated above, meaning that system scheduling errors exist during audio data sampling; generally, the actual execution time corresponding to the sampling sleep operation is longer than the theoretical sleep duration, i.e., the current sleep duration. Therefore, by monitoring the difference between the theoretical sleep duration and the actual execution time corresponding to the sampling sleep operation in real time, and using this difference as the basis for calculating the new (i.e., the current sleep duration in the next cycle) current sleep duration, system scheduling errors can be reduced by adjusting the sleep duration.

[0059] It should be noted that the specific value of the above-mentioned preset value can be set according to the actual situation, and is not specifically limited here. Furthermore, in some optional implementations of the embodiments of this disclosure, the value of the above-mentioned preset value can be 0.

[0060] To enhance understanding, this disclosure also provides a specific implementation scheme based on a particular application scenario. Please refer to the example below. Figure 4 The illustrated process 400 can be specifically applied to a vehicle infotainment system. Specifically, after the vehicle infotainment system receives an audio playback request from the iPhone (corresponding to the mobile terminal in the above embodiment), it starts a thread (corresponding to the preset thread in the above embodiment) specifically for reading audio PCM data (corresponding to the audio data to be played in the above embodiment) from the CarPlay (corresponding to the preset in-vehicle connectivity system in the above embodiment) core plugin and writing it to an intermediate buffer. This intermediate buffer is located in the vehicle infotainment system and its size is sufficient to buffer PCM data at the current audio sampling rate for 1000ms (corresponding to the audio data amount of the first preset duration in the above embodiment). This process 400 may include the following:

[0061] Step 401: Initialize the above intermediate cache.

[0062] Step 402: Read audio PCM data from the core plugin.

[0063] Step 403: Write the audio PCM data to the intermediate buffer.

[0064] Steps 402-403 above may specifically include the following processes:

[0065] (1) Initialize the sleep error (lastSleepDiffTime) to 0;

[0066] (2) Record the current timestamp before reading (timeStamp1 = systemCurrentTime());

[0067] (3) Read audio PCM data;

[0068] (4) Write the read audio PCM data into the intermediate buffer;

[0069] (5) Record the current timestamp after writing (timeStamp2 = systemCurrentTime());

[0070] (6) Obtain the reading operation time (diffTime = timeStamp2 - timeStamp1; corresponding to the operation time in the above embodiment);

[0071] (7) Calculate the current sleep time (sleepTime = 20ms - diffTime - lastSleepDiffTime; corresponding to the current sleep duration in the above embodiment), where lastSleepDiffTime is the error of the previous sleep time (corresponding to the historical sleep duration error in the above embodiment), and 20ms is the current sampling interval (corresponding to the preset sampling interval duration in the above embodiment).

[0072] (8) If the calculated sleep time > 0, then: record the timestamp before sleep (sleeTimeBeginTimeStamp = systemCurrentTime()), and sleep for a specific time according to the calculated sleep time, and then record the timestamp after sleep (sleeTimeEndTimeStampe = systemCurrentTime()). Further, the sleep error (lastSleepDiffTime = sleepTimeEndTimeStampe - sleepTimeBeginTimeStamp - sleepTime; corresponding to the current sleep duration error in the above embodiment) can be calculated as the new last sleep time error.

[0073] (9) If the calculated sleep time is less than or equal to 0, the error value of the previous sleep time is set to 0 or reset to 0. This indicates that the system load is high and the error is large, so the system cannot sleep.

[0074] Through the above process, the system sleeps for a specific time according to the sampling rate (i.e., the current sampling interval). After the sleep is completed, the system scheduling error during this period is calculated. When the system sleeps again, the sleep time is adjusted to reduce this system scheduling error, thereby ensuring that the overall sampling rate error is controlled within the standard range.

[0075] Step 404: Initialize the vehicle's audio playback interface.

[0076] Step 405: Notify the reader to read audio PCM data via the vehicle's audio playback interface.

[0077] Step 406: Read audio PCM data from the intermediate buffer.

[0078] Step 407: Render audio PCM data.

[0079] By adding an intermediate cache reading mechanism between the system playback framework and the CarPlay core plugin, the system playback framework can directly read audio data from the intermediate cache when calling the callback function to obtain audio PCM data. This avoids direct interaction with the CarPlay core plugin, reduces call time, and increases data access efficiency.

[0080] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an audio data processing apparatus, which is similar to... Figure 2 or Figure 3 Corresponding to the method embodiment shown, the device can be specifically applied to in-vehicle infotainment systems.

[0081] like Figure 5 As shown, the audio data processing device 500 of this embodiment may include: a first processing module 501 and a playback module 502.

[0082] The first processing module 501 is configured to, in response to a successful connection with the mobile terminal, store the audio data to be played received from the mobile terminal into the intermediate buffer of the vehicle system via a preset thread; the playback module 502 is configured to retrieve the audio data to be played from the intermediate buffer and render and play it.

[0083] In the audio data processing apparatus 500 of this embodiment: the specific processing of the first processing module 501 and the playback module 502 and the resulting technical effects can be referred to respectively. Figure 2 The relevant descriptions of steps 201-202 in the corresponding embodiments will not be repeated here.

[0084] In some optional implementations of this embodiment, the first processing module 501 is further configured to: store the audio data to be played into an intermediate buffer through a preset thread according to a preset sampling interval duration.

[0085] In some optional implementations of this embodiment, the audio data processing device 500 may further include an acquisition module, a determination module, and a second processing module (not shown in the figure). The acquisition module is configured to acquire the operation duration corresponding to the current time the audio data to be played is stored in the intermediate buffer via a preset thread; the determination module is configured to determine the difference between a preset sampling interval duration and a target duration as the current sleep duration; wherein the target duration is the sum of the operation duration and the error between historical sleep durations; the second processing module is configured to perform a sampling sleep operation within the current sleep duration in response to the current sleep duration being greater than a preset value; wherein the sampling sleep operation includes suspending the operation of storing the audio data to be played in the intermediate buffer according to the preset sampling interval duration.

[0086] In the audio data processing apparatus 500 of this embodiment, the specific processing of the acquisition module, the determination module, and the second processing module, and the resulting technical effects, can be referred to respectively. Figure 3 The relevant descriptions of steps 301-304 in the corresponding embodiments will not be repeated here.

[0087] In some optional implementations of this embodiment, the acquisition module is further configured to acquire the actual execution duration corresponding to the sampling sleep operation; the determination module is further configured to determine the difference between the actual execution duration and the current sleep duration as the current sleep duration error; wherein the current sleep duration error is used as a new historical sleep duration error.

[0088] In some optional implementations of this embodiment, the second processing module is further configured to, in response to the current sleep duration being less than or equal to a preset value, set the value of the historical sleep duration error to 0, and continue to perform the operation of storing the audio data to be played into the intermediate buffer according to the preset sampling interval duration.

[0089] In some optional implementations of this embodiment, the initial storage space size of the intermediate buffer corresponds to the amount of audio data for the first preset duration; and the first processing module 501 is further configured to adjust the storage space size of the intermediate buffer according to the audio type of the audio data to be played by a preset thread in response to the data amount of the audio data to be played being greater than the data amount for the first preset duration.

[0090] In some optional implementations of this embodiment, the first processing module 501 is further configured to: in response to the audio data to be played belonging to a first audio type, expand the initial storage space size to the first storage space size through a preset thread; wherein the first storage space size corresponds to the amount of audio data of a second preset duration; wherein the first audio type includes non-dialogue types; in response to the audio data to be played belonging to a second audio type, expand the initial storage space size to the second storage space size through a preset thread; wherein the second audio type includes dialogue types, and the second storage space size corresponds to the amount of audio data of a third preset duration, wherein the third preset duration is longer than the first preset duration and shorter than the second preset duration.

[0091] In some optional implementations of this embodiment, the second preset duration is less than or equal to the fourth preset duration.

[0092] In some optional implementations of this embodiment, the first processing module 501 is further configured to store the audio data to be played, received from the mobile terminal and located in the vehicle's in-vehicle interconnection system, into an intermediate buffer via a preset thread.

[0093] In some optional implementations of this embodiment, the playback module 502 is further configured to obtain the audio data to be played from the intermediate buffer through the vehicle's preset audio playback interface and render and play it.

[0094] This embodiment exists as a device embodiment corresponding to the above method embodiment. The audio data processing device 500 provided in this embodiment, in response to a successful connection with a mobile terminal, can first store the read audio data received from the mobile terminal in an intermediate buffer added to the vehicle system through a preset thread dedicated to reading the audio data to be played. When there is a need to play audio, the audio data to be played is retrieved from the intermediate buffer for rendering data. This avoids direct interaction with the mobile terminal when retrieving the audio data to be played. By retrieving the audio data from the intermediate buffer of the vehicle system for rendering and playback, the time used to call the audio data is reduced, the data access efficiency is increased, and the sampling of the audio data can be effectively controlled to ensure the playback effect.

[0095] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the audio data processing method described in any of the above embodiments.

[0096] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the audio data processing method described in any of the above embodiments when executed.

[0097] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the audio data processing method described in any of the above embodiments.

[0098] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as in-vehicle systems in vehicles. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0099] like Figure 6 As shown, the electronic device 600 includes a processing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0100] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as a touch screen; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as an embedded multi-media card (EMMC), universal flash storage (UFS), etc.; and communication unit 609, such as a network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] Processing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processing unit 601 performs the various methods and processes described above, such as the interface presentation method. For example, in some embodiments, the interface presentation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by processing unit 601, one or more steps of the interface presentation method described above may be performed. Alternatively, in other embodiments, processing unit 601 may be configured to perform the interface presentation method by any other suitable means (e.g., by means of firmware).

[0102] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0106] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0107] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud microprocessor, a microprocessor product within the cloud computing service system, designed to address the shortcomings of traditional physical microprocessors and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0108] According to the technical solution of this disclosure embodiment, in response to a successful connection with a mobile terminal, a preset thread dedicated to reading the audio data to be played received from the mobile terminal can first store the read audio data in an intermediate buffer added to the vehicle system. When there is a need to play audio, the audio data to be played is retrieved from the intermediate buffer for rendering data. This avoids direct interaction with the mobile terminal when retrieving the audio data to be played. By retrieving the audio data from the intermediate buffer of the vehicle system for rendering and playback, the time used to call the audio data is reduced, and the data access efficiency is increased. This effectively controls the sampling of the audio data and ensures the playback effect.

[0109] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not a limitation herein; and the terms "first," "second," "third" (if any) in this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor do they constitute a specific limitation.

[0110] It should also be understood that expressions such as "comprising," "including," "having," "containing," and / or "comprising" are open-ended rather than closed-ended expressions in this disclosure, indicating the presence of the stated features, elements, and / or components, but not excluding the presence of one or more other features, elements, components, and / or combinations thereof. Furthermore, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features, not just individual elements in the list. Additionally, when describing embodiments of this disclosure, the word "may" is used to mean "one or more embodiments of this disclosure." And the term "exemplary" is intended to refer to an example or illustration.

[0111] Unless otherwise specified, all terms used herein (including engineering and technical terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should also be understood that, unless expressly stated in this disclosure, terms as defined in common dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or overly formalized meaning.

[0112] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An audio data processing method, applied to an in-vehicle infotainment system, comprising: In response to a successful connection with the mobile terminal, the audio data to be played received from the mobile terminal is stored in the intermediate buffer of the vehicle system via a preset thread; The audio data to be played is obtained from the intermediate buffer and rendered for playback.

2. The method according to claim 1, wherein, The step of storing the audio data to be played received from the mobile terminal into the intermediate buffer of the vehicle system via a preset thread includes: The preset thread stores the audio data to be played into the intermediate buffer according to the preset sampling interval.

3. The method according to claim 2, further comprising: Get the operation duration corresponding to the current time when the audio data to be played is stored in the intermediate buffer through the preset thread; The difference between the preset sampling interval and the target duration is determined as the current sleep duration; wherein, the target duration is the sum of the operation duration and the historical sleep duration error; In response to the current sleep duration being greater than a preset value, a sampling sleep operation is performed within the current sleep duration; wherein, the sampling sleep operation includes suspending the operation of storing the audio data to be played into the intermediate buffer according to the preset sampling interval duration.

4. The method according to claim 3, further comprising: Obtain the actual execution time corresponding to the sampling sleep operation; The difference between the actual execution time and the current sleep time is determined as the current sleep time error; wherein, the current sleep time error is used as a new historical sleep time error.

5. The method according to claim 3, further comprising: In response to the current sleep duration being less than or equal to the preset value, the value of the historical sleep duration error is set to 0, and the operation of storing the audio data to be played into the intermediate buffer according to the preset sampling interval duration continues.

6. The method according to claim 1, wherein, The initial storage space size of the intermediate buffer corresponds to the amount of audio data for the first preset duration; as well as The method further includes: in response to the data volume of the audio data to be played being greater than the data volume of the first preset duration, adjusting the storage space size of the intermediate buffer area according to the audio type of the audio data to be played through the preset thread.

7. The method according to claim 6, wherein, The step of adjusting the storage space size of the intermediate buffer according to the audio type of the audio data to be played via the preset thread includes: In response to the audio data to be played belonging to a first audio type, the initial storage space size is expanded to the first storage space size through the preset thread; wherein, the first storage space size corresponds to the amount of audio data of a second preset duration; wherein, the first audio type includes non-dialogue types; In response to the fact that the audio data to be played belongs to the second audio type, the initial storage space size is expanded to the second storage space size through the preset thread; wherein, the second audio type includes dialogue type, and the second storage space size corresponds to the amount of audio data of the third preset duration, and the third preset duration is longer than the first preset duration and shorter than the second preset duration.

8. The method according to claim 7, wherein, The second preset duration is less than or equal to the fourth preset duration.

9. The method according to any one of claims 1 to 8, wherein, The step of storing the audio data to be played received from the mobile terminal into the intermediate buffer of the vehicle system via a preset thread includes: The preset thread stores the audio data to be played, received from the mobile terminal and located in the preset in-vehicle connectivity system of the vehicle, into the intermediate buffer.

10. The method according to any one of claims 1 to 8, wherein, The step of retrieving the audio data to be played from the intermediate buffer and rendering and playing it includes: The audio data to be played is obtained from the intermediate buffer through the vehicle's preset audio playback interface and then rendered and played.

11. An audio data processing device, applied to an in-vehicle infotainment system, comprising: The first processing module is configured to, in response to a successful connection with the mobile terminal, store the audio data to be played received from the mobile terminal into the intermediate buffer of the vehicle system via a preset thread. The playback module is configured to retrieve the audio data to be played from the intermediate buffer and render and play it.

12. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the audio data processing method according to any one of claims 1-10.

13. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the audio data processing method according to any one of claims 1-10.

14. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the audio data processing method according to any one of claims 1-10.

Images