Audio transmission method, vehicle-mounted audio equipment, electronic equipment and vehicle

Abstract

The invention provides an audio transmission method, vehicle-mounted audio equipment, electronic equipment and a vehicle, and relates to the technical field of audio processing. The method comprises the following steps: acquiring a plurality of coded audios; each coded audio corresponds to one data packet; carrying out packet combination processing on the plurality of coded audios to obtain packet combination data; the packet combining data corresponds to one data packet; sending the combined data to the vehicle-mounted audio equipment through a preset transmission channel; the combined data is used for triggering the vehicle-mounted audio equipment to carry out subpackage processing on the combined data to obtain a plurality of coded audios. According to the audio transmission method, the data volume of the coded audio is greatly compressed, so that the coded audio with more data volume can be transmitted at one time through the transmission channel, the bandwidth utilization rate is improved, and the transmission speed of the coded audio is improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to an audio transmission method, a vehicle-mounted audio device, an electronic device and a vehicle. Background Art

[0002] With the rapid development of electronic technology, the interaction scenarios between electronic devices (such as mobile phones) and vehicle information systems (referred to as vehicle computers) are becoming more and more diverse. At present, there are already a variety of applications that support service access, audio and video transmission and playback between mobile phones and vehicle computers.

[0003] Typically, in audio transmission and playback scenarios, when a mobile phone transmits audio data to a vehicle computer, a transmission channel is reused, resulting in low bandwidth utilization and reduced transmission speed. Summary of the invention

[0004] The present application provides an audio transmission method, an in-vehicle audio device, an electronic device and a vehicle, which can transmit a larger amount of encoded audio data at one time through a transmission channel, thereby improving bandwidth utilization and increasing the transmission speed of the encoded audio.

[0005] In a first aspect, the present application provides an audio transmission method applied to an application framework layer of an electronic device, wherein the electronic device is connected to a vehicle-mounted audio device, the method comprising: obtaining multiple encoded audios; performing packet processing on the multiple encoded audios to obtain packetized data; the packetized data corresponds to a data packet; sending the packetized data to the vehicle-mounted audio device through a preset transmission channel; the packetized data is used to trigger the vehicle-mounted audio device to perform packet processing on the packetized data to obtain multiple encoded audios.

[0006] The audio transmission method provided in the embodiment of the present application performs packet processing on the encoded audio, which greatly compresses the data volume of the encoded audio, enables more data volume of encoded audio to be transmitted through the transmission channel at one time, improves bandwidth utilization, and increases the transmission speed of the encoded audio.

[0007] Optionally, the multiple encoded audios in the vehicle audio device are used to trigger the vehicle audio device to decode the multiple encoded audios through a decoder created for the multiple encoded audios to obtain multiple decoded audios; the multiple decoded audios are used by the vehicle audio device to play the multiple decoded audios according to the playback strategy;

[0008] Optionally, the playback strategy includes independently adjusting the playback volume corresponding to each decoded audio, and / or playing the decoded audio corresponding to the designated speaker through the designated speaker.

[0009] In this implementation, the vehicle audio device decodes the encoded audio through a decoder created for the encoded audio, and the decoded audio obtained is a separate audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play and adjust each audio, for example, it can adjust the volume of music, navigation sound and call sound respectively, and can specify different speakers to play different types of audio. As a result, many optimization scenarios for improving user experience can be realized between electronic devices and vehicles, which can meet the personalized needs of vehicle audio playback and improve the driving experience of vehicle drivers and passengers.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the audio transmission method provided in the embodiments of the present application, before obtaining multiple encoded audios, also includes: receiving audio data generated by multiple applications; processing the audio data into sub-audio data; encoding the sub-audio data by using a corresponding number of encoders created for the sub-audio data to obtain multiple encoded audios.

[0011] Optionally, the audio data may include at least one of media audio data, navigation audio data, call audio data, notification audio data, warning audio data, prompt audio data and ringtone audio data.

[0012] Optionally, the sub-audio data may include at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data.

[0013] The mixed sub-audio data is obtained by mixing other sub-audio data except the media sub-audio data, the navigation sub-audio data and the call sub-audio data.

[0014] Optionally, the other sub-audio data may include at least one of notification sub-audio data, warning sub-audio data, prompt sub-audio data and ringtone sub-audio data.

[0015] In this implementation, the electronic device processes the audio data into individual audio track data, which is beneficial for subsequent in-vehicle audio devices to independently play, adjust, and perform other operations on each audio.

[0016] In combination with the first aspect, in some implementations of the first aspect, the audio transmission method provided by the embodiment of the present application further includes: creating a corresponding input buffer for each encoder, where the input buffer corresponding to each encoder is used to store the encoded audio corresponding to each encoder;

[0017] Acquiring multiple encoded audios includes: reading the encoded audios from each input buffer in each preset period to obtain the multiple encoded audios.

[0018] Optionally, the preset period is the same as the storage duration of each encoded audio in each input buffer.

[0019] In this implementation, corresponding input buffers are created for different encoders, so that one buffer only needs to store encoded audio of one audio type, thereby ensuring that operations such as reading and storing encoded audio of different audio types are not interfered with.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the multiple encoded audios are packaged together to obtain packaged data, including: setting a data header for each encoded audio; splicing the data header corresponding to each encoded audio and the data packet corresponding to each encoded audio to obtain the packaged data.

[0021] In this implementation, the encoded audio is packetized, which greatly compresses the data volume of the encoded audio, which is conducive to the subsequent transmission of more encoded audio data at a time through the transmission channel, thereby improving bandwidth utilization and increasing the transmission speed of the encoded audio.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the combined data is sent to the vehicle audio device through a preset transmission channel, including: within a preset period, after detecting that the combined data is successfully acquired, the combined data is sent to the vehicle audio device through a preset transmission channel.

[0023] In this implementation, the combined data is transmitted after it is detected that the combined data is successfully acquired, thereby ensuring the success rate of transmitting the combined data.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the audio transmission method provided in an embodiment of the present application further includes: receiving an instruction to disconnect the electronic device from the vehicle-mounted audio device, and destroying each input buffer.

[0025] In this implementation, the created input buffer is destroyed in time, which effectively prevents the input buffer from occupying system resources for a long time, improves resource utilization, and enhances the performance of the electronic device.

[0026] In the second aspect, the present application provides an audio transmission method, which is applied to the application framework layer of a vehicle-mounted audio device, and the vehicle-mounted audio device is connected to an electronic device, and the method includes: receiving packaged data sent through a preset transmission channel; the packaged data corresponds to a data packet; the packaged data is obtained by the application framework layer of the electronic device by package processing multiple encoded audios; the packaged data is sub-packetized to obtain multiple encoded audios; each encoded audio corresponds to a data packet.

[0027] In this implementation, since the received combined data is obtained after the electronic device combines multiple encoded audios, this processing greatly compresses the data volume of the encoded audio, so that the vehicle audio device can receive more encoded audio data at a time, improves bandwidth utilization, and increases the transmission speed of the encoded audio.

[0028] In combination with the second aspect, in certain implementations of the second aspect, the packaged data includes a data header corresponding to each encoded audio and a data packet corresponding to each encoded audio, and the packaged data is sub-packetized to obtain multiple encoded audios, including: parsing the data header corresponding to each encoded audio in the packaged data; and parsing the data packet corresponding to each encoded audio according to the data packet length of the encoded audio in each data header.

[0029] In this implementation, by parsing the data header in the combined data package, each encoded audio can be accurately and quickly parsed, which is conducive to the subsequent vehicle audio equipment to quickly decode and process the encoded audio, so as to independently play, adjust and perform other operations on each audio.

[0030] In combination with the second aspect, in certain implementations of the second aspect, the audio transmission method provided in an embodiment of the present application also includes: decoding each encoded audio through a decoder created for each encoded audio to obtain multiple decoded audios; playing multiple decoded audios according to a playback strategy; the playback strategy includes independently adjusting the playback volume corresponding to each decoded audio, and / or playing the decoded audio corresponding to the specified speaker through the specified speaker.

[0031] In this implementation, the vehicle audio device decodes the encoded audio through a decoder created for the encoded audio, and the decoded audio obtained is a separate audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play and adjust each audio, for example, it can adjust the volume of music, navigation sound and call sound respectively, and can specify different speakers to play different types of audio. As a result, many optimization scenarios for improving user experience can be realized between electronic devices and vehicles, which can meet the personalized needs of vehicle audio playback and improve the driving experience of vehicle drivers and passengers.

[0032] In a third aspect, the present application provides an electronic device, comprising: one or more processors; one or more memories; a module with multiple applications installed; the memory stores one or more programs, and when one or more programs are executed by the processor, the electronic device executes the method in the above-mentioned first aspect and any possible implementation thereof.

[0033] In a fourth aspect, the present application provides a vehicle-mounted audio device, which includes: one or more processors; one or more memories; a module with multiple application programs installed; the memory stores one or more programs, and when one or more programs are executed by the processor, the vehicle-mounted audio device executes the method in the above-mentioned second aspect and any possible implementation thereof.

[0034] In a fifth aspect, the present application provides a vehicle comprising: one or more processors; one or more memories; the memories storing one or more programs, which, when one or more programs are executed by the processors, enable the vehicle to execute the method in the above-mentioned second aspect and any possible implementation thereof.

[0035] In a sixth aspect, the present application provides a chip, including a processor. The processor is used to read and execute a computer program stored in a memory to execute the method in the first aspect and any possible implementation thereof, or execute the method in the second aspect and any possible implementation thereof.

[0036] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0037] Optionally, the chip also includes a communication interface.

[0038] In the seventh aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes the method in the first aspect and any possible implementation thereof, or causes the processor to execute the method in the second aspect and any possible implementation thereof.

[0039] In an eighth aspect, the present application provides a computer program product, the computer program product comprising: a computer program code, when the computer program code is executed on an electronic device, the electronic device executes the method in the first aspect and any possible implementation thereof. Or, when the computer program code is executed on a vehicle audio device, the vehicle audio device executes the method in the second aspect and any possible implementation thereof.

[0040] The technical effects obtained in the above-mentioned second, third, fourth, fifth, sixth, seventh and eighth aspects can refer to the technical effects obtained by the corresponding technical means in the above-mentioned first and second aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of an application scenario of the audio processing method shown in an embodiment of the present application;

[0042] Figure 2A schematic diagram of another application scenario of the audio processing method shown in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a display interface shown in an embodiment of the present application;

[0044] Figure 4 A software structure block diagram of an electronic device shown as an exemplary embodiment of the present application;

[0045] Figure 5 A system architecture block diagram showing an exemplary embodiment of the present application;

[0046] Figure 6 A flowchart of an audio processing method shown in an embodiment of the present application;

[0047] Figure 7 A flowchart of another audio processing method according to an embodiment of the present application is shown;

[0048] Figure 8 Another system structure block diagram shown in an embodiment of the present application;

[0049] Fig. 9 A flowchart of another audio processing method according to an embodiment of the present application is shown;

[0050] Fig.10 A flowchart of another audio processing method according to an embodiment of the present application is shown;

[0051] Fig.11 A software structure block diagram of another electronic device shown as an exemplary embodiment of the present application;

[0052] Fig.12 This is a schematic diagram of package combining processing shown in an exemplary embodiment of the present application;

[0053] Fig.13 This is a schematic diagram of a sending cycle shown in an exemplary embodiment of the present application;

[0054] Fig.14 This is a schematic diagram of combined package data shown in an exemplary embodiment of the present application;

[0055] Fig.15 Another system architecture block diagram showing an exemplary embodiment of the present application;

[0056] Fig.16 A schematic diagram of subcontracting processing shown as an exemplary embodiment of the present application;

[0057] Fig.17 A schematic diagram of a flow chart of an audio transmission method shown in an embodiment of the present application;

[0058] Fig.18 A schematic diagram of a flow chart of another audio transmission method according to an embodiment of the present application;

[0059] Fig.19 A flowchart of another audio transmission method according to an embodiment of the present application is shown;

[0060] Fig. 20 A flowchart of another audio transmission method according to an embodiment of the present application is shown;

[0061] Fig.21 This is a schematic diagram of the hardware structure of an electronic device shown as an exemplary embodiment of the present application;

[0062] Fig. 22 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0064] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0065] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0066] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0067] In order to better understand the audio processing method and audio transmission method provided in the embodiments of the present application, some terms involved in the embodiments of the present application are first explained below to facilitate understanding by those skilled in the art.

[0068] 1. In-vehicle information system

[0069] The in-vehicle information system is also called a car information system, and is referred to as a vehicle computer in the embodiments of the present application.

[0070] The vehicle information system is a device that enables the driver to timely understand the vehicle operation information and external information during driving.

[0071] 2. Input Buffer

[0072] In the embodiment of the present application, it is also called inBuffer, which can be regarded as a memory area for temporarily storing data.

[0073] In the embodiment of the present application, one encoder corresponds to one inBuffer, and each inBuffer is used to receive and store the encoded audio sent by its corresponding encoder.

[0074] The above is a brief introduction to the terms involved in the embodiments of the present application, which will not be repeated below.

[0075] In the interaction scenario between electronic devices (such as mobile phones) and vehicle computers, such as in the scenario of audio transmission and playback, the mobile phone and the vehicle computer are in a connected state (or interconnected state). When different types of audio data (such as media audio data, navigation audio data, call audio data, notification audio data, etc.) are to be sent from the mobile phone to the vehicle computer, the mobile phone will mix these different types of audio data to obtain mixed data, and then send the mixed data to the vehicle computer.

[0076] Since audio mixing is an irreversible process, the car computer cannot separate the audio mixing data after receiving it, that is, it cannot separate the audio data of different types from the audio mixing data. For example, the car computer cannot separate the audio mixing data into separate media audio data, navigation audio data, call audio data, notification audio data, etc.

[0077] For the above reasons, the car computer can only play and adjust the mixed audio data in a unified manner, but cannot play and adjust each type of audio data separately, which results in many optimization scenarios between the mobile phone and the car computer for improving the user experience being unable to be realized, reducing the user's driving experience. Among them, the optimization scenarios for improving the user experience include but are not limited to the car computer controlling the playback volume of different types of audio data separately, the car computer controlling the navigation audio data to be played separately in the driver's seat speaker, and the car computer controlling the navigation audio data to be played in the driver's seat speaker and the co-driver's seat speaker.

[0078] In view of this, an embodiment of the present application provides an audio processing method, which is applied to an application framework layer of an electronic device, and the electronic device is connected to a vehicle-mounted audio device. The application framework layer of the electronic device receives audio data generated by multiple applications, and processes the audio data into sub-audio data, and the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data. Each sub-audio data is encoded by creating a corresponding number of encoders for the sub-audio data to obtain encoded audio, and the encoded audio is sent to the vehicle-mounted audio device through a preset transmission channel.

[0079] Since the data contained in the sub-audio data are individual audio track data (or each data contained in the sub-audio data is different types of audio data), these individual audio track data are encoded and sent to the vehicle audio device. The vehicle audio device decodes the encoded audio through the decoder created for the encoded audio, and the decoded audio obtained is also individual audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimization scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0080] The following describes the application scenarios of the audio processing method provided in the embodiments of the present application in conjunction with the accompanying drawings.

[0081] It is worth noting that in some embodiments of the present application, the electronic device may be a mobile phone, a smart screen, a tablet computer, a wearable device (such as a smart watch, smart glasses, a smart bracelet, a smart collar, etc.), a television, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc., or may be other devices or apparatuses capable of performing audio processing. The embodiments of the present application do not impose any restrictions on the specific type of the electronic device.

[0082] See also Figure 1 , Figure 1FIG. 1 is a schematic diagram of an application scenario of the audio processing method shown in the embodiment of the present application. Figure 1 As shown, the application scenario includes an electronic device 100 and a vehicle 200. It should be understood that the electronic device 100 is a mobile phone as an example for explanation in the embodiment of the present application.

[0083] Vehicle 200 can be any type of vehicle that carries people and / or objects and is moved by a power system such as a generator and a battery, including but not limited to cars, sedans, trucks, buses, electric cars, motorcycles, RVs, trains, EMU trains, high-speed EMU passenger trains, etc.

[0084] In one possible implementation, the vehicle 200 may be a vehicle driven by a driver. Alternatively, in another possible implementation, the vehicle 200 may also be a vehicle with certain automatic driving capabilities.

[0085] The vehicle 200 may include a vehicle audio device 210. The vehicle audio device 210 and the electronic device 100 may be connected via communication to achieve information exchange between the vehicle audio device 210 and the electronic device 100.

[0086] The communication connection may include a wired communication connection and a short-range wireless communication connection. The wired communication connection may include a Universal Serial Bus (USB) connection, and the short-range wireless communication connection may include but is not limited to: a Bluetooth connection, a Wireless Fidelity (Wi-Fi) connection, a Wi-Fi Peer-to-Peer (P2P) connection, a Zigbee network connection, and a Near Field Communication (NFC) connection.

[0087] In one possible implementation, the vehicle audio device 210 can be carried in a hardware device of the vehicle 200 in the form of software, and the hardware device can be used to access, store, and play the content in the application of the electronic device 100. In another possible implementation, the vehicle audio device 210 can also be implemented as other independent hardware devices that can be connected to the vehicle 200.

[0088] above Figure 1 The corresponding embodiment mainly demonstrates the application scenario of the audio processing method from the perspective of the outside of the vehicle 200. The following demonstrates the application scenario of the audio processing method from the perspective of the inside of the vehicle 200.

[0089] See also Figure 2 , Figure 2 FIG. 1 is another schematic diagram of an application scenario of the audio processing method shown in the embodiment of the present application. Figure 2 As shown, when the in-vehicle audio device 210 is implemented as other independent hardware devices that can be connected to the vehicle 200, the in-vehicle audio device 210 can be set on the right side of the steering wheel of the vehicle 200. This is only an exemplary description, and the in-vehicle audio device 210 can also be set in front of the driver, in front of the co-pilot, etc., without limitation.

[0090] For example, before the driver or passenger wants to transfer the content in the application of the electronic device 100 to the in-vehicle audio device 210, or wants to use the in-vehicle audio device 210 to access, store, play, etc. the content in the application of the electronic device 100, it is necessary to first establish a communication connection between the in-vehicle audio device 210 and the electronic device 100.

[0091] For example, a USB cable may be used to establish a wired communication connection between the vehicle audio device 210 and the electronic device 100. It should be understood that, whether it is the first time to establish a wired communication connection or not, one end of the flat interface of the USB cable is connected to the vehicle audio device 210, and one end of the micro interface of the USB cable is connected to the electronic device 100. Afterwards, operations are performed according to the prompt information for establishing a wired communication connection that pops up on the display interface of the vehicle audio device 210 and / or the display interface of the electronic device 100, thereby establishing a wired communication connection between the vehicle audio device 210 and the electronic device 100.

[0092] For another example, Bluetooth can be used to establish a short-range wireless communication connection between the in-vehicle audio device 210 and the electronic device 100. Exemplarily, when Bluetooth is used to establish a short-range wireless communication connection between the in-vehicle audio device 210 and the electronic device 100 for the first time, the driver or passenger is required to manually perform a pairing connection operation for the in-vehicle audio device 210 and the electronic device 100. For example, the driver or passenger can turn on the Bluetooth of the in-vehicle audio device 210 and the electronic device 100, so that the in-vehicle Bluetooth of the in-vehicle audio device 210 is turned on, and the Bluetooth of the electronic device 100 is turned on, and it is ensured that the in-vehicle Bluetooth of the in-vehicle audio device 210 and the Bluetooth of the electronic device 100 are both in a discoverable state.

[0093] After that, click Search Device on the Bluetooth interface displayed on the electronic device 100 so that the electronic device 100 can search for the vehicle-mounted Bluetooth. Select the vehicle-mounted Bluetooth displayed on the electronic device 100, click the vehicle-mounted Bluetooth, Bluetooth settings, and pairing operations in sequence, and then a prompt box for entering a pairing code will pop up on the display interface of the electronic device 100. Enter the pairing code in the prompt box and click Connect, and click Pairing on the Bluetooth interface displayed on the vehicle-mounted audio device 210. If the pairing code is entered correctly, it will be displayed as connected in the Bluetooth interface displayed on the vehicle-mounted audio device 210, thereby establishing a short-range wireless communication connection between the vehicle-mounted audio device 210 and the electronic device 100.

[0094] It is worth noting that when Bluetooth is not used for the first time to establish a short-range wireless communication connection between the in-vehicle audio device 210 and the electronic device 100, if both the in-vehicle Bluetooth of the in-vehicle audio device 210 and the Bluetooth of the electronic device 100 are turned on, and the distance between the in-vehicle audio device 210 and the electronic device 100 meets the preset connection threshold, the in-vehicle Bluetooth of the in-vehicle audio device 210 and the Bluetooth of the electronic device 100 automatically establish a short-range wireless communication connection.

[0095] It is understandable that the above-mentioned “preset connection threshold” refers to the maximum transmission distance for Bluetooth communication between the in-vehicle audio device 210 and the electronic device 100, such as within 8 meters, within 10 meters, within 15 meters, etc.

[0096] After the communication connection between the in-vehicle audio device 210 and the electronic device 100 is established, the content in the application of the electronic device 100 can be transmitted to the in-vehicle audio device, and the transmitted content can also be displayed on the display interface of the in-vehicle audio device 210. Alternatively, the in-vehicle audio device 210 can be used to access, store, play, etc. the content in the application of the electronic device 100, and the operation results can also be displayed on the display interface of the in-vehicle audio device 210.

[0097] The following describes the display interface of the vehicle audio device 210 in conjunction with the accompanying drawings. It is understandable that when the vehicle audio device 210 and the electronic device 100 do not establish a communication connection, the display interface of the vehicle audio device 210 can also independently display relevant information of the vehicle 200. The relevant information includes, but is not limited to, navigation information, driving information, audio information, video information, call information, power information, time information, volume information, etc.

[0098] In the embodiment of the present application, the content displayed on the display interface of the in-vehicle audio device 210 is described as an example when the in-vehicle audio device 210 and the electronic device 100 establish a communication connection. Figure 3 , Figure 3This is a schematic diagram of a display interface shown in an embodiment of the present application. For example, a media application (such as a music application) in the electronic device 100 sends music to the in-vehicle audio device 210, and the display interface of the in-vehicle audio device 210 displays the relevant content of the music, such as Figure 3 As shown, the display interface of the vehicle audio device 210 displays the song name, singer name, song category (such as I like, local songs, driving radio, etc.), and controls such as previous song, pause, and next song. At the same time, music is played through the speaker of the vehicle 200, realizing an interactive scenario of playing music in the music application of the electronic device 100 through the speaker of the vehicle 200.

[0099] For another example, a navigation application (such as a navigation application) in the electronic device 100 sends navigation data to the in-vehicle audio device 210, and the display interface of the in-vehicle audio device 210 displays the relevant content of the navigation data, such as Figure 3 As shown, the display interface of the vehicle audio device 210 displays route information, such as "turn left at 50km and enter XX Road", and can also display driving time (Time), distance to the destination (Distance), arrival time (Arrival), "end navigation" control, etc. At the same time, the navigation audio is played through the speaker of the vehicle 200, realizing an interactive scenario of playing the navigation sound in the navigation application of the electronic device 100 through the speaker of the vehicle 200.

[0100] For another example, a call application (such as a phone application) in the electronic device 100 sends call data to the in-vehicle audio device 210, and the display interface of the in-vehicle audio device 210 displays relevant controls for controlling the call data, such as Figure 3 As shown, the display interface of the vehicle audio device 210 displays controls such as "Answer", "Hang Up", and "Mute". According to the click operation of the driver or passenger on different controls, the vehicle 200 adopts different response methods. For example, when the driver or passenger clicks the "Answer" control, the call sound is played through the speaker of the vehicle 200. At the same time, the driver or passenger can input his or her own voice through the microphone of the vehicle 200, realizing the interactive scene of answering the call of the electronic device 100 through the speaker and microphone of the vehicle 200.

[0101] like Figure 3 As shown, the display interface of the vehicle audio device 210 can also display power information, time information, alarm information, main menu information, headphone information, etc. This is only an exemplary description and is subject to the actual display and is not limited to this.

[0102] In the related art, when an electronic device sends different types of audio data such as music, navigation data, and call data to a vehicle audio device, the electronic device will mix these different types of audio data to obtain mixed data, and then send the mixed data to the vehicle audio device. Since mixing is an irreversible process, after receiving the mixed data, the vehicle audio device cannot separate the mixed data, that is, it cannot separate music, navigation data, and call data from the mixed data.

[0103] Therefore, the in-vehicle audio equipment can only play and adjust the mixed data in a unified manner, but cannot adjust each type of audio data individually. For example, the playback volume corresponding to the mixed data can only be increased or decreased in a unified manner, which results in the volume of music, navigation sound, and call sound being increased or decreased in a unified manner, but the volume of music, navigation sound, and call sound cannot be adjusted independently. For another example, the mixed data can only be played on the same speaker, and different speakers cannot be specified to play different audio data.

[0104] In the audio processing method provided in the embodiment of the present application, the electronic device processes the audio data generated by multiple applications into sub-audio data, and the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data. Each sub-audio data is encoded by creating a corresponding number of encoders for the sub-audio data to obtain encoded audio, and the encoded audio is sent to the vehicle audio device through a preset transmission channel.

[0105] Since the data contained in the sub-audio data are individual audio track data (or each data contained in the sub-audio data is different types of audio data), these individual audio track data are encoded and sent to the vehicle audio device. The vehicle audio device decodes the encoded audio through the decoder created for the encoded audio, and the decoded audio obtained is also individual audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimization scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0106] The following describes the audio processing method provided by the embodiment of the present application in combination with the software structure. Figure 4 , Figure 4The software structure block diagram of an electronic device shown in an exemplary embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the electronic device 100 is an Android system as an example for explanation. The Android system is divided into three layers, from top to bottom, namely, the application layer, the application framework layer, and the virtual hardware abstraction layer.

[0107] The application layer may include a series of application packages. In the embodiment of the present application, the application layer may include media applications, navigation applications, call applications, shopping applications, tool applications, other applications, etc.

[0108] Among them, media applications may include, but are not limited to, various music applications, various video applications, various short video applications, various social applications, various game applications, and various information applications.

[0109] Navigation applications may include, but are not limited to, various map applications, various travel applications, various taxi applications, and various positioning applications.

[0110] Call applications may include, but are not limited to, telephone applications and various Internet phone applications.

[0111] Shopping applications can include various shopping applications.

[0112] Tool applications may include, but are not limited to, various browser applications, various learning applications, various weather applications, various office applications, and various consulting applications.

[0113] Other applications refer to applications other than media applications, navigation applications, call applications, shopping applications, and tool applications. For example, other applications may include notification applications, short message applications, applications that can provide safety tips during driving, etc.

[0114] It is worth noting that in the embodiments of the present application, no matter which application is used, audio data can be generated during use.

[0115] like Figure 4 As shown, the application package may include media applications, navigation applications, other applications, and call applications. Among them, media applications may include music applications, video applications, and short video applications; navigation applications may include map applications, travel applications, and positioning applications; other applications may include notification applications and short message applications; and call applications may include phone applications.

[0116] It should be understood that different applications can generate different types of audio data, for example, media applications can generate media audio data, navigation applications can generate navigation audio data, call applications can generate call audio data, and other applications can generate notification audio data, warning audio data, ringtone audio data, etc. These different types of audio data will be transmitted to the audio framework of the application framework layer, and will be processed by the audio framework, virtual hardware abstraction layer, and device virtualization service in turn, and finally sent to the vehicle audio device.

[0117] Optionally, the application layer may also include a vehicle-mounted application. The vehicle-mounted application is built into the application layer of the electronic device, and when it is detected that the electronic device and the vehicle-mounted audio device have established a communication connection, the vehicle-mounted application sends an audio diversion instruction to the audio framework of the application framework layer, and sends a cross-device audio streaming capability start instruction to the device virtualization service of the application framework layer.

[0118] The audio splitting instruction is used to instruct the audio framework to process the audio data generated by multiple applications into sub-audio data. It can be understood that the audio splitting instruction is used to instruct the audio framework not to mix the media audio data, navigation audio data, and call audio data, and to mix other audio data (such as notification audio data, warning audio data, ringtone audio data, etc.).

[0119] The cross-device audio streaming capability startup instruction is used to instruct the device virtualization service to create corresponding encoders for different types of sub-audio data, and to encode the sub-audio data to obtain encoded audio.

[0120] Optionally, in a possible implementation, the vehicle-mounted application does not have a corresponding application display interface and is invisible to the user.

[0121] The application framework layer may include an audio framework and a device virtualization service. The audio framework is first introduced below.

[0122] The audio framework is used to receive audio diversion instructions sent by the vehicle-mounted application, and to receive different types of audio data sent by various applications in the application layer. It should be understood that the audio diversion instruction is sent to the audio framework immediately by the vehicle-mounted application when it senses that the electronic device and the vehicle-mounted audio device have established a communication connection. Therefore, for the audio framework, it first receives the audio diversion instruction sent by the vehicle-mounted application, and then receives different types of audio data sent by various applications in the application layer according to the user's use of each application.

[0123] Optionally, in one possible implementation, when the audio framework receives an audio diversion instruction sent by the vehicle application, the received media audio data, navigation audio data, call audio data, and other audio data (such as notification audio data, warning audio data, ringtone audio data, etc.) are no longer mixed.

[0124] Optionally, in another possible implementation, when the audio framework receives an audio diversion instruction sent by the vehicle application, it will no longer perform mixing processing on the received media audio data, navigation audio data, and call audio data, but will perform mixing processing on other received audio data (such as notification audio data, warning audio data, ringtone audio data, etc.).

[0125] The audio framework is also used to identify the audio type of each audio data. The audio type may include media audio type, navigation audio type, call audio type, and other audio types. Among them, other audio types may include notification audio type, warning audio type, ringtone audio type, etc.

[0126] Typically, the system of an electronic device will identify the audio data generated by each application at the application layer, and the audio framework can identify the audio type of each audio data according to the audio type identifier of each audio data.

[0127] For example, the system identifies the audio data generated by the music application as "media-music". When the audio framework receives the audio data, it recognizes that the audio type in the audio data is identified as "media-music" and determines that the audio type of the audio data is a media audio type.

[0128] For another example, the system identifies the audio data generated by the navigation application as "navigation" information. When the audio framework receives the audio data, it recognizes that the audio type in the audio data is identified as "navigation" and determines that the audio type of the audio data is a navigation audio type.

[0129] Optionally, in a possible implementation, an application whitelist is pre-set in the audio framework, and the package name of the navigation application can be recorded in the application whitelist. When the audio framework receives audio data, it determines the package name of the application that sent the audio data; if the package name is detected in the application whitelist, it determines that the audio type of the audio data is a navigation audio type. In this implementation, when the system does not identify the audio data generated by the navigation application, the application whitelist in the audio framework can be used to identify the audio type of the audio data.

[0130] The audio framework is also used to process the audio data generated by multiple applications into sub-audio data after identifying the audio type of each audio data. The sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed audio sub-audio data.

[0131] The mixed sub-audio data is obtained by mixing the sub-audio data other than the media sub-audio data, the navigation sub-audio data and the call sub-audio data by the audio framework.

[0132] For example, the audio data generated by multiple applications may include media audio data, navigation audio data, call audio data, notification audio data, warning audio data, ringtone audio data, etc. This is only an exemplary description, and the amount and type of audio data actually generated are related to the application used by the user and are not limited to this.

[0133] The audio framework recognizes that the audio type of media audio data is media audio type, the audio type of navigation audio data is navigation audio type, the audio type of call audio data is call audio type, and the audio types of notification audio data, warning audio data, and ringtone audio data are all other audio types.

[0134] The audio framework does not perform mixing processing on audio data of the media audio type, navigation audio type, and call audio type. In a possible implementation, after determining the audio types of the media audio data, navigation audio data, and call audio data, the audio framework directly sends the media audio data, navigation audio data, and call audio data to the virtual hardware abstraction layer.

[0135] In another possible implementation, after determining the audio types of the media audio data, navigation audio data, and call audio data, the audio framework determines the media audio data as media sub-audio data, the navigation audio data as navigation sub-audio data, and the call audio data as call sub-audio data, and then sends the media sub-audio data, the navigation sub-audio data, and the call sub-audio data to the virtual hardware abstraction layer.

[0136] The audio framework performs mixing processing on audio data of other audio types. Exemplarily, the audio framework performs mixing processing on notification audio data, warning audio data and ringtone audio data to obtain mixed sub-audio data, and then sends the mixed sub-audio data to the virtual hardware abstraction layer.

[0137] The virtual hardware abstraction layer creates a corresponding number of audio routes for the sub-audio data, or in other words, the virtual hardware abstraction layer creates a corresponding audio route for the sub-audio data of each audio type, so that the created audio route matches the audio type of each sub-audio data.

[0138] The audio route created by the virtual hardware abstraction layer is used to forward the sub-audio data corresponding to the audio route to the encoder matching the sub-audio data. The encoder is created by the device virtualization service in the application framework layer, which will be described in detail later.

[0139] Compared to the related art, in which mixed audio data (data obtained after the mobile phone mixes different types of audio) is forwarded through an audio route, in this implementation, corresponding audio routes are created for sub-audio data of different audio types, so that one type of audio route only needs to forward sub-audio data of one audio type. This not only helps the device virtualization service to quickly distinguish the audio type of the sub-audio data it receives, but also improves forwarding efficiency.

[0140] Optionally, in one possible implementation, if the audio framework does not perform mixing on the received audio data, but instead determines the media audio data as media sub-audio data, the navigation audio data as navigation sub-audio data, the call audio data as call sub-audio data, the notification audio data as notification sub-audio data, the warning audio data as warning sub-audio data, and the ringtone audio data as ringtone sub-audio data, the virtual hardware abstraction layer creates a corresponding audio route for each sub-audio data.

[0141] For example, the virtual hardware abstraction layer creates a first audio route for media sub-audio data, a second audio route for navigation sub-audio data, a third audio route for call sub-audio data, a fourth audio route for notification sub-audio data, a fifth audio route for alert sub-audio data, and a sixth audio route for ringtone sub-audio data.

[0142] Optionally, in another possible implementation, if the audio framework performs mixing processing on audio data of other audio types, the virtual hardware abstraction layer creates a first audio route for the media sub-audio data, a second audio route for the navigation sub-audio data, a third audio route for the mixed sub-audio data, and a call audio route for the call sub-audio data.

[0143] Optionally, when creating an audio route, the virtual hardware abstraction layer identifies the created audio route according to the audio type of the sub-audio data. This is helpful for distinguishing the created audio routes, and is also helpful for the subsequent audio framework to quickly forward the sub-audio data matching each audio route to each audio route according to the identification. It is also helpful for the device virtualization service to quickly distinguish the audio type of the sub-audio data it receives (it can be generally understood that the audio type of the sub-audio data forwarded from a certain audio route can be quickly determined according to the identification of the audio route).

[0144] It should be understood that the method of identifying the created audio routes is not limited. Exemplarily, different audio route names can be used to identify, for example, media audio route, navigation audio route, mixing audio route, call audio route, etc. Another example is the first audio route, the second audio route, the third audio route, the fourth audio route, etc. Another example is route 1, route 2, route 3, etc.

[0145] Optionally, different audio types can be set for different audio routes through the audio framework. For example, the audio type corresponding to the first audio route is set to the media audio type, the audio type corresponding to the second audio route is set to the navigation audio type, the audio type corresponding to the third audio route is set to the mixed audio type, the audio type corresponding to the call audio route is set to the call audio type, etc.

[0146] Optionally, in one possible implementation, the virtual hardware abstraction layer creates an audio route, and a corresponding audio route may be created in advance for sub-audio data of each audio type, and stored in the virtual hardware abstraction layer for a long time, so as to facilitate the subsequent rapid forwarding of each sub-audio data to the encoder through the created audio route.

[0147] Optionally, in another possible implementation, a cycle starts with the establishment of a connection between the electronic device and the vehicle audio device and ends with the disconnection of the electronic device and the vehicle audio device. In each cycle, the virtual hardware abstraction layer creates a corresponding audio route for each audio type of sub-audio data. For example, when the virtual hardware abstraction layer first receives sub-audio data of a certain audio type sent by the audio framework, it creates an audio route corresponding to the audio type. Afterwards, in the cycle, the sub-audio data corresponding to the audio route can be directly forwarded to the encoder through the audio route.

[0148] Optionally, when it is detected that the electronic device is disconnected from the vehicle audio device, the virtual hardware abstraction layer destroys the created audio route. In this implementation, the created audio route is destroyed at the end of a cycle, which effectively avoids the audio route from occupying system resources for a long time, improves resource utilization, and enhances the performance of the electronic device.

[0149] It is understandable that if the virtual hardware abstraction layer has never received sub-audio data of a certain audio type sent by the audio framework within a cycle, then the audio route corresponding to the audio type may not be created within the cycle. This implementation method creates the corresponding audio route when required, which can effectively reduce resource consumption and improve resource utilization.

[0150] Device virtualization services may include virtual audio services, virtual call services, and transmission channels.

[0151] The virtual audio service is used to receive each non-talking sub-audio data forwarded by the virtual hardware abstraction layer through each audio route, and send each non-talking sub-audio data to the corresponding encoder. Among them, the non-talking sub-audio data may include media sub-audio data, navigation sub-audio data, mixed sub-audio data, etc.

[0152] The virtual call service is used to receive call sub-audio data and send the call sub-audio data to the corresponding call encoder.

[0153] The application framework layer creates a corresponding number of encoders for the sub-audio data, or in other words, the application framework layer creates a corresponding encoder for each audio type of sub-audio data, so that the created encoder matches the audio type of each sub-audio data.

[0154] The encoder created by the application framework layer is used to encode the sub-audio data transmitted to the encoder to obtain the corresponding encoded audio. After that, the encoded audio is sent to the vehicle audio device through the transmission channel in the device virtualization service.

[0155] Optionally, in a possible implementation, if the audio framework does not perform mixing processing on the received audio data, and the virtual hardware abstraction layer creates corresponding audio routes for each sub-audio data, the device virtualization service creates corresponding encoders for each sub-audio data forwarded to the device virtualization service through these audio routes. For example, a first encoder is created for the media sub-audio data forwarded to the device virtualization service through the first audio route, and the first encoder is used to encode the media sub-audio data to obtain the encoded audio corresponding to the media sub-audio data; a second encoder is created for the navigation sub-audio data forwarded to the device virtualization service through the second audio route, and the second encoder is used to encode the navigation sub-audio data to obtain the encoded audio corresponding to the navigation sub-audio data; a third encoder is created for the call sub-audio data forwarded to the device virtualization service through the third audio route, and the third encoder is used to encode the call sub-audio data to obtain the encoded audio corresponding to the call sub-audio data. frequency; creating a fourth encoder for the notification sub-audio data forwarded to the device virtualization service through the fourth audio route, the fourth encoder is used to encode the notification sub-audio data to obtain the encoded audio corresponding to the notification sub-audio data; creating a fifth encoder for the warning sub-audio data forwarded to the device virtualization service through the fifth audio route, the fifth encoder is used to encode the warning sub-audio data to obtain the encoded audio corresponding to the warning sub-audio data; creating a sixth encoder for the ringtone sub-audio data forwarded to the device virtualization service through the sixth audio route, the sixth encoder is used to encode the ringtone sub-audio data to obtain the encoded audio corresponding to the ringtone sub-audio data.

[0156] Optionally, in another possible implementation, if the audio framework performs mixing processing on audio data of other audio types, the virtual hardware abstraction layer creates a first audio route for the media sub-audio data, a second audio route for the navigation sub-audio data, a third audio route for the mixing sub-audio data, and a call audio route for the call sub-audio data. The device virtualization service creates a corresponding encoder for each sub-audio data forwarded to the device virtualization service through these audio routes.

[0157] For example, a third encoder is created for the mixed sub-audio data forwarded to the device virtualization service through the third audio route, and the third encoder is used to encode the mixed sub-audio data to obtain the encoded audio corresponding to the mixed sub-audio data; a call encoder is created for the call sub-audio data forwarded to the device virtualization service through the call audio route, and the call encoder is used to encode the call sub-audio data to obtain the encoded audio corresponding to the call sub-audio data.

[0158] In this implementation, corresponding encoders are created for sub-audio data of different audio types, so that one type of encoder only needs to encode sub-audio data of one audio type, which can improve encoding efficiency and effectively avoid encoding jams, so that the audio played on the in-vehicle audio device is smoother. Among them, the encoding jam is caused by one type of encoder encoding sub-audio data of multiple audio types, which increases the waiting time for encoding each sub-audio data.

[0159] Optionally, when creating an encoder, the device virtualization service identifies the created encoder according to the audio type of the received sub-audio data or the identifier of the audio route. This is helpful in distinguishing the created encoders and is also helpful for the subsequent transmission channel to quickly send the encoded audio of each encoder to the decoder corresponding to the vehicle audio device according to the identifier.

[0160] It should be understood that the method of identifying the created encoder is not limited. Exemplarily, it can be identified by different encoder names, such as the first encoder, the second encoder, the third encoder, the call encoder, etc. For another example, the route 1 encoder, the route 2 encoder, the route 3 encoder, etc.

[0161] Optionally, in one possible implementation, the device virtualization service creates an encoder, and a corresponding encoder may be created in advance for the sub-audio data of each audio type, and stored in the device virtualization service for a long time, so as to facilitate the subsequent rapid encoding processing of the sub-audio data of each audio type through the created encoder.

[0162] Optionally, in another possible implementation, a cycle is defined as the time when the electronic device establishes a connection with the vehicle audio device and the time when the electronic device disconnects from the vehicle audio device. In each cycle, the device virtualization service creates a corresponding encoder for each audio type of sub-audio data. For example, when the device virtualization service first receives sub-audio data of a certain audio type forwarded by an audio route, it creates an encoder corresponding to the audio type. Afterwards, in the cycle, the sub-audio data of this audio type can be directly encoded by the encoder.

[0163] Optionally, when it is detected that the electronic device is disconnected from the vehicle audio device, the device virtualization service destroys the created encoder. In this implementation, the created encoder is destroyed at the end of a cycle, which effectively prevents the encoder from occupying system resources for a long time, improves resource utilization, and enhances the performance of the electronic device.

[0164] It is understandable that if the device virtualization service has never received sub-audio data of a certain audio type sent by a certain audio route within a cycle, then the encoder corresponding to the audio type may not be created within the cycle. This implementation method creates the corresponding encoder when there is demand, which can effectively reduce resource consumption and improve resource utilization.

[0165] In the aforementioned implementation, the virtual hardware abstraction layer creates a call audio route for the call sub-audio data, and the call sub-audio data can be forwarded to the encoder of the device virtualization service through the call audio route. Optionally, in a possible implementation, the electronic device may include an adaptive data signal processing (Adaptive Data SignalProcessing, ADSP) framework, and the call audio data generated by the call application is directly transmitted to the virtual hardware abstraction layer through the ADSP framework. The virtual hardware abstraction layer may also include a pre-established call channel, through which the call audio data is transmitted to the virtual call service of the device virtualization service. The virtual call service sends the call audio data to the corresponding call encoder. In this implementation, a separate call channel and a call encoder are established for the call audio data, which can ensure that the transmission process and encoding process of the call audio data are not interfered with, effectively improving the call quality.

[0166] The device virtualization service includes a preset transmission channel, which is used to send the encoded audio obtained after the encoding process of each encoder to the vehicle audio device. Exemplarily, the transmission channel may include multiple sending units, each sending unit is used to send a type of encoded audio.

[0167] For example, the transmission channel may include a first sending unit, a second sending unit, a third sending unit, and a fourth sending unit. The first sending unit sends the encoded audio corresponding to the media sub-audio data to the vehicle audio device; the second sending unit sends the encoded audio corresponding to the navigation sub-audio data to the vehicle audio device; the third sending unit sends the encoded audio corresponding to the mixing sub-audio data to the vehicle audio device; and the fourth sending unit sends the encoded audio corresponding to the call sub-audio data to the vehicle audio device.

[0168] In this implementation, each encoded audio is transmitted through a transmission channel, which reduces resource consumption and effectively saves resources of the electronic device.

[0169] Optionally, in one possible implementation, the transmission channel may include a first transmission channel and a second transmission channel, the first transmission channel being used to transmit encoded audio corresponding to non-call audio sub-data, and the second transmission channel being used to transmit encoded audio corresponding to call sub-audio data.

[0170] The first transmission channel may include a plurality of sending units, each sending unit being used to send a type of encoded audio. The non-talking sub-audio data may include media sub-audio data, navigation sub-audio data, and mixed audio sub-audio data.

[0171] In this implementation, a separate transmission channel is established for the call sub-audio data, which can ensure that the encoded audio corresponding to the call sub-audio data is not disturbed during the transmission process, thereby effectively improving the call quality.

[0172] The audio processing method provided in the embodiment of the present application is applied to the application framework layer of an electronic device, which is connected to a vehicle-mounted audio device. The application framework layer of the electronic device receives audio data generated by multiple applications, processes the audio data into sub-audio data, and the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data. Each sub-audio data is encoded by creating a corresponding number of encoders for the sub-audio data to obtain encoded audio, and the encoded audio is sent to the vehicle-mounted audio device through a preset transmission channel.

[0173] Since the data contained in the sub-audio data are individual audio track data (or each data contained in the sub-audio data is different types of audio data), these individual audio track data are encoded and sent to the vehicle audio device. The vehicle audio device decodes the encoded audio through the decoder created for the encoded audio, and the decoded audio obtained is also individual audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimization scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0174] The audio processing method provided by the embodiment of the present application is described above in combination with the software structure. The audio processing method provided by the embodiment of the present application is described below in combination with the system architecture. Figure 5 , Figure 5 This is a system architecture block diagram showing an exemplary embodiment of the present application. Figure 5 As shown in the system architecture diagram, Figure 4 Based on the software structure diagram of the corresponding electronic equipment, the software structure of the car audio equipment is added. Figure 4 The software structure of the corresponding electronic device will not be described in detail here. The software structure of the car audio device is described in detail below.

[0175] In the embodiment of the present application, the system of the vehicle audio device is divided into two layers, namely, an application layer and an application framework layer, and the application layer and the application framework layer communicate with each other through a software interface.

[0176] The application layer of the in-vehicle audio device may include an in-vehicle computer application, which may be used to sense that the electronic device and the in-vehicle audio device have established a communication connection, and sense that the electronic device and the in-vehicle audio device are disconnected.

[0177] The application framework layer of the in-vehicle audio device may include a device virtualization service and an audio framework.

[0178] Among them, the device virtualization service may include a transmission channel, a decoder, and an audio playback module.

[0179] The device virtualization service includes a transmission channel, which is used to transmit each encoded audio sent by the electronic device to the corresponding decoder. Exemplarily, the transmission channel may include a plurality of receiving units, each receiving unit is used to receive a type of encoded audio sent by the electronic device, and transmit the type of encoded audio to the corresponding decoder.

[0180] For example, the transmission channel may include a first receiving unit, a second receiving unit, a third receiving unit, and a fourth receiving unit. The first receiving unit transmits the encoded audio corresponding to the media sub-audio data; the second receiving unit transmits the encoded audio corresponding to the navigation sub-audio data; the third transmitting unit transmits the encoded audio corresponding to the mixing sub-audio data; and the fourth receiving unit transmits the encoded audio corresponding to the call sub-audio data.

[0181] In this implementation, each encoded audio is transmitted through a transmission channel, which reduces resource consumption and effectively saves resources of the electronic device.

[0182] Optionally, in one possible implementation, the transmission channel may include a third transmission channel and a fourth transmission channel, the third transmission channel being used to transmit the encoded audio corresponding to the non-call audio sub-data sent by the electronic device, and the fourth transmission channel being used to transmit the encoded audio corresponding to the call sub-audio data sent by the electronic device.

[0183] The third transmission channel may include a plurality of receiving units, each receiving unit being used to receive a type of encoded audio and transmit a type of encoded audio to a corresponding decoder. The non-conversation sub-audio data may include media sub-audio data, navigation sub-audio data, and mixed audio sub-audio data.

[0184] In this implementation, a separate transmission channel is established for the encoded audio corresponding to the call sub-audio data, which can ensure that the encoded audio corresponding to the call sub-audio data is not disturbed during the transmission process, thereby effectively improving the call quality.

[0185] The application framework layer creates a corresponding number of decoders for the encoded audio, or in other words, the application framework layer creates a corresponding decoder for each audio type of the encoded audio, so that the created decoder matches the audio type of each encoded audio. It should be understood that the audio type of the encoded audio is the audio type corresponding to the sub-audio data encoded as the encoded audio.

[0186] The decoder created by the application framework layer is used to decode the encoded audio transmitted to the decoder to obtain decoded audio, and then send the decoded audio to the audio framework of the vehicle audio device.

[0187] Optionally, in a possible implementation, if the audio framework of the electronic device does not perform any mixing processing on the received audio data, multiple encoders are eventually created on the electronic device side, such as the first encoder, the second encoder, the third encoder, the fourth encoder, the fifth encoder and the sixth encoder as described above.

[0188] Correspondingly, the application framework layer of the vehicle audio device creates a first decoder for the encoded audio corresponding to the media sub-audio data, and the first decoder is used to decode and process the encoded audio corresponding to the media sub-audio data to obtain the decoded audio corresponding to the media sub-audio data; creates a second decoder for the encoded audio corresponding to the navigation sub-audio data, and the second decoder is used to decode and process the encoded audio corresponding to the navigation sub-audio data to obtain the decoded audio corresponding to the navigation sub-audio data; creates a third decoder for the encoded audio corresponding to the call sub-audio data, and the third decoder is used to decode and process the encoded audio corresponding to the call sub-audio data to obtain the decoded audio corresponding to the call sub-audio data; creates a fourth decoder for the encoded audio corresponding to the notification sub-audio data, and the fourth decoder is used to decode and process the encoded audio corresponding to the notification sub-audio data to obtain the decoded audio corresponding to the notification sub-audio data; creates a fifth decoder for the encoded audio corresponding to the warning sub-audio data, and the fifth decoder is used to decode and process the encoded audio corresponding to the warning sub-audio data to obtain the decoded audio corresponding to the warning sub-audio data; creates a sixth decoder for the encoded audio corresponding to the ringtone sub-audio data, and the sixth decoder is used to decode and process the encoded audio corresponding to the ringtone sub-audio data to obtain the decoded audio corresponding to the ringtone sub-audio data.

[0189] Optionally, in another possible implementation, if the audio framework of the electronic device performs mixing processing on audio data of other audio types, the electronic device side finally creates a first encoder, a second encoder, a third encoder and a call encoder. Accordingly, the application framework layer of the vehicle audio device creates a corresponding decoder for the encoded audio transmitted by each encoder.

[0190] For example, the electronic device side creates a third encoder for the mixed sub-audio data, and the application framework layer of the vehicle audio device creates a third decoder for the encoded audio corresponding to the mixed sub-audio data, and the third decoder is used to decode the encoded audio corresponding to the mixed sub-audio data to obtain the decoded audio corresponding to the mixed sub-audio data; the electronic device side creates a call encoder for the call sub-audio data, and the application framework layer of the vehicle audio device creates a call decoder for the encoded audio corresponding to the call sub-audio data, and the call decoder is used to decode the encoded audio corresponding to the call sub-audio data to obtain the decoded audio corresponding to the call sub-audio data.

[0191] In this implementation, corresponding decoders are created for coded audios of different audio types, so that one type of decoder only needs to decode coded audios of one audio type, effectively avoiding the occurrence of audio stuttering, thereby improving the smoothness of the final played audio and further improving the user experience. The audio stuttering phenomenon is caused by one type of decoder decoding coded audios of multiple audio types, which increases the waiting time for decoding each coded audio.

[0192] Optionally, when creating a decoder, the device virtualization service identifies the created decoder according to the audio type of the received encoded audio. This is helpful for distinguishing the created decoders and for the subsequent audio playback module to individually adjust and play the decoded audio transmitted by each decoder.

[0193] It should be understood that the way to identify the created decoder is not limited. Exemplarily, it can be identified by different decoder names, such as the first decoder, the second decoder, the third decoder, the call decoder, etc. For another example, route 1 decoder, route 2 decoder, route 3 decoder, etc.

[0194] Optionally, in one possible implementation, the device virtualization service creates a decoder, and a corresponding decoder may be created in advance for the encoded audio of each audio type, and stored in the device virtualization service for a long time, so as to facilitate the subsequent rapid decoding processing of the encoded audio of each audio type through the created decoder.

[0195] Optionally, in another possible implementation, a cycle is defined as the time when the electronic device establishes a connection with the vehicle audio device and the time when the electronic device disconnects from the vehicle audio device. In each cycle, the device virtualization service creates a corresponding decoder for each type of coded audio. For example, when the device virtualization service first receives coded audio of a certain type of audio sent by the electronic device, it creates a decoder corresponding to the audio type. After that, in the cycle, the coded audio of this audio type can be directly decoded by the decoder.

[0196] Optionally, when it is detected that the electronic device is disconnected from the vehicle audio device, the device virtualization service destroys the created decoder. In this implementation, the created decoder is destroyed at the end of a cycle, which effectively prevents the decoder from occupying system resources for a long time, improves resource utilization, and enhances the performance of the vehicle audio device.

[0197] It is understandable that if the device virtualization service has never received encoded audio of a certain audio type sent by an electronic device within a cycle, a decoder corresponding to the audio type may not be created within the cycle. This implementation method creates a corresponding decoder when needed, which can effectively reduce resource consumption and improve resource utilization.

[0198] The audio playback module is an interface encapsulated by the vehicle audio device, which is used to temporarily store each decoded audio and also to send each decoded audio to the audio framework. Exemplarily, each decoder sends the decoded audio obtained after decoding to the audio playback module, and the audio playback module temporarily stores each decoded audio, and sends each temporarily stored decoded audio to the audio framework at intervals of a preset duration.

[0199] It is understandable that the above-mentioned "preset duration" can be set and adjusted according to actual conditions. For example, the preset duration can be 10 milliseconds, 20 milliseconds, 30 milliseconds, etc.

[0200] The audio framework obtains the playback strategy, and after receiving each decoded audio sent by the audio playback module, plays and / or adjusts the decoded audio according to the playback strategy. The playback strategy may include independently adjusting the playback volume corresponding to the decoded audio, and / or playing the decoded audio corresponding to the specified speaker through the specified speaker.

[0201] For example, the playback strategy specifically includes increasing the playback volume of media audio (such as music audio), and the audio framework increases the playback volume of the decoded audio corresponding to the media sub-audio data, so that the volume of the media audio finally played by the speaker is increased.

[0202] For another example, the playback strategy specifically includes playing the navigation audio through the driver's seat speaker, the audio framework specifies that the driver's seat speaker plays the decoded audio corresponding to the navigation sub-audio data, and finally the driver's seat speaker plays the decoded audio corresponding to the navigation sub-audio data.

[0203] Optionally, in a possible implementation, a vehicle application of the vehicle audio device generates a playback strategy according to the needs of the driver or the passenger, and sends the playback strategy to the audio framework.

[0204] In the audio processing method provided in the embodiment of the present application, since the vehicle audio device receives not mixed audio data but individual encoded audio, after decoding the encoded audio, the decoded audio obtained is also individual audio (or in other words, each decoded audio obtained by decoding is an audio of a different audio type). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimized scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0205] The audio processing method provided in the embodiment of the present application is described above in conjunction with the system architecture. The audio processing method provided in the embodiment of the present application is described below in conjunction with the flow chart.

[0206] See also Figure 6 , Figure 6 The following is a flow chart of an audio processing method according to an embodiment of the present application. The method includes:

[0207] S101, sending an audio diversion instruction and a cross-device audio streaming capability start-up instruction.

[0208] The application layer of the electronic device may include a vehicle-mounted application. The vehicle-mounted application can sense the establishment or disconnection of a communication connection between the electronic device and the vehicle-mounted audio device. When sensing that a communication connection is established between the electronic device and the vehicle-mounted audio device, the vehicle-mounted application sends an audio diversion instruction to the audio framework of the application framework layer, and sends a cross-device audio streaming capability start instruction to the device virtualization service of the application framework layer.

[0209] The audio splitting instruction is used to instruct the audio framework to process the audio data generated by multiple applications into sub-audio data. In layman's terms, the audio splitting instruction is used to instruct the audio framework not to mix the media audio data, navigation audio data, and call audio data, but to mix other audio data (such as notification audio data, warning audio data, ringtone audio data, etc.).

[0210] The cross-device audio streaming capability startup instruction is used to instruct the device virtualization service to create corresponding encoders for sub-audio data of different audio types, and to encode the sub-audio data to obtain encoded audio.

[0211] Among them, the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data. The mixed sub-audio data is obtained by mixing other sub-audio data except media sub-audio data, navigation sub-audio data and call sub-audio data.

[0212] S102: Enable the audio splitting function and the cross-device audio streaming function.

[0213] After receiving the audio diversion instruction, the audio framework in the electronic device turns on the audio diversion function.

[0214] The audio splitting function refers to the function of the audio framework to process the audio data generated by multiple applications into sub-audio data. In layman's terms, the audio splitting function is the function of the audio framework not mixing media audio data, navigation audio data, and call audio data, but mixing other audio data (such as notification audio data, warning audio data, ringtone audio data, etc.).

[0215] It can be understood that after the audio diversion function is turned on, the audio framework does not mix the media audio data, navigation audio data and call audio data when receiving them, but mixes the notification audio data, warning audio data, ringtone audio data, etc. when receiving them.

[0216] Optionally, in a possible implementation, the audio diversion function may also refer to the function of the audio framework not performing any mixing processing on any audio data, which is beneficial for subsequent vehicle audio equipment to independently adjust and play decoded audio of any audio type.

[0217] After receiving the cross-device audio streaming capability start instruction, the device virtualization service in the electronic device starts the cross-device audio streaming function.

[0218] The cross-device audio streaming function refers to the function of the device virtualization service creating an encoder and using the encoder to perform encoding processing. Specifically, the cross-device audio streaming function refers to the function of the device virtualization service creating corresponding encoders for sub-audio data of different audio types, and using the encoder to encode the sub-audio data that matches the encoder.

[0219] S103: The user triggers an input operation.

[0220] Exemplarily, a plurality of applications are pre-installed in the electronic device. In the embodiment of the present application, the plurality of applications may be one or more of media applications, navigation applications, call applications, shopping applications, tool applications, other applications, etc.

[0221] In the embodiment of the present application, multiple applications are taken as media applications (such as music applications), navigation applications (such as map applications), call applications (such as telephone applications) and other applications (such as short message applications) for example.

[0222] It is worth noting that in the embodiments of the present application, no matter which application is used, audio data can be generated during use.

[0223] The input operation may be a click operation, for example, a user clicks an icon corresponding to an application in an electronic device.

[0224] Optionally, in an embodiment of the present application, the input operation may be an operation of starting an application through voice instructions, the input operation may be an operation of starting an application through gesture instructions, the input operation may be an operation of starting an application through eye instructions, and so on. The present application does not impose any limitations on this.

[0225] Exemplarily, after the electronic device detects the user's input operation on the icon corresponding to a certain application, it responds to the input operation triggered by the user and starts and runs the certain application. For example, the user can instruct the electronic device to start and run the music application by clicking the icon of the music application. After that, the music application starts playing music.

[0226] S104: Receive audio data generated by multiple applications.

[0227] Exemplarily, different applications generate different types of audio data when running, which may include media audio data, navigation audio data, call audio data, notification audio data, warning audio data, ringtone audio data, prompt audio data, etc.

[0228] For example, a media application may generate media audio data, a navigation application may generate navigation audio data, a call application may generate call audio data, and other applications may generate notification audio data, warning audio data, ringtone audio data, prompt audio data, and the like.

[0229] Among them, media audio data may include, but is not limited to, audio data generated by various music applications, various video applications, various short video applications, various social applications, various game applications, and various information applications.

[0230] Navigation audio data may include, but is not limited to, audio data generated by various map applications, various travel applications, various taxi applications, and various positioning applications.

[0231] The call audio data may include, but is not limited to, audio data generated by telephone applications and various Internet telephone applications.

[0232] After multiple applications in the application layer generate audio data, the audio data is sent to the audio framework of the application framework layer, and the audio framework receives the audio data generated by the multiple applications. It should be understood that the amount and type of audio data actually generated are related to the application used by the user and are not limited to this.

[0233] S105: Process the audio data into sub-audio data.

[0234] In the related art, after receiving audio data generated by multiple applications, the audio framework will mix all the received audio data regardless of the audio type, and finally send the mixed data to the vehicle audio device. Since mixing is an irreversible process, the vehicle audio device cannot split the mixed data after receiving the mixed data, that is, it cannot split the various types of audio data from the mixed data. The mixed data can only be played and adjusted uniformly, and each type of audio data cannot be played and adjusted separately, resulting in many optimization scenarios between electronic devices and vehicles for improving user experience being unable to be realized, reducing the user's driving experience.

[0235] In the audio processing method provided in the embodiment of the present application, when the audio framework receives audio data generated by multiple applications, it identifies the audio type of each audio data.

[0236] The audio type may include a media audio type, a navigation audio type, a call audio type, and other audio types. The other audio types may include a notification audio type, a warning audio type, a ringtone audio type, a prompt audio type, and the like.

[0237] In one possible implementation, the audio framework does not perform mixing processing on audio data of media audio type, navigation audio type, call audio type, and other audio types.

[0238] In another possible implementation, the audio framework does not mix audio data of media audio types, navigation audio types, and call audio types, but mixes audio data of other audio types (such as notification audio types, warning audio types, ringtone audio types, prompt audio types, etc.).

[0239] Exemplarily, when identifying the audio type of each audio data, usually, the system of the electronic device will identify the audio data generated by each application at the application layer, and the audio framework identifies the audio type of each audio data according to the audio type identifier carried by each audio data.

[0240] It should be understood that the audio type of each audio data is consistent with the audio type of the sub-audio data. Therefore, at least two of the media sub-audio data, navigation sub-audio data, call sub-audio data, and other sub-audio data can also be identified from the audio data according to different audio type identifiers carried by the audio data.

[0241] For example, after determining the audio types of media audio data, navigation audio data, and call audio data, the audio framework determines the media audio data as media sub-audio data, the navigation audio data as navigation sub-audio data, and the call audio data as call sub-audio data.

[0242] Exemplarily, if other sub-audio data are identified, mixing processing is performed on the other sub-audio data to obtain mixed sub-audio data.

[0243] For example, after determining the audio types of notification audio data, warning audio data, ringtone audio data, and prompt audio data, the audio framework determines the notification audio data as notification sub-audio data, the warning audio data as warning sub-audio data, the ringtone audio data as ringtone sub-audio data, and the prompt audio data as prompt sub-audio data. The notification sub-audio data, the warning sub-audio data, the ringtone sub-audio data, and the prompt sub-audio data are mixed to obtain mixed sub-audio data.

[0244] S106: Forward the sub-audio data through the audio routing.

[0245] The audio processing method provided in the embodiment of the present application also includes creating an audio route through a virtual hardware abstraction layer.

[0246] Exemplarily, the virtual hardware abstraction layer creates a corresponding number of audio routes for the sub-audio data, or in other words, the virtual hardware abstraction layer creates corresponding audio routes for the sub-audio data of each audio type, so that the created audio routes match the audio types of the respective sub-audio data.

[0247] Optionally, in a possible implementation, the audio framework does not perform any mixing processing on the received audio data, and the virtual hardware abstraction layer creates a corresponding audio route for each sub-audio data.

[0248] Optionally, in another possible implementation, the audio framework mixes audio data of other audio types, and the virtual hardware abstraction layer creates a first audio route for the media sub-audio data, a second audio route for the navigation sub-audio data, a third audio route for the mixed sub-audio data, and a call audio route for the call sub-audio data.

[0249] It can be understood that in some implementations of the present application, the information represented by the call audio data and the call sub-audio data is consistent, but the names are different.

[0250] S107. Forward the sub-audio data through the virtual audio service.

[0251] The device virtualization service may include a virtual audio service and a virtual call service. Exemplarily, the audio routing of the virtual hardware abstraction layer first transmits the non-call sub-audio data to the virtual audio service of the device virtualization service, and transmits the call sub-audio data to the virtual call service of the device virtualization service.

[0252] Among them, the non-call sub-audio data may include media sub-audio data, navigation sub-audio data, and mixed sub-audio data.

[0253] The virtual audio service transmits the non-call sub-audio data to the corresponding encoder, and the virtual call service transmits the call sub-audio data to the corresponding call encoder.

[0254] The audio processing method provided in the embodiment of the present application also includes creating an encoder through a device virtualization service in the application framework layer.

[0255] Exemplarily, the device virtualization service in the application framework layer creates a corresponding number of encoders for the sub-audio data, or in other words, the device virtualization service in the application framework layer creates corresponding encoders for the sub-audio data of each audio type, so that the created encoders match the audio types of each sub-audio data.

[0256] S108: Encode the sub-audio data and send the encoded audio.

[0257] In the embodiment of the present application, encoding processing refers to compressing the sub-audio data, which can reduce the data volume of the sub-audio data and is conducive to reducing the bandwidth when the encoded audio is subsequently transmitted.

[0258] Optionally, the encoding process may also be to convert the sub-audio data into a network signal.

[0259] Exemplarily, the sub-audio data matched with each encoder is encoded by the encoder created for the sub-audio data to obtain the encoded audio, and then the encoded audio is sent to the transmission channel.

[0260] S109: Send the encoded audio through a transmission channel.

[0261] The device virtualization service includes a preset transmission channel, which is used to send the encoded audio obtained after the encoding process of each encoder to the vehicle audio device. Exemplarily, the transmission channel may include multiple sending units, each sending unit is used to send a type of encoded audio.

[0262] In this implementation, each encoded audio is transmitted through a transmission channel, which reduces resource consumption while ensuring performance and effectively saves resources of the electronic device.

[0263] Optionally, in a possible implementation, different transmission channels are established for coded audio of different audio types, and each transmission channel is used to transmit coded audio of one audio type. In this implementation, coded audio of multiple different audio types can be transmitted at the same time, which improves the transmission rate, and each transmission channel is used to transmit coded audio of one audio type, which effectively avoids mutual interference of the coded audio during transmission.

[0264] Optionally, in one possible implementation, the transmission channel may include a first transmission channel and a second transmission channel, the first transmission channel being used to transmit encoded audio corresponding to non-call audio sub-data, and the second transmission channel being used to transmit encoded audio corresponding to call sub-audio data.

[0265] The first transmission channel may include a plurality of sending units, each sending unit being used to send a type of encoded audio. The non-talking sub-audio data may include media sub-audio data, navigation sub-audio data, and mixed audio sub-audio data.

[0266] In this implementation, a separate transmission channel is established for the call sub-audio data, which can ensure that the encoded audio corresponding to the call sub-audio data is not disturbed during the transmission process, thereby effectively improving the call quality.

[0267] Optionally, when sending the encoded audio through a transmission channel, a data header can be added for each encoded audio. The data header is used to identify the audio type of the encoded audio (that is, the audio type of the sub-audio data corresponding to the encoded audio). This is beneficial for the transmission channel of the vehicle audio device to receive the encoded audio. It can quickly determine the audio type of the encoded audio by identifying the data header of the encoded audio, thereby quickly sending the encoded audio to the corresponding decoder.

[0268] The audio processing method provided in the embodiment of the present application is applied to the application framework layer of an electronic device, which is connected to a vehicle-mounted audio device. The application framework layer of the electronic device receives audio data generated by multiple applications, processes the audio data into sub-audio data, and the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data. Each sub-audio data is encoded by creating a corresponding number of encoders for the sub-audio data to obtain encoded audio, and the encoded audio is sent to the vehicle-mounted audio device through a preset transmission channel.

[0269] Since the data contained in the sub-audio data are individual audio track data (or each data contained in the sub-audio data is different types of audio data), these individual audio track data are encoded and sent to the vehicle audio device. The vehicle audio device decodes the encoded audio through the decoder created for the encoded audio, and the decoded audio obtained is also individual audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimization scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0270] See also Figure 7 , Figure 7 This is a flow chart of another audio processing method according to an embodiment of the present application. The method includes:

[0271] S201, sending an audio diversion instruction and a cross-device audio streaming capability start-up instruction.

[0272] S202: Enable the audio splitting function and the cross-device audio streaming function.

[0273] S203: The user triggers an input operation.

[0274] S204: Receive audio data generated by multiple applications.

[0275] S205: Process the audio data into sub-audio data.

[0276] S206: Forward the sub-audio data through the audio router.

[0277] S207: Forward the sub-audio data through the virtual audio service.

[0278] S208: Encode the sub-audio data and send the encoded audio.

[0279] S209: Send the encoded audio through a transmission channel.

[0280] For the specific contents of steps S201 to S209, reference may be made to the method described in the aforementioned steps S101 to S109, which will not be repeated here.

[0281] S210: Receive and send encoded audio.

[0282] The application framework layer of the vehicle audio device may include a device virtualization service and an audio framework, wherein the device virtualization service may include a transmission channel, a decoder, and an audio playback module.

[0283] Exemplarily, the transmission channel of the electronic device sends each encoded audio to the transmission channel of the vehicle audio device. After receiving each encoded audio, the transmission channel of the vehicle audio device transmits each encoded audio to its corresponding decoder.

[0284] S211. Decode the encoded audio to obtain decoded audio.

[0285] In the embodiment of the present application, decoding processing refers to decompressing the encoded audio, so that the data volume of the sub-audio data can be restored, which is conducive to the subsequent smooth playback of the decoded audio.

[0286] Optionally, the decoding process may also be to convert the network signal into an audio signal.

[0287] Exemplarily, the device virtualization service of the vehicle audio device creates a corresponding number of decoders for the encoded audio, and decodes the encoded audio transmitted to the decoder through the created decoder to obtain decoded audio, and then sends the decoded audio to the audio playback module of the vehicle audio device.

[0288] S212: Send the decoded audio.

[0289] Exemplarily, the audio playback module temporarily stores each decoded audio sent by the decoder, and sends each temporarily stored decoded audio to the audio framework at intervals of a preset duration.

[0290] S213: Play the decoded audio according to the playback strategy.

[0291] Exemplarily, the audio framework obtains a playback strategy, and after receiving each decoded audio sent by the audio playback module, plays and / or adjusts the decoded audio according to the playback strategy. The playback strategy may include independently adjusting the playback volume corresponding to the decoded audio, and / or playing the decoded audio corresponding to the specified speaker through the specified speaker.

[0292] For example, the playback strategy specifically includes increasing the playback volume of the navigation audio and reducing the playback volume of the music audio at the fork in the road. For another example, the playback strategy specifically includes playing the navigation audio through the driver's seat speaker and the co-driver's seat speaker. For another example, the playback strategy specifically includes playing the music audio through all speakers in the vehicle, etc.

[0293] Optionally, in a possible implementation, the playback strategy can be determined according to the distribution status of people in the vehicle. For example, if it is detected that there is only one driver in the vehicle, the playback strategy may include playing navigation audio and media music through the driver's seat speaker. For another example, if it is detected that there are two people in the vehicle, the driver and the front passenger, the playback strategy may include playing navigation audio and media music through the driver's seat speaker and the front passenger seat speaker. For another example, if it is detected that there are people in the driver's seat, the front passenger seat and the back seat in the vehicle, the playback strategy may include playing navigation audio through the driver's seat speaker, playing music audio through all speakers in the vehicle, and so on. This is only an exemplary description and is not limited to this.

[0294] In the audio processing method provided in the embodiment of the present application, since the vehicle audio device receives not mixed audio data but individual encoded audio, after decoding the encoded audio, the decoded audio obtained is also individual audio (or in other words, each decoded audio obtained by decoding is an audio of a different audio type). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimized scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0295] See also Figure 8 , Figure 8 FIG. 1 is another system structure block diagram of an embodiment of the present application. Figure 8 As shown, the electronic device may include an audio framework, a virtual hardware abstraction layer, and a device virtualization service, and the in-vehicle audio device may include a device virtualization service and an audio framework.

[0296] In the embodiment of the present application, the application programs are music application and navigation application as examples for explanation. The user performs input operation on the music application and the navigation application, the music application generates music audio data, and the navigation application generates navigation audio data. After the audio framework identifies the audio type of the music audio data and the navigation audio data, the music audio data and the navigation audio data are processed into music sub-audio data and navigation sub-audio data respectively. The audio framework sends the music sub-audio data and the navigation sub-audio data to the virtual hardware abstraction layer, and the virtual hardware abstraction layer forwards the music sub-audio data and the navigation sub-audio data to the device virtualization service through the audio routes created for the music sub-audio data and the navigation sub-audio data respectively.

[0297] The device virtualization service encodes the music sub-audio data through the first encoder created for the music sub-audio data to obtain the first encoded audio, and sends the first encoded audio to the transmission channel. The device virtualization service encodes the navigation sub-audio data through the second encoder created for the navigation sub-audio data to obtain the second encoded audio, and sends the second encoded audio to the transmission channel.

[0298] In this implementation, encoding sub-audio data of different audio types through different encoders can improve encoding efficiency and effectively avoid encoding jams, so that the audio finally played on the in-vehicle audio device is smoother.

[0299] The first encoded audio and the second encoded audio are sent to the transmission channel of the vehicle audio device through the transmission channel in the device virtualization service. In this implementation, encoded audio of different audio types is transmitted through one transmission channel, which realizes the multiplexing of the transmission channel, reduces resource consumption while ensuring performance, and effectively saves resources of the electronic device.

[0300] The transmission channel of the vehicle audio device receives the first encoded audio and the second encoded audio sent by the electronic device, and sends the first encoded audio to the first decoder, and sends the second encoded audio to the second decoder. The first decoder is a decoder created by the device virtualization service of the vehicle audio device for the encoded audio of the media audio type, and the second decoder is a decoder created by the device virtualization service of the vehicle audio device for the encoded audio of the navigation audio type.

[0301] In this implementation, at the vehicle audio device end, encoded audio of different audio types is transmitted through one transmission channel, thereby realizing multiplexing of the transmission channel, reducing resource consumption while ensuring performance, and effectively saving resources of the vehicle audio device.

[0302] The first encoded audio is decoded and processed by the first decoder to obtain the first decoded audio, and the first decoded audio is sent to the audio frame of the vehicle audio device; the second encoded audio is decoded and processed by the second decoder to obtain the second decoded audio, and the second decoded audio is sent to the audio frame of the vehicle audio device.

[0303] In this implementation, different decoders are used to decode encoded audio of different audio types, which effectively avoids the occurrence of audio freezes, thereby improving the smoothness of the final played audio and further improving the user experience.

[0304] The audio framework of the vehicle audio device plays the first decoded audio and the second decoded audio according to the playback strategy. For example, the first decoded audio is played through the first player (such as the co-pilot seat speaker), and the second decoded audio is played through the second player (such as the driver's seat speaker).

[0305] Since the in-vehicle audio device receives not mixed audio data but individual encoded audio, after decoding the encoded audio, the decoded audio obtained is also individual audio (or in other words, each decoded audio obtained by decoding is an audio of a different audio type). Therefore, the in-vehicle audio device can independently play and adjust each audio, for example, it can adjust the volume of music, navigation sound and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimized scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0306] Compared with related technologies, this application is very different in the way of transmitting data and the audio routing design. It realizes the multiplexing of transmission channels, reduces resource consumption while ensuring performance, effectively saves resources of electronic equipment and vehicle audio equipment, and improves the interactive experience between electronic equipment and vehicles.

[0307] The following mainly describes the audio processing method provided by this application from the electronic device side. Fig. 9 , Fig. 9 The following is a flow chart of another audio processing method according to an embodiment of the present application. The method includes:

[0308] S301: Receive audio data generated by multiple applications.

[0309] S302: Process the audio data into sub-audio data.

[0310] S303: Encode the sub-audio data by using a corresponding number of encoders created for the sub-audio data to obtain encoded audio.

[0311] S304: Send the encoded audio to the vehicle audio device through a preset transmission channel.

[0312] For the specific contents of step S301 to step S304, please refer to the previous description and will not be repeated here.

[0313] The audio processing method provided in the embodiment of the present application is applied to the application framework layer of an electronic device, which is connected to a vehicle-mounted audio device. The application framework layer of the electronic device receives audio data generated by multiple applications, processes the audio data into sub-audio data, and the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data. Each sub-audio data is encoded by creating a corresponding number of encoders for the sub-audio data to obtain encoded audio, and the encoded audio is sent to the vehicle-mounted audio device through a preset transmission channel.

[0314] Since the data contained in the sub-audio data are individual audio track data (or each data contained in the sub-audio data is different types of audio data), these individual audio track data are encoded and sent to the vehicle audio device. The vehicle audio device decodes the encoded audio through the decoder created for the encoded audio, and the decoded audio obtained is also individual audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimization scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0315] The following mainly describes the audio processing method provided by this application based on the vehicle audio device side. Fig.10 , Fig.10 The following is a flow chart of another audio processing method according to an embodiment of the present application. The method includes:

[0316] S401: Receive encoded audio sent through a preset transmission channel.

[0317] S402: Decode the encoded audio using a decoder created for the encoded audio to obtain decoded audio.

[0318] S403: Play the decoded audio according to the playback strategy.

[0319] For the specific contents of step S401 to step S403, please refer to the previous description and will not be repeated here.

[0320] The audio processing method provided in the embodiment of the present application, since the data contained in the sub-audio data is a separate track data (or each data contained in the sub-audio data is a different type of audio data), these separate track data are encoded and sent to the vehicle audio device, and the vehicle audio device decodes the encoded audio through the decoder created for the encoded audio, and the decoded audio obtained is also a separate audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, it can adjust the volume of music, navigation sound, and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimization scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0321] Optionally, the audio processing method provided in the embodiment of the present application can also be applied to interactive scenarios where there is cross-device audio flow between electronic devices, such as interactive scenarios between a mobile phone and a tablet computer, a mobile phone and a smart screen, a mobile phone and a computer, a tablet computer and a smart screen, a tablet computer and a computer, etc.

[0322] Exemplarily, the interaction scenario between a mobile phone and a tablet computer is used as an example for explanation. For example, the mobile phone and the tablet computer establish a communication connection via Bluetooth, and audio data transmission and audio data processing are performed between the mobile phone and the tablet computer. For specific how to perform audio data transmission and audio data processing, reference can be made to the description in the above steps S201 to S213, which will not be repeated here.

[0323] It is worth noting that the tablet computer itself has a speaker, and can also be connected to a wired headset or a wireless headset as a speaker. The corresponding playback strategy of the tablet computer can be to independently adjust the volume of the decoded audio played by different speakers. For example, when playing call audio through a wireless headset, increase the call volume.

[0324] The playback strategy corresponding to the tablet computer may also be to play the decoded audio corresponding to the designated speaker through the designated speaker, for example, playing the call audio through a wireless headset, playing the music audio through the speaker of the tablet computer itself, and so on.

[0325] The audio processing method provided in this application can meet the personalized needs of audio playback of electronic devices and improve the user experience.

[0326] In the aforementioned audio processing method, a transmission channel is multiplexed, that is, the electronic device sends the encoded audio of different audio types to the vehicle audio device through a transmission channel. In this case, usually for each encoded audio acquired, an encoded audio is sent through the transmission channel, resulting in low bandwidth utilization. In order to improve bandwidth utilization, the present application also provides an audio transmission method that can effectively improve bandwidth utilization and increase the transmission speed of encoded audio.

[0327] The following describes the audio transmission method provided by the embodiment of the present application in combination with the software structure. Fig.11 , Fig.11 The software structure block diagram of another electronic device is shown as an exemplary embodiment of the present application.

[0328] Fig.11 The software structure diagram of the electronic equipment shown is Figure 4 Based on the software structure diagram of the electronic device shown, a package module is added. Fig.11 As shown, the device virtualization service of the electronic device may also include a package combination module. The following mainly describes the package combination module. For other software structures, please refer to the previous Figure 4 The description is not repeated here.

[0329] Exemplarily, each encoder in the device virtualization service encodes the sub-audio data to obtain multiple encoded audios; the multiple encoded audios are packaged together through the packaging module to obtain packaged data; the packaging module sends the packaged data to the transmission channel, and the transmission channel sends the packaged data to the vehicle audio device.

[0330] The packet processing refers to splicing / combining multiple coded audios, and the data obtained by splicing / combining is the packetized data. It can be generally understood that multiple data packets (each coded audio corresponds to a data packet) are spliced / combined into one data packet (the packetized data corresponds to one data packet).

[0331] If multiple encoded audios are not packaged together, each time an encoded audio is obtained, one encoded audio is sent through the transmission channel, which will result in low bandwidth utilization. For example, if the data volume of an encoded audio is 400 bytes, only 400 bytes of data can be transmitted at a time through the transmission channel, and the next encoded audio needs to wait for the previous encoded audio to be transmitted before it can be transmitted. The audio transmission method provided in the present application packages multiple encoded audios each time, and then sends the packaged data obtained by the package processing to the vehicle audio device at one time through the transmission channel, which increases the amount of data transmitted each time, improves bandwidth utilization, and improves the transmission speed of the encoded audio.

[0332] Optionally, in one possible implementation, all encoders (including call encoders) send the encoded audio obtained after their respective encoding processing to the packaging module; the packaging module performs packaging processing on all received encoded audio to obtain packaged data; the packaging module sends the packaged data to the transmission channel, and the transmission channel sends the packaged data to the vehicle audio device.

[0333] In this implementation, the packetization module performs packetization on the encoded audio of all audio types, which greatly compresses the data volume of the encoded audio, enables more data volume of the encoded audio to be transmitted through the transmission channel at one time, improves bandwidth utilization, and increases the transmission speed of the encoded audio.

[0334] Optionally, in a possible implementation, except for the call encoder, other encoders send the encoded audio obtained after their respective encoding processing to the package combining module; the package combining module performs package combining processing on the received encoded audio to obtain packaged data; the package combining module sends the packaged data to the transmission channel, and the transmission channel sends the packaged data to the vehicle audio device. The call encoder sends the encoded audio corresponding to the call audio data to the transmission channel separately, and the transmission channel sends the encoded audio corresponding to the call audio data to the vehicle audio device separately.

[0335] In this implementation, a separate transmission channel is established for the coded audio corresponding to the call audio data, which can ensure that the coded audio corresponding to the call audio data is not disturbed during the transmission process, thereby improving the call quality. At the same time, the packet combination module performs packet combination processing on the coded audio sent by other encoders except the call encoder, compressing the data volume of the coded audio, so that more coded audio data can be transmitted through the transmission channel at one time, improving bandwidth utilization and increasing the transmission speed of the coded audio.

[0336] The following is a detailed description of the packaging process with reference to the accompanying drawings. Fig.12 , Fig.12 The figure is a schematic diagram of package combining processing shown as an exemplary embodiment of the present application.

[0337] Exemplarily, the package module creates an input buffer (inBuffer) for each encoder, that is, one encoder corresponds to one inBuffer, and inBuffer is used to receive and store the encoded audio sent by its corresponding encoder. By creating corresponding input buffers for different encoders, one buffer only needs to store the encoded audio of one audio type, which can ensure that the reading, storage and other operations of the encoded audio between different audio types are not interfered.

[0338] For example, the packetization module creates a first input buffer for the first encoder and creates a second input buffer for the second encoder.

[0339] In one example, the packaging module creates an input buffer, which may be a corresponding input buffer created in advance for encoders of different audio types and stored in the packaging module for a long time, so as to facilitate the rapid reception and storage of encoded audio through the created input buffer later.

[0340] In another example, a cycle starts with the establishment of a connection between the electronic device and the vehicle audio device and ends with the disconnection of the electronic device and the vehicle audio device. In each cycle, the package combining module creates a corresponding input buffer for each encoder created by the device virtualization service. For example, when the package combining module first receives encoded audio of a certain audio type sent by a certain encoder, it creates an input buffer corresponding to the encoder. After that, in the cycle, the encoded audio sent by the encoder corresponding to the input buffer can be directly received and stored through the input buffer.

[0341] Optionally, when it is detected that the electronic device is disconnected from the vehicle audio device, the packet merging module destroys the created input buffer. In this implementation, the created input buffer is destroyed at the end of a cycle, which effectively avoids the input buffer from occupying system resources for a long time, improves resource utilization, and enhances the performance of the electronic device.

[0342] It is understandable that if the packet combining module has never received the encoded audio sent by an encoder of a certain audio type in a cycle, then the input buffer corresponding to the encoder may not be created in the cycle. This implementation method creates the corresponding input buffer when there is demand, which can effectively reduce resource consumption and improve resource utilization.

[0343] Optionally, the package combining module may also establish a thread for each encoder, where the thread is used to improve the fluency of the process corresponding to the thread and ensure that the process corresponding to the thread is not interfered by the processes corresponding to other threads.

[0344] For example, the package combining module establishes a first thread for the first encoder and a second thread for the second encoder. The process corresponding to the first thread is that the first encoder sends (or writes) the first encoded audio to the first input buffer, and the process corresponding to the second thread is that the second encoder sends (or writes) the second encoded audio to the second input buffer. With the existence of the first thread and the second thread, the process of the first encoder sending the first encoded audio to the first input buffer does not interfere with the process of the second encoder sending the second encoded audio to the second input buffer, thereby improving the fluency of sending the encoded audio.

[0345] Optionally, the packetization module may further include a packetizer, which is used to read the encoded audio from each input buffer, splice / combine the read encoded audio to obtain packetized data, and send the packetized data to the transmission channel.

[0346] It is worth noting that the packet transmitter works periodically. For example, at each sending cycle (i.e., the preset cycle), the packet transmitter reads the encoded audio from each input buffer and reads out all the encoded audio in the current input buffer. No matter how much encoded audio is read in this sending cycle, the read encoded audio is spliced / combined to obtain the combined data, and the combined data is sent to the transmission channel. After that, continue to wait for the arrival of the next sending cycle, and then repeat the process of reading the encoded audio, splicing / combining the encoded audio, and sending the combined data until the electronic device is disconnected from the vehicle audio device.

[0347] Among them, the sending period can be set and adjusted according to the encoding period of the encoder. For example, the encoding period can be 10 milliseconds, 20 milliseconds, 30 milliseconds, 40 milliseconds, etc. Correspondingly, the sending period can be 10 milliseconds, 20 milliseconds, 30 milliseconds, 40 milliseconds, etc.

[0348] It is worth noting that if the encoding cycles of the encoders are different, in order to ensure that the audio is not stuck, the sending cycle is set according to the minimum encoding cycle. For example, if the encoding cycle of the first encoder is 10 milliseconds and the encoding cycle of the second encoder is 20 milliseconds, the sending cycle is 10 milliseconds.

[0349] It is understandable that, since the packet sender reads the encoded audio from each input buffer in each sending cycle, the storage duration of the encoded audio temporarily stored in the input buffer is the same as the sending cycle. For example, when the sending cycle is 20 milliseconds, the temporary storage duration of the encoded audio in the input buffer is also 20 milliseconds. This is only an exemplary description and is not limited to this.

[0350] Optionally, the package combining module can also establish a thread for the packet sender, which is used to improve the smoothness of the process to be executed by the packet sender (such as the process of reading encoded audio, splicing / combining encoded audio, and sending packaged data), and ensure that the process to be executed by the packet sender is not interfered by the process corresponding to other threads. For example, the package combining module establishes a third thread for the packet sender.

[0351] Optionally, in a possible implementation, the packet sender splices / combines the read encoded audio to obtain the combined data. If the combined data is successfully obtained, the combined data is sent to the transmission channel. After that, the packet sender waits for the sending interval, or in other words, the packet sender continues to wait for the arrival of the next sending cycle.

[0352] If the combined data is not obtained, the step of obtaining the combined data is returned. It should be understood that the packet sender may not read the encoded audio in a certain sending cycle due to encoder jitter, network freeze, etc., resulting in failure to obtain the combined data.

[0353] In this implementation, the encoded audio within a sending cycle can be spliced / combined through the packetizer and sent to the vehicle audio device through the transmission channel, thereby sending a large amount of encoded audio in one sending cycle, improving bandwidth utilization and increasing the transmission speed of the encoded audio.

[0354] Please refer to Fig.13 , Fig.13 FIG. 1 is a schematic diagram of a sending cycle shown in an exemplary embodiment of the present application. Fig.13 As shown, one transmission cycle is defined between two transmission points. For example, one transmission cycle is defined between the first transmission point and the second transmission point, which is recorded as the first transmission cycle; one transmission cycle is defined between the second transmission point and the third transmission point, which is recorded as the second transmission cycle; one transmission cycle is defined between the third transmission point and the fourth transmission point, which is recorded as the third transmission cycle; and one transmission cycle is defined between the fourth transmission point and the fifth transmission point, which is recorded as the fourth transmission cycle.

[0355] Exemplarily, three encoded audios are received in the first sending cycle, which are respectively encoded audios of the media audio type in the first row, encoded audios of the navigation audio type in the second row, and encoded audios of the mixed audio type in the third row. These three encoded audios are packaged together to obtain packaged data, and the packaged data is sent to the transmission channel at the second sending point. One encoded audio is received in the second sending cycle, and the encoded audio is packaged together to obtain packaged data, and the packaged data is sent to the transmission channel at the third sending point. Five encoded audios are received in the third sending cycle, and the five encoded audios are packaged together to obtain packaged data, and the packaged data is sent to the transmission channel at the fourth sending point.

[0356] It should be understood that the encoded audio with a cross symbol received in the third sending cycle should have been received in the second sending cycle, but may have been received in the third sending cycle due to encoder jitter, network freeze, etc.

[0357] It is worth noting that there is no limit on the data volume of the combined data. The more encoded audio is received in a sending cycle, the larger the generated combined data is; correspondingly, the fewer encoded audio is received in a sending cycle, the smaller the generated combined data is. For example, the data volume corresponding to a coded audio is 400 bytes, and three coded audios are received in a sending cycle, and the data volume of the generated combined data is 1200 bytes. For another example, the data volume corresponding to a coded audio is 400 bytes, and five coded audios are received in a sending cycle, and the data volume of the generated combined data is 2000 bytes.

[0358] Please refer to Fig.14 , Fig.14 This is a schematic diagram of combined package data shown as an exemplary embodiment of the present application.

[0359] In an exemplary embodiment, multiple encoded audios are packaged together to obtain packaged data, which may include: setting a transmission protocol header, splicing / combining each encoded audio (each encoded audio is a data packet), and at the same time, setting a data header for each encoded audio during the process of splicing / combining the encoded audios.

[0360] The transport protocol header is used to indicate that the combined data is data obtained through combined processing, or that the combined data is encapsulated data. After the subsequent in-vehicle audio device receives the combined data, it can quickly determine that the combined data is data obtained through combined processing by identifying the transport protocol header in the combined data, and thus perform sub-packaging on the combined data.

[0361] The data header may include attribute information of the encoded audio. For example, the data header may store attribute information such as the identifier of the audio route corresponding to the encoded audio, the audio type corresponding to the encoded audio, the application corresponding to the encoded audio, and the length of the data packet corresponding to the encoded audio. Optionally, a reserved field may also be stored in the data header, so that the reserved field may be used to add identifiers, notes, and other information to the encoded audio in the future, which is beneficial to improving the compatibility of electronic devices.

[0362] It is worth noting that the order of splicing / combining each encoded audio can be determined according to the order in which each encoded audio is read. For example, the first encoded audio is read from the first input buffer, and the second encoded audio is read from the second input buffer in sequence. Set the transmission protocol header, set the first data header for the first encoded audio, splice / combine the first data header and the first data packet (the data packet corresponding to the first encoded audio); set the second data header for the second encoded audio, splice / combine the second data header and the first data packet (the data packet corresponding to the second encoded audio), and obtain the combined data. Fig.14 As shown, the combined data includes a transmission protocol header, a first data header, a first data packet, a second data header, and a second data packet.

[0363] This implementation method adds a data header to the encoded audio during the process of splicing / combining the encoded audio, so that the subsequent on-board audio equipment can quickly sub-packetize the combined data according to the data header and quickly obtain the attribute information of each encoded audio.

[0364] In another exemplary embodiment, multiple encoded audios are packaged together to obtain packaged data, which may include: setting a transport protocol header, and sequentially splicing / combining each encoded audio (each encoded audio corresponds to a data packet).

[0365] This implementation method directly splices / combines the encoded audio without adding a data header to the encoded audio, which improves the speed of packet processing and further improves the speed of generating packet data.

[0366] The above describes the audio transmission method provided by the embodiment of the present application in combination with the software structure. The following describes the audio transmission method provided by the embodiment of the present application in combination with the system architecture. Fig.15 , Fig.15 FIG. 1 is another system architecture block diagram showing an exemplary embodiment of the present application. Fig.15 As shown in the system architecture diagram, Figure 5 Based on the corresponding system architecture diagram, the package assembly module and the sub-package module are added. The following mainly describes the sub-package module of the car audio device. For other software structures, please refer to the previous Figure 5 as well as Fig.11 The description is not repeated here.

[0367] Exemplarily, the transmission channel of the electronic device sends the combined data to the transmission channel of the vehicle audio device. The transmission channel of the electronic device may include a sending unit, through which the combined data is sent to the transmission channel of the vehicle audio device.

[0368] In this implementation, since multiple encoded audios are packaged together, the packaged data is ultimately sent to the transmission channel of the vehicle audio device, rather than sending multiple encoded audios directly to the transmission channel of the vehicle audio device. Therefore, the number of sending units is greatly reduced (for example, originally three sending units are required to send three encoded audios, but here only one sending unit is required), which effectively improves the efficiency of sending packaged data and improves the bandwidth utilization between the electronic device and the vehicle audio device.

[0369] Exemplarily, after the transmission channel of the vehicle audio device receives the combined data, the combined data is divided into packages by the sub-packaging module in the device virtualization service to obtain multiple encoded audios; each decoder in the device virtualization service decodes each encoded audio to obtain multiple decoded audios. Each decoder sends the decoded audio obtained after the decoding process to the audio playback module, and the audio playback module temporarily stores each decoded audio, and sends each temporarily stored decoded audio to the audio framework at intervals of a preset duration.

[0370] The audio framework obtains the playback strategy, and after receiving each decoded audio sent by the audio playback module, plays and / or adjusts the decoded audio according to the playback strategy. The playback strategy may include independently adjusting the playback volume corresponding to the decoded audio, and / or playing the decoded audio corresponding to the specified speaker through the specified speaker.

[0371] The packet processing refers to splitting the packetized data into one or more encoded audios.

[0372] For car audio equipment, when receiving packaged data through the transmission channel, it can receive a large amount of data at one time, which increases the data transmission rate and improves bandwidth utilization.

[0373] Optionally, in a possible implementation, if at the electronic device end, all encoders (including the call encoder) send the encoded audio obtained after their respective encoding processing to the package combining module; the package combining module performs package combining processing on all the received encoded audio to obtain package combining data; the package combining module sends the package combining data to the transmission channel, and the transmission channel sends the package combining data to the vehicle audio device. Accordingly, the transmission channel of the vehicle audio device receives the package combining data in this scenario.

[0374] In this implementation, since the packet combining module on the electronic device side performs packet combining on all types of encoded audio, the data volume of the encoded audio is greatly compressed, so that more data volume of encoded audio can be transmitted at one time through the transmission channel of the electronic device, thereby enabling the transmission channel of the vehicle audio device to receive more data volume of encoded audio at one time, thereby improving the bandwidth utilization between the electronic device and the vehicle audio device and increasing the transmission speed of the encoded audio.

[0375] Optionally, in a possible implementation, if on the electronic device side, except for the call encoder, other encoders send the encoded audio obtained after their respective encoding processing to the package combining module; the package combining module performs package combining processing on the received encoded audio to obtain packaged data; the package combining module sends the packaged data to the transmission channel, and the transmission channel sends the packaged data to the vehicle audio device. Accordingly, the transmission channel of the vehicle audio device receives the packaged data in this scenario. The call encoder on the electronic device side sends the encoded audio corresponding to the call audio data to the transmission channel separately, and the transmission channel sends the encoded audio corresponding to the call audio data to the vehicle audio device separately. Accordingly, the transmission channel of the vehicle audio device receives the encoded audio corresponding to the call audio data separately.

[0376] In this implementation, a separate transmission channel is established for the encoded audio corresponding to the call audio data, which can ensure that the encoded audio corresponding to the call audio data is not disturbed during the transmission process, thereby improving the call quality. At the same time, because the packet combination module on the electronic device side performs packet combination processing on the encoded audio sent by other encoders except the call encoder, the data volume of the encoded audio is compressed, so that more data volume of encoded audio can be transmitted at one time through the transmission channel of the electronic device, so that the transmission channel of the vehicle audio device can receive more data volume of encoded audio at one time, which improves the bandwidth utilization between the electronic device and the vehicle audio device and improves the transmission speed of the encoded audio.

[0377] The following is a detailed description of the subcontracting process with the help of the attached figure. Fig.16 , Fig.16 The figure is a schematic diagram of subpackaging processing shown as an exemplary embodiment of the present application.

[0378] Exemplarily, the packetization module creates an input buffer for each decoder, that is, one decoder corresponds to one inBuffer, and the inBuffer is used to receive and store the encoded audio separated from the combined packet data.

[0379] For example, the packetization module creates a first input buffer for the first decoder and a second input buffer for the second decoder.

[0380] In one example, the packetization module creates an input buffer, which may be a corresponding input buffer created in advance for decoders of different audio types and stored in the packetization module for a long time, so as to facilitate the rapid reception and storage of encoded audio through the created input buffer later.

[0381] In another example, a cycle starts with the establishment of a connection between the electronic device and the vehicle audio device and ends with the disconnection of the electronic device and the vehicle audio device. In each cycle, the packetization module creates a corresponding input buffer for each decoder created by the device virtualization service. For example, when the packetization module first splits out the encoded audio of a certain audio type, it creates an input buffer for the decoder corresponding to the audio type. After that, in the cycle, the encoded audio corresponding to the input buffer can be directly received and stored through the input buffer.

[0382] Optionally, when it is detected that the electronic device is disconnected from the vehicle audio device, the subpacketization module destroys the created input buffer. In this implementation, the created input buffer is destroyed at the end of a cycle, which effectively avoids the input buffer from occupying system resources for a long time, improves resource utilization, and enhances the performance of the electronic device.

[0383] It is understandable that if the packetization module has never split out the encoded audio corresponding to a certain audio type in a cycle, then the input buffer of the decoder corresponding to the audio type may not be created in the cycle. This implementation method creates the corresponding input buffer when there is demand, which can effectively reduce resource consumption and improve resource utilization.

[0384] Optionally, the packetization module may also establish a thread for each decoder, where the thread is used to improve the fluency of the process corresponding to the thread and ensure that the process corresponding to the thread is not interfered by the processes corresponding to other threads.

[0385] For example, the subpacketization module establishes a first thread for the first decoder and a second thread for the second decoder. The process corresponding to the first thread is that the first decoder reads the first encoded audio from the first input buffer, and the process corresponding to the second thread is that the second decoder reads the second encoded audio from the second input buffer. With the existence of the first thread and the second thread, the process of the first decoder reading the first encoded audio from the first input buffer does not interfere with the process of the second decoder reading the second encoded audio from the second input buffer, thereby improving the fluency of reading the encoded audio.

[0386] Optionally, the packet combining module may also establish a thread, such as a third thread, for splitting the combined packet data. The process of the third thread for splitting the combined packet data is not interfered by the processes corresponding to other threads.

[0387] Exemplarily, after each decoder reads the encoded audio, it decodes the encoded audio to obtain a decoded audio. Each decoder sends the decoded audio obtained after the decoding process to the audio playback module, and the audio playback module temporarily stores each decoded audio, and sends each temporarily stored decoded audio to the audio framework at intervals of a preset duration.

[0388] Optionally, the audio playback module may further include multiple playback interfaces, through which the audio framework may be accessed. Exemplarily, each decoder sends the decoded audio obtained after decoding processing to the audio framework through the playback interface. For example, the first decoder sends the decoded audio obtained after decoding processing to the audio framework through the first playback interface; the second decoder sends the decoded audio obtained after decoding processing to the audio framework through the second playback interface.

[0389] The audio framework obtains the playback strategy, and after receiving each decoded audio sent by the audio playback module, plays and / or adjusts the decoded audio according to the playback strategy. This allows the in-vehicle audio device to independently play and adjust each audio, and can achieve many optimization scenarios between electronic devices and vehicles to improve user experience, meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and passengers.

[0390] In an exemplary embodiment, packet processing is performed on the combined data to obtain encoded audio, which may include: identifying the transmission protocol header, parsing the data header corresponding to each encoded audio in the combined data, and splitting each encoded audio in turn according to the data packet length of the encoded audio in each data header (each encoded audio is a data packet).

[0391] Exemplarily, after receiving the combined data, the in-vehicle audio device identifies the transport protocol header in the combined data, quickly determines that the combined data is data obtained through combined processing, and then parses the combined data.

[0392] For each encoded audio, by parsing the attribute information in the data header, the identifier of the audio route corresponding to the encoded audio, the audio type corresponding to the encoded audio, the application corresponding to the encoded audio, the data packet length corresponding to the encoded audio, etc. are determined.

[0393] Then, the encoded audio is accurately extracted from the combined data by the data packet length corresponding to the encoded audio. For example, the end position of the data header is obtained, the start position of the encoded audio is determined according to the end position of the data header, and the encoded audio is extracted from the combined data according to the start position of the encoded audio and the data packet length.

[0394] In this implementation, by parsing the data header in the combined data package, each encoded audio can be accurately and quickly parsed, which is conducive to the subsequent on-board audio equipment to quickly decode and process the encoded audio, so as to independently play, adjust and perform other operations on each audio.

[0395] The audio transmission method provided in the embodiment of the present application is described above in combination with the system architecture. The audio transmission method provided in the embodiment of the present application is described below in combination with the flow chart.

[0396] See also Fig.17 , Fig.17 The following is a flow chart of an audio transmission method according to an embodiment of the present application. The method includes:

[0397] S501, sending an audio diversion instruction and a cross-device audio streaming capability start-up instruction.

[0398] S502: Enable the audio splitting function and the cross-device audio streaming function.

[0399] S503: The user triggers an input operation.

[0400] S504: Receive audio data generated by multiple applications.

[0401] S505: Process the audio data into sub-audio data.

[0402] S506: Forward the sub-audio data through the audio router.

[0403] S507: Forward the sub-audio data through the virtual audio service.

[0404] S508: Encode the sub-audio data and send the encoded audio.

[0405] S509: Packetize multiple encoded audios to obtain packaged data.

[0406] S510: Send the combined data through a transmission channel.

[0407] For the specific contents of step S501 to step S510, please refer to the previous description and will not be repeated here.

[0408] The audio transmission method provided in the embodiment of the present application, on the one hand, since the data contained in the sub-audio data processed by the electronic device is a separate track data (or each data contained in the sub-audio data is a different type of audio data), these separate track data are encoded and sent to the vehicle audio device, and the vehicle audio device decodes the encoded audio through the decoder created for the encoded audio, and the decoded audio obtained is also a separate audio (or each decoded audio obtained is a different type of audio). Therefore, the vehicle audio device can independently play, adjust, and perform other operations on each audio, for example, the volume of music, navigation sound, and call sound can be adjusted respectively, and different speakers can be specified to play different types of audio, etc. This can realize many optimization scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0409] On the other hand, electronic devices perform packet processing on the encoded audio, which greatly compresses the data volume of the encoded audio, so that more data volume of the encoded audio can be transmitted through the transmission channel at one time, improving bandwidth utilization and increasing the transmission speed of the encoded audio.

[0410] See also Fig.18 , Fig.18 This is a flow chart of another audio transmission method according to an embodiment of the present application. The method includes:

[0411] S601, sending an audio diversion instruction and a cross-device audio streaming capability start-up instruction.

[0412] S602: Enable the audio splitting function and the cross-device audio streaming function.

[0413] S603: The user triggers an input operation.

[0414] S604: Receive audio data generated by multiple applications.

[0415] S605: Process the audio data into sub-audio data.

[0416] S606: Forward the sub-audio data through the audio router.

[0417] S607: Forward the sub-audio data through the virtual audio service.

[0418] S608: Encode the sub-audio data and send the encoded audio.

[0419] S609: perform packetization processing on the encoded audio to obtain packetized data.

[0420] S610: Send the combined data through a transmission channel.

[0421] S611, receiving and sending packaged data.

[0422] S612: Sub-packetize the combined data to obtain multiple encoded audios.

[0423] S613: Decode the multiple encoded audios to obtain multiple decoded audios.

[0424] S614: Send multiple decoded audios.

[0425] S615: Play multiple decoded audios according to the playback strategy.

[0426] For the specific contents of step S601 to step S615, please refer to the previous description and will not be repeated here.

[0427] In this implementation, on the one hand, since multiple encoded audios are packaged together, the packaged data is ultimately sent to the transmission channel of the vehicle audio device, rather than sending multiple encoded audios directly to the transmission channel of the vehicle audio device. Therefore, the number of sending units is greatly reduced (for example, three sending units are required to send three encoded audios, but only one sending unit is required here), which effectively improves the efficiency of sending packaged data and improves the bandwidth utilization between the electronic device and the vehicle audio device.

[0428] On the other hand, since the vehicle audio device receives not mixed audio data but individual encoded audio, after decoding the encoded audio, the decoded audio obtained is also individual audio (or in other words, each decoded audio obtained by decoding is an audio of a different audio type). Therefore, the vehicle audio device can independently play and adjust each audio, for example, it can adjust the volume of music, navigation sound and call sound respectively, and can specify different speakers to play different types of audio. Therefore, the audio processing method provided by the present application can realize many optimized scenarios between electronic devices and vehicles for improving user experience, can meet the personalized needs of vehicle audio playback, and improve the driving experience of vehicle drivers and vehicle passengers.

[0429] The following mainly describes the audio transmission method provided by this application from the electronic device side. Fig.19 , Fig.19 The following is a flow chart of another audio transmission method according to an embodiment of the present application. The method includes:

[0430] S701. Obtain multiple encoded audios.

[0431] S702: Packetize multiple encoded audios to obtain packaged data.

[0432] S703: Send the combined data to the vehicle audio device through a preset transmission channel.

[0433] For the specific contents of step S701 to step S703, please refer to the previous description and will not be repeated here.

[0434] In this implementation, since the electronic device performs packet processing on the encoded audio, the data volume of the encoded audio is greatly compressed, so that more encoded audio data can be transmitted through the transmission channel at one time, improving bandwidth utilization and increasing the transmission speed of the encoded audio.

[0435] The following mainly describes the audio transmission method provided by this application based on the vehicle audio device side. Fig. 20 , Fig. 20The following is a flow chart of another audio transmission method according to an embodiment of the present application. The method includes:

[0436] S801: Receive packaged data sent through a preset transmission channel.

[0437] S802: Sub-packetize the combined data to obtain multiple encoded audios.

[0438] For the specific contents of step S801 to step S802, please refer to the previous description and will not be repeated here.

[0439] In this implementation, since multiple encoded audios are packaged together, the packaged data is ultimately sent to the transmission channel of the vehicle audio device, rather than sending multiple encoded audios directly to the transmission channel of the vehicle audio device. Therefore, the number of sending units is greatly reduced (for example, originally three sending units are required to send three encoded audios, but here only one sending unit is required), which effectively improves the efficiency of sending packaged data and improves the bandwidth utilization between the electronic device and the vehicle audio device.

[0440] Optionally, the audio transmission method provided in the embodiment of the present application can also be applied to interactive scenarios where there is cross-device audio flow between electronic devices, such as interactive scenarios between a mobile phone and a tablet computer, a mobile phone and a smart screen, a mobile phone and a computer, a tablet computer and a smart screen, a tablet computer and a computer, etc.

[0441] Exemplarily, the interaction scenario between a mobile phone and a tablet computer is used as an example for explanation. For example, the mobile phone and the tablet computer establish a communication connection via Bluetooth, and audio data transmission and audio data processing are performed between the mobile phone and the tablet computer. For specific how to perform audio data transmission and audio data processing, reference can be made to the description in the above steps S601 to S614, which will not be repeated here.

[0442] The audio transmission method provided in the present application performs packet processing on the transmitted encoded audio, which greatly compresses the data volume of the encoded audio, so that more data volume of encoded audio can be transmitted through the transmission channel at one time, thereby improving bandwidth utilization, increasing the transmission speed of the encoded audio, and improving user experience.

[0443] The hardware structure of the electronic device involved in the embodiments of the present application is briefly introduced below with reference to the accompanying drawings.

[0444] See also Fig.21 , Fig.21 The figure is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment of the present application.

[0445] like Fig.21As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0446] It is to be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Fig.21 More or fewer components than those shown, or the electronic device 100 may include Fig.21 A combination of some of the components shown, or the electronic device 100 may include Fig.21 Subassemblies of some of the components shown. Fig.21 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0447] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc.

[0448] Among them, different processing units can be independent devices or integrated into one or more processors. Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0449] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0450] In the embodiment of the present application, the processor 110 can execute each step in the audio processing method and the audio transmission method. For example, the processor 110 can run the software code of the audio processing method and the audio transmission method provided in the embodiment of the present application.

[0451] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI) interface, a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0452] It is understandable that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0453] The wireless communication function of the electronic device 100 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0454] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0455] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and the vehicle audio device through wireless communication technology.

[0456] The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU can also be used to perform mathematical and posture calculations for graphics rendering, etc. The processor 110 may include one or more GPUs, which can generate or change display information by executing program instructions.

[0457] In the embodiment of the present application, the display screen 194 can be used to display the interfaces of various application programs.

[0458] The display screen 194 in the embodiment of the present application may be a touch screen. A touch sensor 180K may be integrated in the display screen 194. The touch sensor 180K may also be referred to as a "touch panel". In other words, the display screen 194 may include a display panel and a touch panel, and the touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. After the touch operation detected by the touch sensor 180K, it can be passed to the upper layer by the driver of the kernel layer (such as the TP driver) to determine the type of touch event. Visual output related to the touch operation can be provided by the display screen 194. In other embodiments, the touch sensor 180K may also be arranged on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0459] The audio processing method and the audio transmission method provided in the embodiments of the present application can be implemented in the electronic device 100 having the above-mentioned hardware structure.

[0460] The above describes in detail the examples of the audio processing method and the audio transmission method provided by the embodiments of the present application. It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in conjunction with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0461] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each function can be divided into each functional module, or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0462] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0463] The electronic device provided in this embodiment is used to execute the above-mentioned audio processing method and audio transmission method, and thus can achieve the same effect as the above-mentioned implementation method.

[0464] In the case of an integrated unit, the electronic device may further include a processing module, a storage module and a communication module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device to execute stored program codes and data, etc. The communication module may be used to support the electronic device to communicate with other devices.

[0465] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a WiFi chip, etc.

[0466] An embodiment of the present application also provides a vehicle-mounted audio device, which includes one or more processors; one or more memories; a module with multiple application programs installed; and the memory stores one or more programs. When one or more programs are executed by the processor, the vehicle-mounted audio device executes the audio processing method and audio transmission method in the above-mentioned embodiment.

[0467] An embodiment of the present application also provides a vehicle, which includes: one or more processors; one or more memories; the memories store one or more programs, and when one or more programs are executed by the processors, the vehicle executes the audio processing method and audio transmission method in the above embodiments.

[0468] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the audio processing method and the audio transmission method of any of the above embodiments.

[0469] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the above-mentioned related steps to implement the audio processing method and the audio transmission method in the above-mentioned embodiment.

[0470] The present application also provides a chip. Fig. 22 , Fig. 22 A schematic diagram of the structure of a chip provided in an embodiment of the present application. Fig. 22 The chip shown can be a general-purpose processor or a dedicated processor. The chip includes a processor 310. The processor 310 is used to execute the audio processing method and the audio transmission method of any of the above embodiments.

[0471] Optionally, the chip further includes a transceiver 320, which is used to accept control of the processor and to support the communication device in executing the technical solution shown above.

[0472] Optional, Fig. 22 The chip shown may also include: a storage medium 330 .

[0473] It should be noted that Fig. 22 The chip shown can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.

[0474] Among them, the electronic device, vehicle audio device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0475] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0476] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0477] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0478] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0479] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0480] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An audio transmission method, It is characterized in that An application framework layer applied to an electronic device, the electronic device being connected to a vehicle audio device, the method comprising: Acquire multiple encoded audios, each of which corresponds to a data packet; Performing packet processing on the multiple encoded audios to obtain packetized data; the packetized data corresponds to a data packet; The combined data is sent to the vehicle audio device through a preset transmission channel; the combined data is used to trigger the vehicle audio device to perform sub-packetization processing on the combined data to obtain multiple encoded audios.

2. The method according to claim 1, It is characterized in that Before obtaining the plurality of encoded audios, the method further includes: Receive audio data generated by multiple applications; Processing the audio data into sub-audio data, wherein the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data; the mixed sub-audio data is obtained by mixing other sub-audio data except the media sub-audio data, the navigation sub-audio data, and the call sub-audio data; The sub-audio data is encoded by a corresponding number of encoders created for the sub-audio data to obtain the multiple encoded audios.

3. The method according to claim 2, It is characterized in that The method further comprises: Create a corresponding input buffer for each encoder, where the input buffer for each encoder is used to store the encoded audio corresponding to each encoder; The obtaining of the plurality of encoded audios includes: reading the encoded audios from each input buffer within each preset period to obtain the plurality of encoded audios, wherein the preset period is the same as the storage duration of each encoded audio in each input buffer.

4. The method according to any one of claims 1 to 3, It is characterized in that The step of performing packet processing on the plurality of encoded audios to obtain packetized data includes: Set the data header for each encoded audio; The data header corresponding to each encoded audio and the data packet corresponding to each encoded audio are concatenated to obtain the combined data.

5. The method according to claim 4, It is characterized in that The data header corresponding to each encoded audio includes the audio type corresponding to each encoded audio, the application corresponding to each encoded audio, and the data packet length of each encoded audio. The audio type includes any one of a media audio type, a navigation audio type, a call audio type, and a mixed audio type.

6. The method according to any one of claims 3 to 5, It is characterized in that The step of sending the combined data to the vehicle audio device through a preset transmission channel includes: Within the preset period, after detecting that the combined data is successfully acquired, the combined data is sent to the vehicle audio device through a preset transmission channel.

7. The method according to any one of claims 4 to 6, It is characterized in that The multiple encoded audios in the vehicle audio device are used to trigger the vehicle audio device to decode the multiple encoded audios through a decoder created for the multiple encoded audios to obtain multiple decoded audios; the multiple decoded audios are used by the vehicle audio device to play the multiple decoded audios according to a playback strategy; the playback strategy includes independently adjusting the playback volume corresponding to each decoded audio, and / or playing the decoded audio corresponding to the designated speaker through the designated speaker.

8. The method according to any one of claims 2 to 7, It is characterized in that The method further comprises: An instruction to disconnect the electronic device from the in-vehicle audio device is received, and each input buffer is destroyed.

9. An audio transmission method, It is characterized in that An application framework layer applied to a vehicle audio device, wherein the vehicle audio device is connected to an electronic device, and the method comprises: Receiving packaged data sent through a preset transmission channel; the packaged data corresponds to a data packet; the packaged data is obtained by package-processing a plurality of encoded audios by the application framework layer of the electronic device; The combined data is packetized to obtain a plurality of encoded audios; each of the encoded audios corresponds to a data packet.

10. The method according to claim 9, It is characterized in that The combined data includes a data header corresponding to each encoded audio and a data packet corresponding to each encoded audio, and the combined data is processed into packets to obtain multiple encoded audios, including: Parsing the data header corresponding to each encoded audio in the combined data; According to the data packet length of the encoded audio in each data header, parse out the data packet corresponding to each encoded audio.

11. The method according to claim 9, It is characterized in that The method further comprises: By using a decoder created for each encoded audio, each encoded audio is decoded to obtain multiple decoded audios; The multiple decoded audios are played according to a playback strategy; the playback strategy includes independently adjusting the playback volume corresponding to each decoded audio, and / or playing the decoded audio corresponding to the designated speaker through a designated speaker.

12. An electronic device, It is characterized in that include: one or more processors; one or more memories; The memory stores one or more programs, and when the one or more programs are executed by the processor, the electronic device executes the method according to any one of claims 1 to 8.

13. A car audio device, It is characterized in that include: one or more processors; one or more memories; The memory stores one or more programs, and when the one or more programs are executed by the processor, the in-vehicle audio device executes the method according to any one of claims 9 to 11.

14. A vehicle, It is characterized in that include: one or more processors; one or more memories; The memory stores one or more programs, and when the one or more programs are executed by the processor, the vehicle executes the method according to any one of claims 9 to 11.

15. A chip, It is characterized in that include: A processor, used to call and run a computer program from a memory, so that an electronic device equipped with the chip executes a method as described in any one of claims 1 to 8, or an in-vehicle audio device equipped with the chip executes a method as described in any one of claims 9 to 11.

16. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the method according to any one of claims 1 to 8, or the processor executes the method according to any one of claims 9 to 11.

Images