Audio processing method, vehicle-mounted audio device, electronic device, and vehicle

Abstract

The application provides an audio processing method, a vehicle-mounted audio device, an electronic device and a vehicle, and relates to the technical field of audio processing. The method comprises the following steps: receiving audio data generated by a plurality of application programs; processing the audio data into sub-audio data; performing encoding processing on the sub-audio data through a corresponding number of encoders created for the sub-audio data to obtain encoded audio; sending the encoded audio to the vehicle-mounted audio device through a preset transmission channel; the encoded audio is used to trigger the vehicle-mounted audio device to perform decoding processing on the encoded audio through a decoder created for the encoded audio to obtain decoded audio; and the decoded audio is used for the vehicle-mounted audio device to play the decoded audio according to a playing strategy. The audio processing method can realize many optimization scenarios between the electronic device and the vehicle for improving user experience, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of users.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio processing, and in particular to an audio processing method, a vehicle-mounted audio device, an electronic device, and a vehicle. BACKGROUND

[0002] With the rapid development of electronic technology, the interaction scenarios between electronic devices (such as mobile phones) and vehicle information systems (referred to as car machines) are becoming more and more rich. At present, a variety of application programs support service access, audio and video transmission and playing, and other operations between mobile phones and car machines.

[0003] Under normal circumstances, in the scenario of audio transmission and playing, the mobile phone and the car machine are in a connected state, and the mobile phone sends the mixed audio of different types to the car machine after mixing. The car machine can only perform unified playing, adjustment, and other operations on the mixed audio, which leads to the fact that many optimization scenarios for improving user experience between the mobile phone and the car machine cannot be implemented, and the driving and riding experience of the user is reduced. SUMMARY

[0004] The present application provides an audio processing method, a vehicle-mounted audio device, an electronic device, and a vehicle, which can implement many optimization scenarios for improving user experience between the electronic device and the vehicle, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the user.

[0005] In a first aspect, the present application provides an audio processing method applied to an application program framework layer of an electronic device, the electronic device being connected with a vehicle-mounted audio device, and the method comprising: receiving audio data generated by a plurality of application programs; processing the audio data into sub-audio data; encoding the sub-audio data through a corresponding number of encoders created for the sub-audio data to obtain encoded audio; sending the encoded audio to the vehicle-mounted audio device through a preset transmission channel; the encoded audio being used to trigger the vehicle-mounted audio device to decode the encoded audio through a decoder created for the encoded audio to obtain decoded audio; and the decoded audio being used for the vehicle-mounted audio device to play the decoded audio according to a playing strategy.

[0006] Optionally, the audio data can include at least one of media audio data, navigation audio data, call audio data, notification audio data, warning audio data, prompt audio data, and ringtone audio data.

[0007] Optionally, the sub-audio data can include at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data.

[0008] Among them, the mixed sub-audio data is obtained by mixing other sub-audio data except the media sub-audio data, the navigation sub-audio data, and the call sub-audio data.

[0009] Optionally, the other sub-audio data can include at least one of notification sub-audio data, warning sub-audio data, prompt sub-audio data, and ringtone sub-audio data.

[0010] Optionally, the playing strategy can include independently adjusting a playing volume corresponding to the decoded audio, and / or playing the decoded audio corresponding to the specified speaker through the specified speaker.

[0011] Optionally, the specified speaker can include a driver seat speaker, and an audio type of the decoded audio corresponding to the specified speaker is a navigation audio type.

[0012] The audio processing method provided by the embodiments of the present application can independently play, adjust, and the like each audio by the vehicle-mounted audio device, for example, can independently adjust a volume of each of music, navigation sound, and call sound, and can specify different speakers to play different types of audio. Thus, the audio processing method provided by the present application can realize many optimization scenarios between the electronic device and the vehicle for improving user experience, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the driver and the passenger of the vehicle.

[0013] In combination with the first aspect, in some implementations of the first aspect, the audio processing method provided by the embodiments of the present application further includes: obtaining an audio type of the sub-audio data; and creating an encoder matched with the audio type of the sub-audio data.

[0014] In the implementation, the audio type of the media sub-audio data is a media audio type, the audio type of the navigation sub-audio data is a navigation audio type, the audio type of the call sub-audio data is a call audio type, and the audio type of the mixed sub-audio data is a mixed audio type.

[0015] In the implementation, the encoder corresponding to each audio type of the sub-audio data is created, so that one type of encoder only needs to encode and process the sub-audio data of one type of audio, which can improve the encoding efficiency, effectively avoid the encoding lag phenomenon, and thus make the audio played on the vehicle-mounted audio device more smooth.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the electronic device further includes a virtual hardware abstraction layer. The audio processing method provided in this application embodiment further includes: an audio route created through the virtual hardware abstraction layer, which forwards the sub-audio data to an encoder that matches the sub-audio data.

[0017] Optionally, the created audio route matches the audio type of the sub-audio data.

[0018] In this implementation, corresponding audio routes are created for sub-audio data of different audio types, so that an audio route of one type only needs to forward sub-audio data of one audio type. This not only helps the subsequent device virtualization service to quickly distinguish the audio type of the sub-audio data it receives, but also improves forwarding efficiency.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, when the sub-audio data is call sub-audio data, the audio processing method provided in this application embodiment further includes: transmitting the call sub-audio data to an encoder that matches the call sub-audio data through a call channel in a virtual hardware abstraction layer.

[0020] In this implementation, a separate call channel and call encoder are established for the call audio data, which can ensure that the transmission and encoding process of the call audio data is not interfered with, and effectively improve the call quality.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the transmission channel may include a first transmission channel and a second transmission channel, wherein the first transmission channel is used to transmit encoded audio corresponding to non-call audio sub-data, and the second transmission channel is used to transmit encoded audio corresponding to call audio sub-data.

[0022] The non-call sub-audio data may include media sub-audio data, navigation sub-audio data, and mixing sub-audio data.

[0023] In this implementation, a separate transmission channel is established for the call sub-audio data, which ensures that the encoded audio corresponding to the call sub-audio data is not interfered with during transmission, effectively improving call quality.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, audio data is processed into sub-audio data, including: identifying at least two of media sub-audio data, navigation sub-audio data, call sub-audio data, and other sub-audio data from the audio data based on different audio type identifiers carried by the audio data; if other sub-audio data is identified, mixing processing is performed on the other sub-audio data to obtain mixed sub-audio data.

[0025] In the implementation manner, the sub-audio data of different audio types is identified through the audio type identification, different processing is made on the sub-audio data of different audio types, so as to obtain individual audio track data, thereby facilitating independent adjustment and playing of the audio corresponding to the sub-audio data of different audio types by the subsequent vehicle-mounted audio device.

[0026] With reference to the first aspect, in some implementations of the first aspect, the audio processing method provided by the embodiments of the present application further includes: receiving an audio shunting instruction before receiving the audio data generated by the plurality of application programs.

[0027] The audio shunting instruction is used to instruct to process the audio data into sub-audio data.

[0028] It can be understood that the audio shunting instruction is used to instruct the audio framework to not perform sound mixing processing on the media audio data, the navigation audio data and the call audio data, and to perform sound mixing processing on other audio data (such as notification audio data, warning audio data, ringtone audio data, etc.).

[0029] In the implementation manner, the audio shunting instruction is used to instruct the audio framework to make different processing on the audio data of different audio types, so as to obtain individual audio track data, thereby facilitating independent adjustment and playing of the audio corresponding to the sub-audio data of different audio types by the subsequent vehicle-mounted audio device.

[0030] With reference to the first aspect, in some implementations of the first aspect, the audio processing method provided by the embodiments of the present application further includes: receiving a cross-device audio stream conversion capability starting instruction before receiving the audio data generated by the plurality of application programs.

[0031] The cross-device audio stream conversion capability starting instruction is used to instruct the device virtualization service to create corresponding encoders for different types of sub-audio data, and to perform encoding processing on the sub-audio data to obtain encoded audio.

[0032] In the implementation manner, the cross-device audio stream conversion capability starting instruction is used to instruct the device virtualization service to create corresponding encoders for different types of sub-audio data, so that an encoder of one type only needs to perform encoding processing on sub-audio data of one type of audio, which can improve the encoding efficiency and effectively avoid encoding lag phenomenon, thereby making the audio played on the vehicle-mounted audio device more smooth.

[0033] With reference to the first aspect, in some implementations of the first aspect, the audio processing method provided by the embodiments of the present application further includes: generating the audio data in response to an input operation on the plurality of application programs before receiving the audio data generated by the plurality of application programs.

[0034] The application programs can include media application programs, navigation application programs, call application programs, shopping application programs, tool application programs, other application programs, and the like.

[0035] Optionally, the media application programs can include, but are not limited to, various music applications, various video applications, various short video applications, various social applications, various game applications, and various information applications.

[0036] The navigation application programs can include, but are not limited to, various map applications, various travel applications, various taxi-hailing applications, and various positioning applications.

[0037] The call application programs can include, but are not limited to, telephone applications and various network telephone applications.

[0038] The shopping application programs can include various shopping applications.

[0039] The tool application programs can include, but are not limited to, various browser applications, various learning applications, various weather applications, various office applications, and various consultation applications.

[0040] The other application programs refer to application programs other than the media application programs, the navigation application programs, the call application programs, the shopping application programs, and the tool application programs. For example, the other application programs can include notification applications, short message applications, and applications capable of providing safety prompts during driving.

[0041] In this implementation manner, different application programs can generate different types of audio data, which provides a basis for subsequent processing of the audio data into individual audio track data.

[0042] With reference to the first aspect, in some implementations of the first aspect, the audio processing method provided by the embodiments of the present application further includes: receiving an instruction for disconnecting the electronic device from the vehicle-mounted audio device, destroying the encoder, and instructing the virtual hardware abstraction layer to destroy the audio route.

[0043] In this implementation manner, the created encoder and the audio route are destroyed in time, which effectively avoids long-term occupation of system resources by the encoder and the audio route, improves the resource utilization rate, and improves the performance of the electronic device.

[0044] In a second aspect, the present application provides an audio processing method applied to an application program framework layer of a vehicle-mounted audio device, the vehicle-mounted audio device being connected to an electronic device. The method includes: receiving encoded audio sent through a preset transmission channel; decoding the encoded audio through a decoder created for the encoded audio to obtain decoded audio; and playing the decoded audio according to a playing strategy. The playing strategy includes independently adjusting a playing volume corresponding to the decoded audio, and / or playing the decoded audio corresponding to a specified loudspeaker through the specified loudspeaker.

[0045] The encoded audio is obtained by an encoder created by the application framework layer of the electronic device for the sub-audio data, the sub-audio data is obtained by the application framework layer of the electronic device for the audio data, and the audio data is generated by multiple applications in the electronic device.

[0046] The sub-audio data can include at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data, and the mixed sub-audio data is obtained by the application framework layer of the electronic device for mixing other sub-audio data except the media sub-audio data, the navigation sub-audio data, and the call sub-audio data.

[0047] In the audio processing method provided by the embodiments of the present application, the vehicle-mounted audio device receives not mixed audio data but separate encoded audio, and the decoded audio obtained by decoding the encoded audio is also separate audio (or each decoded audio obtained by decoding is audio of different audio types). Therefore, the vehicle-mounted audio device can independently play, adjust, and the like each audio, for example, can adjust the volume of music, navigation sound, and call sound respectively, and can specify different loudspeakers to play different types of audio. Thus, the audio processing method provided by the present application can realize many optimization scenarios for improving user experience between the electronic device and the vehicle, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the driver and passengers of the vehicle.

[0048] In combination with the second aspect, in some implementations of the second aspect, the audio processing method provided by the embodiments of the present application further includes: obtaining an audio type corresponding to the encoded audio; and creating a decoder matched with the audio type corresponding to the encoded audio.

[0049] Optionally, the audio type corresponding to the encoded audio of the media sub-audio data is a media audio type, the audio type corresponding to the encoded audio of the navigation sub-audio data is a navigation audio type, the audio type corresponding to the encoded audio of the call sub-audio data is a call audio type, and the audio type corresponding to the encoded audio of the mixed sub-audio data is a mixed audio type.

[0050] In this implementation, the decoders corresponding to different audio types are created for the encoded audio of different audio types, so that a decoder of one type only needs to decode the encoded audio of one type of audio, effectively avoiding the occurrence of audio lag, thereby improving the fluency of the finally played audio, and further improving the user experience.

[0051] In a third aspect, the present application provides an electronic device, comprising: one or more processors; one or more memories; a module installed with a plurality of application programs; the memory stores one or more programs, when the one or more programs are executed by the processor, the electronic device executes the method in the first aspect and any possible implementation manner thereof.

[0052] In a fourth aspect, the present application provides a vehicle-mounted audio device, comprising: one or more processors; one or more memories; a module installed with a plurality of application programs; the memory stores one or more programs, when the one or more programs are executed by the processor, the vehicle-mounted audio device executes the method in the second aspect and any possible implementation manner thereof.

[0053] In a fifth aspect, the present application provides a vehicle, comprising: one or more processors; one or more memories; the memory stores one or more programs, when the one or more programs are executed by the processor, the vehicle executes the method in the second aspect and any possible implementation manner thereof.

[0054] In a sixth aspect, the present application provides a chip, comprising a processor. The processor is used to read and execute a computer program stored in a memory, so as to execute the method in the first aspect and any possible implementation manner thereof, or execute the method in the second aspect and any possible implementation manner thereof.

[0055] Optionally, the chip further comprises a memory, and the memory is connected with the processor through a circuit or a wire.

[0056] Optionally, the chip further comprises a communication interface.

[0057] In a seventh aspect, the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, when the computer program is executed by a processor, the processor executes the method in the first aspect and any possible implementation manner thereof, or the processor executes the method in the second aspect and any possible implementation manner thereof.

[0058] In an eighth aspect, the present application provides a computer program product, comprising: computer program code, when the computer program code runs on an electronic device, the electronic device executes the method in the first aspect and any possible implementation manner thereof. Or, when the computer program code runs on a vehicle-mounted audio device, the vehicle-mounted audio device executes the method in the second aspect and any possible implementation manner thereof.

[0059] The technical effects obtained by the second aspect, the third aspect, the fourth aspect, the fifth aspect, the sixth aspect, the seventh aspect and the eighth aspect can refer to the technical effects obtained by the corresponding technical means in the first aspect and the second aspect, which will not be described herein. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 An application scenario of the audio processing method shown in the embodiments of the present application is illustrated;

[0061] Figure 2 Another application scenario of the audio processing method shown in the embodiments of the present application is illustrated;

[0062] Figure 3 A display interface shown in an example embodiment of the present application is illustrated;

[0063] Figure 4 A software structure block diagram of an electronic device shown in an example embodiment of the present application is illustrated;

[0064] Figure 5 A system architecture block diagram shown in an example embodiment of the present application is illustrated;

[0065] Figure 6 A flowchart of an audio processing method shown in an example embodiment of the present application is illustrated;

[0066] Figure 7 A flowchart of another audio processing method shown in an example embodiment of the present application is illustrated;

[0067] Figure 8 Another system architecture block diagram shown in an example embodiment of the present application is illustrated;

[0068] Figure 9 A flowchart of another audio processing method shown in an example embodiment of the present application is illustrated;

[0069] Figure 10 A flowchart of another audio processing method shown in an example embodiment of the present application is illustrated;

[0070] Figure 11 A software structure block diagram of another electronic device shown in an example embodiment of the present application is illustrated;

[0071] Figure 12 A package processing diagram shown in an example embodiment of the present application is illustrated;

[0072] Figure 13 A sending cycle diagram shown in an example embodiment of the present application is illustrated;

[0073] Figure 14 A package data diagram shown in an example embodiment of the present application is illustrated;

[0074] Figure 15 Another system architecture diagram shown for an exemplary embodiment of the present application;

[0075] Figure 16 A sub-packet processing diagram shown for an exemplary embodiment of the present application;

[0076] Figure 17 A flow diagram of an audio transmission method shown for an embodiment of the present application;

[0077] Figure 18 A flow diagram of another audio transmission method shown for an embodiment of the present application;

[0078] Figure 19 A flow diagram of yet another audio transmission method shown for an embodiment of the present application;

[0079] Figure 20 A flow diagram of yet another audio transmission method shown for an embodiment of the present application;

[0080] Figure 21 A hardware structure diagram of an electronic device shown for an exemplary embodiment of the present application;

[0081] Figure 22 A structure diagram of a chip provided for an embodiment of the present application. DETAILED DESCRIPTION

[0082] The technical solutions in the present application will be described below with reference to the drawings.

[0083] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the present application only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0084] Hereinafter, the terms "first" and "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features. In the description of the embodiments, unless otherwise specified, the meaning of "multiple" is two or more than two.

[0085] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specified

[0086] In order to better understand the audio processing method and the audio transmission method provided by the embodiments of the present application, the following first explains some terms involved in the embodiments of the present application, so as to facilitate the understanding of those skilled in the art.

[0087] 1. Vehicle information system

[0088] The vehicle information system is also called the automobile information system, and is referred to as a car machine in the embodiments of the present application.

[0089] The vehicle information system is a device that enables a driver to timely understand vehicle operation information and external information during driving.

[0090] 2. Input buffer

[0091] The input buffer is also referred to as inBuffer in the embodiments of the present application, and can be regarded as a memory area for temporarily storing data.

[0092] In the embodiments of the present application, one encoder corresponds to one inBuffer, and each inBuffer is used to receive and store encoded audio sent by the corresponding encoder.

[0093] The above is a brief introduction to the terms involved in the embodiments of the present application, and the following will not be described again.

[0094] In the interaction scenario between an electronic device (such as a mobile phone) and a car machine, for example, in the scenario of audio transmission and playing, the mobile phone and the car machine are in a connected state (or an interconnected state). When different types of audio data (such as media audio data, navigation audio data, call audio data, notification audio data, etc.) are to be sent from the mobile phone to the car machine, the mobile phone will perform audio mixing processing on the different types of audio data, obtain mixed audio data, and then send the mixed audio data to the car machine.

[0095] Since the mixing processing is an irreversible process, after receiving the mixed audio data, the car machine cannot split the mixed audio data, that is, cannot split each type of audio data from the mixed audio data. For example, the car machine cannot split the individual media audio data, navigation audio data, call audio data, notification audio data, etc. from the mixed audio data.

[0096] Based on the above reasons, the car machine can only uniformly play, adjust, etc. the mixed audio data, and cannot individually play, adjust, etc. each type of audio data, which leads to many optimization scenarios for improving user experience between the mobile phone and the car machine cannot be implemented, and reduces the driving and riding experience of the user. Among them, the optimization scenarios for improving user experience include but are not limited to that the car machine controls the playback volume of different types of audio data respectively, the car machine controls the navigation audio data to be played on the driver's seat speaker alone, the car machine controls the navigation audio data to be played on the driver's seat speaker and the copilot's seat speaker, etc.

[0097] Therefore, the embodiments of the present application provide an audio processing method, which is applied to an application framework layer of an electronic device connected with a vehicle-mounted audio device. The application framework layer of the electronic device receives audio data generated by a plurality of applications, processes the audio data into sub-audio data, the sub-audio data including at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed audio sub-audio data, encodes each sub-audio data through a corresponding number of encoders created for the sub-audio data to obtain encoded audio, and sends the encoded audio to the vehicle-mounted audio device through a preset transmission channel.

[0098] Since the data contained in the sub-audio data is a single audio track data (or each data contained in the sub-audio data is different type of audio data), after encoding the single audio track data respectively and sending to the vehicle-mounted audio device, the vehicle-mounted audio device decodes the encoded audio through a decoder created for the encoded audio to obtain decoded audio, which is also a single audio (or each decoded audio is different type of audio). Therefore, the vehicle-mounted audio device can independently play, adjust, etc. each audio, such as adjusting the volume of music, navigation sound, and call sound respectively, and specifying different speakers to play different types of audio. Thus, the audio processing method provided by the present application can implement many optimization scenarios for improving user experience between the electronic device and the vehicle, can meet the personalized needs of vehicle audio playback, and improves the driving and riding experience of the driver and the passenger of the vehicle.

[0099] The application scenarios of the audio processing method provided by the embodiments of the present application are described below in conjunction with the accompanying drawings.

[0100] It should be noted that in some embodiments of the present application, the electronic device can be a mobile phone, a smart screen, a tablet computer, a wearable device (such as a smart watch, smart glasses, a smart bracelet, a smart necklace, etc.), a television, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc., or can be other devices or apparatuses capable of audio processing, and the specific type of the electronic device is not limited in the embodiments of the present application.

[0101] Please refer to Figure 1 , Figure 1 An application scenario of the audio processing method shown in the embodiments of the present application is shown. As shown in Figure 1 , the application scenario includes an electronic device 100 and a vehicle 200, and it should be understood that the electronic device 100 is taken as a mobile phone for example in the embodiments of the present application.

[0102] The vehicle 200 can be any type of vehicle that carries people and / or objects and moves through a power system such as a transmitter and a battery, including but not limited to a car, a sedan, a truck, a bus, an electric vehicle, a motorcycle, a motor home, a train, a multiple-unit train, a high-speed multiple-unit passenger train, etc.

[0103] In one possible implementation manner, the vehicle 200 can be a vehicle driven by a driver. Alternatively, in another possible implementation manner, the vehicle 200 can also be a vehicle with certain automatic driving capability.

[0104] The vehicle 200 can include an in-vehicle audio device 210. The in-vehicle audio device 210 and the electronic device 100 can be connected through communication to realize information interaction between the in-vehicle audio device 210 and the electronic device 100.

[0105] The communication connection can include wired communication connection and short-distance wireless communication connection. The wired communication connection can include Universal Serial Bus (USB) connection, and the short-distance wireless communication connection can include but is not limited to Bluetooth connection, WIreless Fidelity (Wi-Fi) connection, Wi-Fi Peer-to-Peer (P2P) connection, Zigbee connection, Near Field Communication (NFC) connection, etc.

[0106] In one possible implementation, the in-vehicle audio device 210 can be implemented as software within the hardware of the vehicle 200, which can be used to access, retrieve, and play content from the application of the electronic device 100. In another possible implementation, the in-vehicle audio device 210 can also be implemented as another independent hardware device capable of connecting to the vehicle 200.

[0107] above Figure 1 The corresponding embodiments mainly demonstrate the application scenarios of the audio processing method from the perspective of the outside of the vehicle 200. The following demonstrates the application scenarios of the audio processing method from the perspective of the inside of the vehicle 200.

[0108] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating another application scenario of the audio processing method shown in the embodiments of this application. For example... Figure 2 As shown, when the in-vehicle audio device 210 is implemented as another independent hardware device capable of connecting to the vehicle 200, the in-vehicle audio device 210 can be located on the right side of the steering wheel of the vehicle 200. This is only an illustrative example; the in-vehicle audio device 210 can also be located in front of the driver, in front of the passenger, etc., and there is no limitation thereto.

[0109] For example, before a driver or passenger wants to transmit content from the application of electronic device 100 to in-vehicle audio device 210, or wants to use in-vehicle audio device 210 to access, retrieve, play, or perform other operations on the content in the application of electronic device 100, a communication connection between in-vehicle audio device 210 and electronic device 100 needs to be established.

[0110] For example, a wired communication connection can be established between the in-vehicle audio device 210 and the electronic device 100 using a USB cable. It should be understood that whether establishing a wired communication connection for the first time or not, the flat end of the USB cable is connected to the in-vehicle audio device 210, and the micro end is connected to the electronic device 100. Then, following the prompts displayed on the in-vehicle audio device 210's display and / or the electronic device 100's display for establishing the wired communication connection, the wired communication connection between the in-vehicle audio device 210 and the electronic device 100 is established.

[0111] For example, Bluetooth can be used to establish the short-range wireless communication connection between the in-vehicle audio device 210 and the electronic device 100. For example, when Bluetooth is used to establish the short-range wireless communication connection between the in-vehicle audio device 210 and the electronic device 100 for the first time, the driver or the passenger needs to manually perform a pairing operation for the in-vehicle audio device 210 and the electronic device 100. For example, the driver or the passenger can turn on Bluetooth for the in-vehicle audio device 210 and the electronic device 100, and ensure that the Bluetooth of the in-vehicle audio device 210 and the Bluetooth of the electronic device 100 are in a discoverable state.

[0112] Then, the driver or the passenger can click the search device on the Bluetooth interface displayed by the electronic device 100 to enable the electronic device 100 to search for the Bluetooth of the in-vehicle audio device 210. The driver or the passenger can click the Bluetooth of the in-vehicle audio device 210, the Bluetooth setting, and the pairing operation in sequence. At this time, the display interface of the electronic device 100 can pop up a prompt box for inputting the pairing code. The driver or the passenger can input the pairing code in the prompt box and click the connection, and click the pairing on the Bluetooth interface displayed by the in-vehicle audio device 210. If the pairing code is input correctly, the in-vehicle audio device 210 can display the connection in the Bluetooth interface, thereby establishing the short-range wireless communication connection between the in-vehicle audio device 210 and the electronic device 100.

[0113] It is worth noting that when Bluetooth is used to establish the short-range wireless communication connection between the in-vehicle audio device 210 and the electronic device 100 for the second time, if the Bluetooth of the in-vehicle audio device 210 and the Bluetooth of the electronic device 100 are in an open state, and the distance between the in-vehicle audio device 210 and the electronic device 100 satisfies the preset connection threshold, the Bluetooth of the in-vehicle audio device 210 and the Bluetooth of the electronic device 100 can automatically establish the short-range wireless communication connection.

[0114] It is understood that the "preset connection threshold" refers to the maximum transmission distance for the Bluetooth communication between the in-vehicle audio device 210 and the electronic device 100, for example, within 8 meters, within 10 meters, within 15 meters, and the like.

[0115] After the communication connection between the in-vehicle audio device 210 and the electronic device 100 is established, the content in the application of the electronic device 100 can be transmitted to the in-vehicle audio device, and the transmitted content can be displayed in the display interface of the in-vehicle audio device 210. Alternatively, the content in the application of the electronic device 100 can be accessed, stored, played, and the like by the in-vehicle audio device 210, and the operation result can be displayed in the display interface of the in-vehicle audio device 210.

[0116] The display interface of the vehicle audio device 210 is described below with reference to the accompanying drawings. It can be understood that the display interface of the vehicle audio device 210 can also independently display relevant information of the vehicle 200 without establishing a communication connection between the vehicle audio device 210 and the electronic device 100. The relevant information includes but is not limited to navigation information, driving information, audio information, video information, call information, power information, time information, volume information, and the like.

[0117] In the embodiments of the present application, the content displayed on the display interface of the vehicle audio device 210 is described by way of example in the case where the vehicle audio device 210 and the electronic device 100 establish a communication connection. Please refer to Figure 3 , Figure 3 A display interface diagram is shown in the embodiments of the present application. For example, a media application (such as a music application) in the electronic device 100 sends music to the vehicle audio device 210, and the display interface of the vehicle audio device 210 displays relevant content of the music, such as shown in Figure 3 The display interface of the vehicle audio device 210 displays the song name, the singer name, the classification of the song (such as I like, local songs, driving radio, and the like), and can also display the previous song, pause, next song, and the like. At the same time, the music is played through the loudspeaker of the vehicle 200, realizing the interactive scenario of playing the music in the music application of the electronic device 100 through the loudspeaker of the vehicle 200.

[0118] For another example, a navigation application (such as a navigation application) in the electronic device 100 sends navigation data to the vehicle audio device 210, and the display interface of the vehicle audio device 210 displays relevant content of the navigation data, such as shown in Figure 3 The display interface of the vehicle audio device 210 displays route information such as "50km left turn, enter XX road", and can also display the driving time (Time), the distance to the destination (Distance), the arrival time (Arrival), the "end navigation" control, and the like. At the same time, the navigation audio is played through the loudspeaker of the vehicle 200, realizing the interactive scenario of playing the navigation sound in the navigation application of the electronic device 100 through the loudspeaker of the vehicle 200.

[0119] For another example, a call application (such as a telephone application) in the electronic device 100 sends call data to the vehicle audio device 210, and the display interface of the vehicle audio device 210 displays relevant controls for controlling the call data, such as shown in Figure 3As shown, the display interface of the vehicle-mounted audio device 210 displays "answer", "hang up", "mute" and the like, and the vehicle 200 takes different responses according to the click operation of the driver or the passenger on different controls. For example, when the driver or the passenger clicks the "answer" control, the vehicle 200 plays the call sound through the loudspeaker, and at the same time, the driver or the passenger can input his own voice through the microphone of the vehicle 200, realizing the interactive scene of answering the phone of the electronic device 100 through the loudspeaker and the microphone of the vehicle 200.

[0120] As shown, the display interface of the vehicle-mounted audio device 210 can also display power information, time information, alarm information, main menu information, earphone information and the like, which are only exemplary and actual display is subject to change. Figure 3

[0121] In the related art, when the electronic device sends different types of audio data such as music, navigation data, call data to the vehicle-mounted audio device, the electronic device performs mixing processing on these different types of audio data to obtain mixed audio data, and then sends the mixed audio data to the vehicle-mounted audio device. Since mixing processing is an irreversible process, the vehicle-mounted audio device cannot split the mixed audio data after receiving the mixed audio data, that is, cannot separately split the music, navigation data and call data from the mixed audio data.

[0122] Therefore, the vehicle-mounted audio device can only uniformly play, adjust and the like the mixed audio data, and cannot separately adjust each type of audio data. For example, it can only uniformly increase or decrease the playback volume corresponding to the mixed audio data, which causes the volume of music, navigation sound and call sound to be uniformly increased or decreased, and cannot independently adjust the volume of music, navigation sound and call sound. For another example, it can only play the mixed audio data on a unified loudspeaker, and cannot specify different loudspeakers to play different audio data.

[0123] In the audio processing method provided by the embodiment of the present application, the electronic device processes the audio data generated by the plurality of application programs into sub-audio data, the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data and mixed audio data, encodes each sub-audio data through the corresponding number of encoders created for the sub-audio data to obtain encoded audio, and sends the encoded audio to the vehicle-mounted audio device through a preset transmission channel.

[0124] ​Since the data contained in the sub-audio data is single audio track data (or each data contained in the sub-audio data is different type of audio data), the single audio track data is respectively encoded and sent to the vehicle-mounted audio device, and the vehicle-mounted audio device decodes the encoded audio through the decoder created for the encoded audio to obtain the decoded audio which is also single audio (or each decoded audio is different type of audio). Therefore, the vehicle-mounted audio device can independently play, adjust, etc. each audio, for example, can respectively adjust the volume of music, navigation sound and call sound, and can specify different speakers to play different types of audio. Thus, the audio processing method provided by the present application can realize many optimization scenarios between the electronic device and the vehicle for improving user experience, can meet the personalized needs of vehicle audio playback, and improves the driving and riding experience of the vehicle driver and passengers.

[0125] The audio processing method provided by the embodiment of the present application will be described below in combination with the software structure. Please refer to Figure 4 , Figure 4 The software structure block diagram of the electronic device shown in an exemplary embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the electronic device 100 is taken as an example of an Android system, and the Android system is divided into three layers, from top to bottom, the application layer, the application framework layer, and the virtual hardware abstraction layer.

[0126] The application layer can include a series of application packages. In the embodiment of the present application, the application layer can include media class applications, navigation class applications, call class applications, shopping class applications, tool class applications, other applications, etc.

[0127] Among them, the media class application can include but is not limited to various music applications, various video applications, various short video applications, various social applications, various game applications, and various information applications.

[0128] The navigation class application can include but is not limited to various map applications, various travel applications, various taxi applications, and various positioning applications.

[0129] The call class application can include but is not limited to telephone applications and various network telephone applications.

[0130] The shopping class application can include various shopping applications.

[0131] The tool class application can include but is not limited to various browser applications, various learning applications, various weather applications, various office applications, and various consultation applications.

[0132] The other application refers to an application other than the media application, the navigation application, the call application, the shopping application, and the tool application. For example, the other application can include a notification application, a short message application, an application capable of providing a safety prompt during driving, and the like.

[0133] It should be noted that, in the embodiments of the present application, no matter which application, audio data can be generated during use.

[0134] As shown in Figure 4 The application package can include a media application, a navigation application, an other application, and a call application. The media application can include a music application, a video application, and a short video application. The navigation application can include a map application, a travel application, and a positioning application. The other application can include a notification application and a short message application. The call application can include a telephone application.

[0135] It should be understood that different applications can generate different types of audio data. For example, the media application can generate media audio data, the navigation application can generate navigation audio data, the call application can generate call audio data, and the other application can generate notification audio data, warning audio data, and ringtone audio data. These different types of audio data will be transmitted to the audio framework of the application framework layer, and after being processed by the audio framework, the virtual hardware abstraction layer, and the device virtualization service in turn, they will be finally sent to the vehicle audio device.

[0136] Optionally, the application layer can further include a car application. The car application is built in the application layer of the electronic device. When it is detected that the electronic device and the vehicle audio device establish a communication connection, the car application sends an audio shunting instruction to the audio framework of the application framework layer, and sends a cross-device audio stream conversion capability starting instruction to the device virtualization service of the application framework layer.

[0137] The audio shunting instruction is used to instruct the audio framework to process the audio data generated by the plurality of applications into sub-audio data. It can be understood that the audio shunting instruction is used to instruct the audio framework to not perform audio mixing processing on the media audio data, the navigation audio data, and the call audio data, and to perform audio mixing processing on the other audio data (such as notification audio data, warning audio data, and ringtone audio data).

[0138] The cross-device audio stream conversion capability starting instruction is used to instruct the device virtualization service to create a corresponding encoder for different types of sub-audio data, and to perform encoding processing on the sub-audio data to obtain encoded audio.

[0139] Optionally, in one possible implementation, the car machine application does not have a corresponding application display interface and is invisible to the user.

[0140] The application framework layer can include an audio framework and a device virtualization service. The audio framework is introduced first.

[0141] The audio framework is used to receive the audio split instruction sent by the car machine application and receive different types of audio data sent by each application in the application program layer. It should be understood that the audio split instruction is sent by the car machine application immediately when it senses that the electronic device and the vehicle audio device establish a communication connection. Therefore, for the audio framework, it first receives the audio split instruction sent by the car machine application, and then receives different types of audio data sent by each application in the application program layer according to the use of each application by the user.

[0142] Optionally, in one possible implementation, when the audio framework receives the audio split instruction sent by the car machine application, the received media audio data, navigation audio data, call audio data, and other audio data (such as notification audio data, warning audio data, ringtone audio data, etc.) are no longer mixed.

[0143] Optionally, in another possible implementation, when the audio framework receives the audio split instruction sent by the car machine application, the received media audio data, navigation audio data, and call audio data are no longer mixed, and the received other audio data (such as notification audio data, warning audio data, ringtone audio data, etc.) are mixed.

[0144] The audio framework is also used to identify the audio type of each audio data. The audio type can include media audio type, navigation audio type, call audio type, and other audio type. The other audio type can include notification audio type, warning audio type, ringtone audio type, etc.

[0145] Generally, the system of the electronic device identifies the audio data generated by each application in the application program layer, and the audio framework can identify the audio type of each audio data according to the audio type identification of each audio data.

[0146] For example, the system identifies the audio data generated by the music application as "media-music" information, and the audio framework identifies the audio type identification in the audio data as "media-music" when receiving the audio data, and determines that the audio type of the audio data is media audio type.

[0147] For example, the system identifies "navigation" information from audio data generated by a navigation application, and the audio framework identifies the audio type in the audio data as "navigation" when receiving the audio data.

[0148] Optionally, in a possible implementation, an application white list is set in the audio framework in advance, and the package name of the navigation application can be recorded in the application white list. When the audio framework receives the audio data, the package name of the application sending the audio data is determined. If it is detected that the package name is in the application white list, it is determined that the audio type of the audio data is the navigation audio type. In this implementation, the application white list in the audio framework can be used to identify the audio type of the audio data generated by the navigation application without identification by the system.

[0149] The audio framework is further configured to process the audio data generated by the plurality of applications into sub-audio data after identifying the audio types of the audio data. The sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data.

[0150] The mixed sub-audio data is obtained by mixing other sub-audio data except the media sub-audio data, the navigation sub-audio data, and the call sub-audio data.

[0151] For example, the audio data generated by the plurality of applications can include media audio data, navigation audio data, call audio data, notification audio data, alarm audio data, and ringtone audio data. The number and types of the audio data generated are related to the applications used by the user, and are not limited.

[0152] The audio framework identifies that the audio type of the media audio data is the media audio type, the audio type of the navigation audio data is the navigation audio type, the audio type of the call audio data is the call audio type, and the audio types of the notification audio data, the alarm audio data, and the ringtone audio data are other audio types.

[0153] The audio framework does not mix the audio data with the audio types of the media audio type, the navigation audio type, and the call audio type. In a possible implementation, the audio framework directly sends the media audio data, the navigation audio data, and the call audio data to the virtual hardware abstraction layer after determining the audio types of the media audio data, the navigation audio data, and the call audio data.

[0154] In another possible implementation, after determining the audio types of the media audio data, the navigation audio data and the call audio data, the audio framework determines the media audio data as media sub-audio data, determines the navigation audio data as navigation sub-audio data, and determines the call audio data as call sub-audio data, and then sends the media sub-audio data, the navigation sub-audio data and the call sub-audio data to the virtual hardware abstraction layer.

[0155] The audio framework performs mixing processing on the audio data of the other audio type. For example, the audio framework performs mixing processing on the notification audio data, the alert audio data and the ringtone audio data to obtain mixed sub-audio data, and then sends the mixed sub-audio data to the virtual hardware abstraction layer.

[0156] The virtual hardware abstraction layer creates a corresponding number of audio routes for the sub-audio data, or in other words, the virtual hardware abstraction layer creates a corresponding audio route for the sub-audio data of each audio type, so that the created audio routes match the audio types of the respective sub-audio data.

[0157] The audio route created by the virtual hardware abstraction layer is used to forward the sub-audio data corresponding to the audio route to an encoder that matches the sub-audio data. The encoder is created by a device virtualization service in the application framework layer, which will be described in detail below.

[0158] Compared with the related art, in which mixed audio data (data obtained after mixing different types of audio by a mobile phone) is forwarded through one audio route, in this implementation, a corresponding audio route is created for the sub-audio data of different audio types, so that an audio route of one type only needs to forward sub-audio data of one audio type. This not only helps the device virtualization service to quickly distinguish the audio types of the sub-audio data it receives, but also improves the forwarding efficiency.

[0159] Optionally, in a possible implementation, if the audio framework does not perform mixing processing on the received audio data, but determines the media audio data as media sub-audio data, the navigation audio data as navigation sub-audio data, the call audio data as call sub-audio data, the notification audio data as notification sub-audio data, the alert audio data as alert sub-audio data, and the ringtone audio data as ringtone sub-audio data, then the virtual hardware abstraction layer creates a corresponding audio route for each type of sub-audio data.

[0160] For example, the virtual hardware abstraction layer creates a first audio route for the media sub-audio data, a second audio route for the navigation sub-audio data, a third audio route for the call sub-audio data, a fourth audio route for the notification sub-audio data, a fifth audio route for the warning sub-audio data, and a sixth audio route for the ringtone sub-audio data.

[0161] Optionally, in another possible implementation, if the audio framework mixes the audio data of other audio types, the virtual hardware abstraction layer creates a first audio route for the media sub-audio data, a second audio route for the navigation sub-audio data, and a third audio route for the mixed sub-audio data, and a call audio route for the call sub-audio data.

[0162] Optionally, the virtual hardware abstraction layer identifies the created audio routes according to the audio types of the sub-audio data when creating the audio routes, which is conducive to distinguishing the created audio routes, and is conducive to the audio framework quickly forwarding the sub-audio data matched with the audio routes to the audio routes according to the identification, and is conducive to the device virtualization service quickly distinguishing the audio types of the received sub-audio data (which can be understood as the sub-audio data transferred from an audio route, and the audio type of the sub-audio data can be quickly determined according to the identification of the audio route).

[0163] It should be understood that the manner of identifying the created audio routes is not limited. For example, the audio routes can be identified by different audio route names, such as a media audio route, a navigation audio route, a mixed audio route, a call audio route, etc. For another example, a first audio route, a second audio route, a third audio route, a fourth audio route, etc. For yet another example, a route 1, a route 2, a route 3, etc.

[0164] Optionally, the audio framework can also set different audio types for different audio routes. For example, the audio framework sets the audio type corresponding to the first audio route as a media audio type, sets the audio type corresponding to the second audio route as a navigation audio type, sets the audio type corresponding to the third audio route as a mixed audio type, and sets the audio type corresponding to the call audio route as a call audio type.

[0165] Optionally, in one possible implementation, the virtual hardware abstraction layer can create the audio routes in advance for each audio type of sub-audio data, and store the created audio routes in the virtual hardware abstraction layer for a long time, so as to facilitate the subsequent quick forwarding of each sub-audio data to the encoder through the created audio routes.

[0166] Optionally, in another possible implementation, a cycle starts from the electronic device establishing a connection with the in-vehicle audio device and ends when the electronic device disconnects from the in-vehicle audio device. In each cycle, the virtual hardware abstraction layer creates an audio route corresponding to each type of sub-audio data. For example, the virtual hardware abstraction layer creates an audio route corresponding to a certain type of sub-audio data when it first receives the sub-audio data sent by the audio framework. Thereafter, in the cycle, the sub-audio data corresponding to the audio route can be directly forwarded to the encoder through the audio route.

[0167] Optionally, when it is detected that the electronic device disconnects from the in-vehicle audio device, the virtual hardware abstraction layer destroys the created audio route. This implementation destroys the created audio route at the end of a cycle, effectively avoiding long-term occupation of system resources by the audio route, improving resource utilization, and improving the performance of the electronic device.

[0168] It can be understood that if the virtual hardware abstraction layer never receives a certain type of sub-audio data sent by the audio framework in a cycle, the audio route corresponding to the type of sub-audio data can not be created in the cycle. This implementation creates a corresponding audio route when needed, which can effectively reduce resource consumption and improve resource utilization.

[0169] The device virtualization service can include a virtual audio service, a virtual call service, and a transmission channel.

[0170] The virtual audio service is configured to receive each non-call sub-audio data forwarded by the virtual hardware abstraction layer through each audio route and send each non-call sub-audio data to a corresponding encoder. The non-call sub-audio data can include media sub-audio data, navigation sub-audio data, and mixed audio sub-audio data.

[0171] The virtual call service is configured to receive call sub-audio data and send the call sub-audio data to a corresponding call encoder.

[0172] The application framework layer creates a corresponding number of encoders for sub-audio data, or in other words, the application framework layer creates a corresponding encoder for each type of sub-audio data, so that the created encoders match the audio types of each sub-audio data.

[0173] The encoder created by the application framework layer is configured to encode the sub-audio data transmitted to the encoder to obtain a corresponding encoded audio. Thereafter, the encoded audio is sent to the in-vehicle audio device through the transmission channel in the device virtualization service.

[0174] Optionally, in one possible implementation, if the audio framework does not perform mixing processing on the received audio data, and the virtual hardware abstraction layer creates an audio route corresponding to each of the sub-audio data, the device virtualization service creates an encoder corresponding to each of the sub-audio data forwarded to the device virtualization service through the audio routes. For example, a first encoder is created for the media sub-audio data forwarded to the device virtualization service through the first audio route, the first encoder is configured to perform encoding processing on the media sub-audio data to obtain encoded audio corresponding to the media sub-audio data; a second encoder is created for the navigation sub-audio data forwarded to the device virtualization service through the second audio route, the second encoder is configured to perform encoding processing on the navigation sub-audio data to obtain encoded audio corresponding to the navigation sub-audio data; a third encoder is created for the call sub-audio data forwarded to the device virtualization service through the third audio route, the third encoder is configured to perform encoding processing on the call sub-audio data to obtain encoded audio corresponding to the call sub-audio data; a fourth encoder is created for the notification sub-audio data forwarded to the device virtualization service through the fourth audio route, the fourth encoder is configured to perform encoding processing on the notification sub-audio data to obtain encoded audio corresponding to the notification sub-audio data; a fifth encoder is created for the warning sub-audio data forwarded to the device virtualization service through the fifth audio route, the fifth encoder is configured to perform encoding processing on the warning sub-audio data to obtain encoded audio corresponding to the warning sub-audio data; and a sixth encoder is created for the ringtone sub-audio data forwarded to the device virtualization service through the sixth audio route, the sixth encoder is configured to perform encoding processing on the ringtone sub-audio data to obtain encoded audio corresponding to the ringtone sub-audio data.

[0175] Optionally, in another possible implementation, if the audio framework performs mixing processing on the audio data of other audio types, the virtual hardware abstraction layer creates a first audio route for the media sub-audio data, a second audio route for the navigation sub-audio data, a third audio route for the mixed sub-audio data, and a call audio route for the call sub-audio data. The device virtualization service creates an encoder corresponding to each of the sub-audio data forwarded to the device virtualization service through the audio routes.

[0176] For example, a third encoder is created for the mixed sub-audio data forwarded to the device virtualization service through the third audio route, the third encoder is configured to perform encoding processing on the mixed sub-audio data to obtain encoded audio corresponding to the mixed sub-audio data; and a call encoder is created for the call sub-audio data forwarded to the device virtualization service through the call audio route, the call encoder is configured to perform encoding processing on the call sub-audio data to obtain encoded audio corresponding to the call sub-audio data.

[0177] In this implementation, the respective encoder is created for the sub-audio data of different audio types, so that the encoder of one type only encodes the sub-audio data of one type of audio, which can improve the encoding efficiency and effectively avoid the encoding lag phenomenon, so that the audio played on the vehicle-mounted audio device is smoother. The encoding lag phenomenon is caused by the encoding of the sub-audio data of multiple types of audio by the encoder of one type, which increases the waiting time for encoding each sub-audio data.

[0178] Optionally, the device virtualization service identifies the created encoder according to the audio type of the received sub-audio data or the identifier of the audio route when creating the encoder, which is beneficial to distinguish the created encoders and is also beneficial to the subsequent transmission channel for quickly sending the encoded audio of each encoder to the corresponding decoder of the vehicle-mounted audio device.

[0179] It should be understood that the manner of identifying the created encoder is not limited. For example, the encoder can be identified by different encoder names, such as a first encoder, a second encoder, a third encoder, a call encoder, etc. For another example, a route 1 encoder, a route 2 encoder, a route 3 encoder, etc.

[0180] Optionally, in one possible implementation, the device virtualization service creates the encoder, which can be pre-created for each type of sub-audio data and stored in the device virtualization service for a long time, so as to facilitate the subsequent quick encoding of each type of sub-audio data by the created encoder.

[0181] Optionally, in another possible implementation, a period starts from the establishment of the connection between the electronic device and the vehicle-mounted audio device and ends when the connection between the electronic device and the vehicle-mounted audio device is disconnected. In each period, the device virtualization service creates an encoder corresponding to each type of sub-audio data. For example, the device virtualization service creates an encoder corresponding to a certain type of audio when it first receives the sub-audio data of the type of audio forwarded by a certain audio route. Then, in this period, the sub-audio data of this type of audio can be directly encoded by the encoder.

[0182] Optionally, when it is detected that the electronic device is disconnected from the vehicle-mounted audio device, the device virtualization service destroys the created encoder. In this implementation, the created encoder is destroyed at the end of a period, which effectively avoids the long-term occupation of system resources by the encoder, improves the resource utilization, and improves the performance of the electronic device.

[0183] It can be understood that if the device virtualization service does not receive the sub-audio data of a certain audio type sent by a certain audio routing in a cycle, the encoder corresponding to the audio type can not be created in the cycle. In this implementation manner, the corresponding encoder is created when needed, which can effectively reduce the consumption of resources and improve the resource utilization.

[0184] In the foregoing implementation manner, the virtual hardware abstraction layer creates a call audio routing for the call sub-audio data, and the call sub-audio data can be forwarded to the encoder of the device virtualization service through the call audio routing. Optionally, in a possible implementation manner, the electronic device can include an adaptive data signal processing (ADSP) framework, and the call audio data generated by the call application is directly transmitted to the virtual hardware abstraction layer through the ADSP framework. The virtual hardware abstraction layer can further include a pre-established call channel, through which the call audio data is transmitted to the virtual call service of the device virtualization service. The virtual call service sends the call audio data to the corresponding call encoder. In this implementation manner, the separate call channel and the call encoder are established for the call audio data, which can ensure that the transmission process and the encoding process of the call audio data are not disturbed, and the call quality is effectively improved.

[0185] The device virtualization service includes a preset transmission channel, which is used to send the encoded audio obtained after the encoding processing of each encoder to the vehicle-mounted audio device. For example, the transmission channel can include a plurality of sending units, and each sending unit is used to send an encoded audio of a type.

[0186] For example, the transmission channel can include a first sending unit, a second sending unit, a third sending unit, and a fourth sending unit. The first sending unit is used to send the encoded audio corresponding to the media sub-audio data to the vehicle-mounted audio device; the second sending unit is used to send the encoded audio corresponding to the navigation sub-audio data to the vehicle-mounted audio device; the third sending unit is used to send the encoded audio corresponding to the mixed sub-audio data to the vehicle-mounted audio device; and the fourth sending unit is used to send the encoded audio corresponding to the call sub-audio data to the vehicle-mounted audio device.

[0187] In this implementation manner, each encoded audio is transmitted through one transmission channel, which reduces the resource consumption and effectively saves the resources of the electronic device.

[0188] Optionally, in a possible implementation manner, the transmission channel can include a first transmission channel and a second transmission channel, the first transmission channel is used to transmit the encoded audio corresponding to the non-call audio sub-data, and the second transmission channel is used to transmit the encoded audio corresponding to the call sub-audio data.

[0189] The first transmission channel may include multiple transmitting units, each transmitting a type of encoded audio. Non-call sub-audio data may include media sub-audio data, navigation sub-audio data, and mixing sub-audio data.

[0190] In this implementation, a separate transmission channel is established for the sub-audio data of the call, which ensures that the encoded audio corresponding to the sub-audio data is not interfered with during transmission, thus effectively improving the call quality.

[0191] The audio processing method provided in this application embodiment is applied to the application framework layer of an electronic device. The electronic device is connected to an in-vehicle audio device. The application framework layer of the electronic device receives audio data generated by multiple applications, processes the audio data into sub-audio data, and the sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixing sub-audio data. By creating a corresponding number of encoders for the sub-audio data, each sub-audio data is encoded to obtain encoded audio, and the encoded audio is sent to the in-vehicle audio device through a preset transmission channel.

[0192] Since the sub-audio data contains individual audio tracks (or each piece of data within the sub-audio data is of a different type), these individual audio tracks are encoded and sent to the in-vehicle audio device. The in-vehicle audio device then decodes the encoded audio using a decoder created for the encoded audio, resulting in individual audio files (or each decoded audio file is of a different type). Therefore, the in-vehicle audio device can independently play and adjust each audio file. For example, it can adjust the volume of music, navigation sounds, and call sounds separately, and can assign different speakers to play different types of audio. Thus, the audio processing method provided in this application enables many optimization scenarios between electronic devices and vehicles to enhance user experience, meets the personalized needs of vehicle audio playback, and improves the driving and riding experience for drivers and passengers.

[0193] The audio processing method provided in the embodiments of this application has been described above in conjunction with the software structure. The audio processing method provided in the embodiments of this application will now be described below in conjunction with the system architecture. Please refer to... Figure 5 , Figure 5 This is a system architecture block diagram illustrating an exemplary embodiment of this application. Figure 5 As shown, the system architecture block diagram is in Figure 4 Based on the corresponding electronic device software architecture diagram, the software architecture of in-vehicle audio equipment has been added. About Figure 4 The software structure of the corresponding electronic devices will not be elaborated here. The software structure of the in-vehicle audio device will be described in detail below.

[0194] In the embodiments of the present application, the system of the in-vehicle audio device is divided into two layers, namely an application layer and an application framework layer, and the application layer and the application framework layer communicate through a software interface.

[0195] The application layer of the in-vehicle audio device can include a car application, which can be used to sense that the electronic device and the in-vehicle audio device establish a communication connection and sense that the electronic device and the in-vehicle audio device are disconnected.

[0196] The application framework layer of the in-vehicle audio device can include a device virtualization service and an audio framework.

[0197] The device virtualization service can include a transmission channel, a decoder, and an audio playing module.

[0198] The device virtualization service includes a transmission channel, which is used to transmit each encoded audio sent by the electronic device to a corresponding decoder. For example, the transmission channel can include a plurality of receiving units, each of which is used to receive one type of encoded audio sent by the electronic device, and simultaneously transmit the one type of encoded audio to a corresponding decoder.

[0199] For example, the transmission channel can include a first receiving unit, a second receiving unit, a third receiving unit, and a fourth receiving unit. The first receiving unit is used to transmit the encoded audio corresponding to the media sub-audio data; the second receiving unit is used to transmit the encoded audio corresponding to the navigation sub-audio data; the third receiving unit is used to transmit the encoded audio corresponding to the mixed sub-audio data; and the fourth receiving unit is used to transmit the encoded audio corresponding to the call sub-audio data.

[0200] In this implementation manner, each encoded audio is transmitted through one transmission channel, which reduces resource consumption and effectively saves the resources of the electronic device.

[0201] Optionally, in a possible implementation manner, the transmission channel can include a third transmission channel and a fourth transmission channel, the third transmission channel is used to transmit the encoded audio corresponding to the non-call sub-audio data sent by the electronic device, and the fourth transmission channel is used to transmit the encoded audio corresponding to the call sub-audio data sent by the electronic device.

[0202] The third transmission channel can include a plurality of receiving units, each of which is used to receive one type of encoded audio and simultaneously transmit the one type of encoded audio to a corresponding decoder. The non-call sub-audio data can include media sub-audio data, navigation sub-audio data, and mixed sub-audio data.

[0203] In this implementation, a separate transmission channel is established for the encoded audio corresponding to the call sub-audio data, so that the encoded audio corresponding to the call sub-audio data is not disturbed during transmission, effectively improving the call quality.

[0204] The application framework layer creates a corresponding number of decoders for the encoded audio, or in other words, the application framework layer creates a corresponding decoder for each type of encoded audio, so that the created decoder matches the audio type of each encoded audio. It should be understood that the audio type of the encoded audio is the audio type corresponding to the sub-audio data encoded into the encoded audio.

[0205] The decoder created by the application framework layer is used to decode the encoded audio transmitted into the decoder to obtain decoded audio. Then, the decoded audio is sent to the audio framework of the vehicle-mounted audio device.

[0206] Optionally, in a possible implementation, if the audio framework of the electronic device does not perform mixing processing on the received audio data, so that the electronic device side finally creates a plurality of encoders, such as the first, second, third, fourth, fifth, and sixth encoders described above.

[0207] Correspondingly, the application framework layer of the vehicle-mounted audio device creates a first decoder for the encoded audio corresponding to the media sub-audio data, the first decoder being used to decode the encoded audio corresponding to the media sub-audio data to obtain decoded audio corresponding to the media sub-audio data; a second decoder for the encoded audio corresponding to the navigation sub-audio data, the second decoder being used to decode the encoded audio corresponding to the navigation sub-audio data to obtain decoded audio corresponding to the navigation sub-audio data; a third decoder for the encoded audio corresponding to the call sub-audio data, the third decoder being used to decode the encoded audio corresponding to the call sub-audio data to obtain decoded audio corresponding to the call sub-audio data; a fourth decoder for the encoded audio corresponding to the notification sub-audio data, the fourth decoder being used to decode the encoded audio corresponding to the notification sub-audio data to obtain decoded audio corresponding to the notification sub-audio data; a fifth decoder for the encoded audio corresponding to the warning sub-audio data, the fifth decoder being used to decode the encoded audio corresponding to the warning sub-audio data to obtain decoded audio corresponding to the warning sub-audio data; and a sixth decoder for the encoded audio corresponding to the ringtone sub-audio data, the sixth decoder being used to decode the encoded audio corresponding to the ringtone sub-audio data to obtain decoded audio corresponding to the ringtone sub-audio data.

[0208] Optionally, in another possible implementation manner, if the audio framework of the electronic device performs mixing processing on the audio data of other audio types, the electronic device side finally creates a first encoder, a second encoder, a third encoder, and a call encoder. Correspondingly, the application framework layer of the in-vehicle audio device creates a corresponding decoder for the encoded audio transmitted by each encoder.

[0209] For example, the electronic device side creates a third encoder for the mixed sub-audio data, the application framework layer of the in-vehicle audio device creates a third decoder for the encoded audio corresponding to the mixed sub-audio data, the third decoder is used for decoding processing on the encoded audio corresponding to the mixed sub-audio data to obtain decoded audio corresponding to the mixed sub-audio data; the electronic device side creates a call encoder for the call sub-audio data, and the application framework layer of the in-vehicle audio device creates a call decoder for the encoded audio corresponding to the call sub-audio data, the call decoder is used for decoding processing on the encoded audio corresponding to the call sub-audio data to obtain decoded audio corresponding to the call sub-audio data.

[0210] In this implementation manner, a corresponding decoder is created for the encoded audio of different audio types, so that a decoder of one type only needs to decode the encoded audio of one audio type, effectively avoiding the occurrence of audio lag phenomenon, thereby improving the fluency of the finally played audio, and further improving the user experience. The audio lag phenomenon is caused by a decoder of one type decoding the encoded audio of multiple audio types, resulting in an increase in the waiting time for decoding each encoded audio.

[0211] Optionally, when creating the decoder, the device virtualization service identifies the created decoder according to the audio type of the received encoded audio, which is beneficial to distinguishing the created decoders and is also beneficial to subsequent audio playing module for separately adjusting and playing the decoded audio transmitted by each decoder.

[0212] It should be understood that the manner of identifying the created decoder is not limited. For example, different decoder names can be used for identification, such as a first decoder, a second decoder, a third decoder, a call decoder, and the like. For another example, a route 1 decoder, a route 2 decoder, a route 3 decoder, and the like.

[0213] Optionally, in one possible implementation manner, the device virtualization service can create a decoder, which can be a corresponding decoder pre-created for each type of encoded audio and stored in the device virtualization service for a long time, so as to facilitate subsequent quick decoding processing on each type of encoded audio by using the created decoder.

[0214] Optionally, in another possible implementation, a period starts from the electronic device establishing a connection with the in-vehicle audio device and ends when the electronic device disconnects from the in-vehicle audio device. In each period, the device virtualization service creates a corresponding decoder for each type of encoded audio. For example, the device virtualization service creates a decoder corresponding to a certain type of encoded audio when it first receives the encoded audio from the electronic device. Then, in the period, the decoder can be directly used to decode the encoded audio of the type.

[0215] Optionally, when it is detected that the electronic device disconnects from the in-vehicle audio device, the device virtualization service destroys the created decoder. This implementation destroys the created decoder at the end of a period, effectively avoiding the decoder occupying system resources for a long time, improving resource utilization, and improving the performance of the in-vehicle audio device.

[0216] It can be understood that if the device virtualization service does not receive any encoded audio of a certain type from the electronic device in a period, the device virtualization service can not create a decoder corresponding to the type of audio in the period. This implementation creates a corresponding decoder when needed, which can effectively reduce resource consumption and improve resource utilization.

[0217] The audio playing module is an interface encapsulated by the in-vehicle audio device, which is used to temporarily store each decoded audio and send each decoded audio to the audio framework. For example, each decoder sends the decoded audio obtained after decoding to the audio playing module, and the audio playing module temporarily stores each decoded audio. Every interval of a preset time length, the audio playing module sends the temporarily stored decoded audio to the audio framework.

[0218] It can be understood that the "preset time length" can be set and adjusted according to actual conditions. For example, the preset time length can be 10 milliseconds, 20 milliseconds, 30 milliseconds, etc.

[0219] The audio framework obtains a playing strategy, and after receiving each decoded audio sent by the audio playing module, plays and / or adjusts the decoded audio according to the playing strategy. The playing strategy can include independently adjusting the playing volume of the decoded audio, and / or playing the decoded audio corresponding to a specified speaker through the specified speaker.

[0220] For example, the playing strategy specifically includes increasing the playing volume of media audio (such as music audio). The audio framework increases the playing volume of the decoded audio corresponding to the media sub-audio data, so that the volume of the media audio played by the final speaker is increased.

[0221] For example, the playing strategy specifically includes playing navigation audio through the driver seat speaker, and the audio framework specifies the driver seat speaker to play the decoded audio corresponding to the navigation sub-audio data, and finally the decoded audio corresponding to the navigation sub-audio data is played by the driver seat speaker.

[0222] Optionally, in a possible implementation, the car machine application of the in-vehicle audio device generates a playing strategy according to the needs of the driver or the passenger, and sends the playing strategy to the audio framework.

[0223] In the audio processing method provided by the embodiments of the present application, the in-vehicle audio device receives not mixed audio data but individual encoded audios, and the decoded audios obtained after decoding the encoded audios are also individual audios (or each decoded audio obtained after decoding is an audio of a different audio type). Therefore, the in-vehicle audio device can independently play, adjust, and the like each audio, for example, can respectively adjust the volume of music, navigation sound, and call sound, and can specify different speakers to play different types of audios. Thus, the audio processing method provided by the present application can realize many optimization scenarios for improving user experience between the electronic device and the vehicle, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the driver and the passenger of the vehicle.

[0224] The audio processing method provided by the embodiments of the present application is described above in combination with the system architecture, and is described below in combination with a flowchart.

[0225] Please refer to Figure 6 , Figure 6 A flowchart of an audio processing method provided by an embodiment of the present application is shown. The method includes:

[0226] S101, sending an audio shunting instruction and a cross-device audio stream transfer capability starting instruction.

[0227] The application program layer of the electronic device can include a car machine application. The car machine application can perceive that a communication connection is established between the electronic device and the in-vehicle audio device, or the connection is disconnected, and when it is perceived that the communication connection is established between the electronic device and the in-vehicle audio device, the car machine application sends an audio shunting instruction to the audio framework of the application program framework layer, and sends a cross-device audio stream transfer capability starting instruction to the device virtualization service of the application program framework layer.

[0228] The audio split instruction is used to instruct the audio framework to process audio data generated by multiple applications into sub-audio data. In a simple understanding, the audio split instruction is used to instruct the audio framework to not perform mixing processing on media audio data, navigation audio data and call audio data, and to perform mixing processing on other audio data (such as notification audio data, warning audio data, bell audio data, etc.).

[0229] The cross-device audio stream transfer capability start instruction is used to instruct the device virtualization service to create a corresponding encoder for sub-audio data of different audio types, and to perform encoding processing on the sub-audio data, thereby obtaining encoded audio.

[0230] The sub-audio data includes at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data, and the mixed sub-audio data is obtained by performing mixing processing on other sub-audio data except the media sub-audio data, the navigation sub-audio data, and the call sub-audio data.

[0231] S102, start the audio split function and the cross-device audio stream transfer function.

[0232] After the audio framework in the electronic device receives the audio split instruction, the audio split function is started.

[0233] The audio split function refers to the function of the audio framework processing audio data generated by multiple applications into sub-audio data. In a simple understanding, the audio split function is the function of the audio framework not performing mixing processing on media audio data, navigation audio data and call audio data, and performing mixing processing on other audio data (such as notification audio data, warning audio data, bell audio data, etc.).

[0234] It can be understood that after the audio split function is started, the audio framework does not perform mixing processing on media audio data, navigation audio data and call audio data when receiving media audio data, navigation audio data and call audio data, but performs mixing processing on notification audio data, warning audio data, bell audio data, etc. when receiving notification audio data, warning audio data, bell audio data, etc.

[0235] Optionally, in a possible implementation manner, the audio split function can also refer to the function of the audio framework not performing mixing processing on any audio data, which is beneficial to subsequent independent adjustment and playback of any audio type of decoded audio by the vehicle-mounted audio device.

[0236] After the device virtualization service in the electronic device receives the cross-device audio stream transfer capability start instruction, the cross-device audio stream transfer function is started.

[0237] The cross-device audio stream transfer function refers to a function of creating an encoder by the device virtualization service and performing encoding processing by using the encoder. Specifically, the cross-device audio stream transfer function refers to a function of creating an encoder corresponding to sub-audio data of different audio types by the device virtualization service, and performing encoding processing on sub-audio data matched with the encoder by using the encoder.

[0238] S103, the user triggers an input operation.

[0239] Exemplarily, a plurality of application programs are pre-installed in the electronic device. In the embodiments of the present application, the plurality of application programs can be one or more of media application programs, navigation application programs, call application programs, shopping application programs, tool application programs, other application programs, etc.

[0240] In the embodiments of the present application, taking the plurality of application programs as media applications (such as music applications), navigation applications (such as map applications), call applications (such as telephone applications), and other applications (such as short message applications) as examples for description.

[0241] It is worth noting that in the embodiments of the present application, no matter which application program, audio data can be generated in the use process.

[0242] The input operation can be a click operation. For example, the user performs a click operation on the icon corresponding to the application program in the electronic device.

[0243] Alternatively, in the embodiments of the present application, the input operation can also be an operation of starting the application program by voice indication, the input operation can also be an operation of starting the application program by gesture indication, the input operation can also be an operation of starting the application program by eye indication, etc., and the present application does not make any limitation on this.

[0244] Exemplarily, after the electronic device detects the input operation of the user on the icon corresponding to the application program, the electronic device starts and runs the application program in response to the input operation triggered by the user. For example, the user can click the icon of the music application to instruct the electronic device to start and run the music application. Then, the music application starts playing music.

[0245] S104, receiving audio data generated by the plurality of application programs.

[0246] Exemplarily, different application programs generate different types of audio data when running. The audio data can include media audio data, navigation audio data, call audio data, notification audio data, warning audio data, ringtone audio data, prompt audio data, etc.

[0247] For example, the media application can generate media audio data, the navigation application can generate navigation audio data, the call application can generate call audio data, and other applications can generate notification audio data, warning audio data, ringtone audio data, prompt audio data, etc.

[0248] The media audio data can include, but is not limited to, audio data generated by various music applications, various video applications, various short video applications, various social applications, various game applications, and various information applications.

[0249] The navigation audio data can include, but is not limited to, audio data generated by various map applications, various travel applications, various taxi-hailing applications, and various positioning applications.

[0250] The call audio data can include, but is not limited to, audio data generated by telephone applications and various network telephone applications.

[0251] After the multiple applications in the application layer generate audio data, the audio data is sent to the audio framework in the application framework layer, and the audio framework receives the audio data generated by the multiple applications. It should be understood that the number and type of actually generated audio data are related to the applications used by the user, and are not limited.

[0252] S105, processing the audio data into sub-audio data.

[0253] In the related art, after the audio framework receives the audio data generated by the multiple applications, it will perform mixing processing on all the received audio data regardless of the audio type of the received audio data, and finally send the mixed data to the vehicle-mounted audio device. Since mixing processing is an irreversible process, the vehicle-mounted audio device cannot split the mixed data after receiving the mixed data, that is, it cannot split the mixed data into audio data of various types. Only unified playing, adjusting, etc. operations can be performed on the mixed data, and individual playing, adjusting, etc. operations cannot be performed on the audio data of various types, which results in that many optimization scenarios for improving user experience between the electronic device and the vehicle cannot be implemented, and the driving experience of the user is reduced.

[0254] In the audio processing method provided by the embodiments of the present application, when the audio framework receives the audio data generated by the multiple applications, the audio type of each audio data is identified.

[0255] The audio type can include a media audio type, a navigation audio type, a call audio type, and other audio types. The other audio types can include a notification audio type, a warning audio type, a ringtone audio type, a prompt audio type, etc.

[0256] In a possible implementation, the audio framework does not perform mixing processing on audio data of the media audio type, the navigation audio type, the call audio type, and other audio types.

[0257] In another possible implementation, the audio framework does not perform mixing processing on audio data of the media audio type, the navigation audio type, and the call audio type, and performs mixing processing on audio data of other audio types (for example, the notification audio type, the alert audio type, the ringtone audio type, and the prompt audio type).

[0258] For example, when identifying the audio types of the audio data, the system of the electronic device generally identifies audio data generated by each application in the application layer, and the audio framework identifies the audio types of the audio data according to audio type identifiers carried in the audio data.

[0259] It should be understood that the audio types of the audio data are consistent with the audio types of the sub-audio data. Therefore, at least two of the media sub-audio data, the navigation sub-audio data, the call sub-audio data, and other sub-audio data can also be identified from the audio data according to different audio type identifiers carried in the audio data.

[0260] For example, after determining the audio types of the media audio data, the navigation audio data, and the call audio data, the audio framework determines the media audio data as the media sub-audio data, determines the navigation audio data as the navigation sub-audio data, and determines the call audio data as the call sub-audio data.

[0261] For example, after determining the audio types of the media audio data, the navigation audio data, and the call audio data, the audio framework determines the media audio data as the media sub-audio data, determines the navigation audio data as the navigation sub-audio data, and determines the call audio data as the call sub-audio data.

[0262] For example, after determining the audio types of the notification audio data, the alert audio data, the ringtone audio data, and the prompt audio data, the audio framework determines the notification audio data as the notification sub-audio data, determines the alert audio data as the alert sub-audio data, determines the ringtone audio data as the ringtone sub-audio data, and determines the prompt audio data as the prompt sub-audio data. The notification sub-audio data, the alert sub-audio data, the ringtone sub-audio data, and the prompt sub-audio data are subjected to mixing processing to obtain mixed sub-audio data.

[0263] S106, forwarding the sub-audio data through the audio route.

[0264] The audio processing method provided in the embodiments of the present application further includes creating an audio route through a virtual hardware abstraction layer.

[0265] Exemplarily, the virtual hardware abstraction layer creates a corresponding number of audio routes for the sub-audio data, or in other words, the virtual hardware abstraction layer creates a corresponding audio route for each sub-audio data of each audio type, so that the created audio routes match the audio types of the sub-audio data.

[0266] Optionally, in a possible implementation, the audio framework does not perform mixing processing on the received audio data, and the virtual hardware abstraction layer creates an audio route corresponding to each sub-audio data.

[0267] Optionally, in another possible implementation, the audio framework performs mixing processing on the audio data of other audio types, and the virtual hardware abstraction layer creates a first audio route for the media sub-audio data, a second audio route for the navigation sub-audio data, a third audio route for the mixing sub-audio data, and a call audio route for the call sub-audio data.

[0268] It can be understood that, in some implementations of the present application, the call audio data and the call sub-audio data represent the same information, only the names are different.

[0269] S107, forwarding the sub-audio data through the virtual audio service.

[0270] The device virtualization service can include a virtual audio service and a virtual call service. Exemplarily, the audio route of the virtual hardware abstraction layer transmits non-call sub-audio data to the virtual audio service of the device virtualization service, and transmits call sub-audio data to the virtual call service of the device virtualization service.

[0271] The non-call sub-audio data can include media sub-audio data, navigation sub-audio data, and mixing sub-audio data.

[0272] The virtual audio service transmits the non-call sub-audio data to a corresponding encoder, and the virtual call service transmits the call sub-audio data to a corresponding call encoder.

[0273] The audio processing method provided by the embodiment of the present application further includes creating an encoder through a device virtualization service in an application framework layer.

[0274] Exemplarily, the device virtualization service in the application framework layer creates a corresponding number of encoders for the sub-audio data, or in other words, the device virtualization service in the application framework layer creates a corresponding encoder for each sub-audio data of each audio type, so that the created encoders match the audio types of the sub-audio data.

[0275] S108, performing encoding processing on the sub-audio data and sending encoded audio.

[0276] In the embodiments of the present application, the encoding processing refers to compressing the sub-audio data, so as to reduce the data amount of the sub-audio data, and facilitate subsequent transmission of the encoded audio to reduce the bandwidth.

[0277] Optionally, the encoding processing can also be converting the sub-audio data into network signals.

[0278] For example, the sub-audio data is encoded by the created encoders to obtain the encoded audio. Then, the encoded audio is transmitted to the transmission channel.

[0279] S109, transmitting the encoded audio through the transmission channel.

[0280] The device virtualization service includes a preset transmission channel for transmitting the encoded audio obtained by the encoding processing of each encoder to the in-vehicle audio device. For example, the transmission channel can include a plurality of sending units, each of which is configured to transmit one type of encoded audio.

[0281] In this implementation, the resource consumption is reduced by transmitting each encoded audio through one transmission channel, while ensuring the performance, thereby effectively saving the resources of the electronic device.

[0282] Optionally, in a possible implementation, different transmission channels are established for encoded audios of different audio types, and each transmission channel is configured to transmit encoded audio of one type of audio. In this implementation, encoded audios of multiple different audio types can be transmitted at the same time, thereby improving the transmission rate, and each transmission channel is configured to transmit encoded audio of one type of audio, thereby effectively avoiding interference between the encoded audios in the transmission process.

[0283] Optionally, in a possible implementation, the transmission channel can include a first transmission channel and a second transmission channel, the first transmission channel is configured to transmit encoded audio corresponding to non-conversation sub-audio data, and the second transmission channel is configured to transmit encoded audio corresponding to conversation sub-audio data.

[0284] The first transmission channel can include a plurality of sending units, each of which is configured to transmit one type of encoded audio. The non-conversation sub-audio data can include media sub-audio data, navigation sub-audio data, and mixed sub-audio data.

[0285] In this implementation, a separate transmission channel is established for the conversation sub-audio data, so as to ensure that the encoded audio corresponding to the conversation sub-audio data is not disturbed in the transmission process, thereby effectively improving the conversation quality.

[0286] Optionally, when sending the encoded audio through the transmission channel, a data header can be added to each encoded audio, which is used to identify the audio type of the encoded audio (i.e. the audio type of the sub-audio data corresponding to the encoded audio), so that when the vehicle-mounted audio device receives the encoded audio, it can quickly determine the audio type of the encoded audio by identifying the data header of the encoded audio, and then quickly send the encoded audio to the corresponding decoder.

[0287] The audio processing method provided by the embodiments of the present application is applied to an application framework layer of an electronic device connected with a vehicle-mounted audio device. The application framework layer of the electronic device receives audio data generated by multiple application programs, processes the audio data into sub-audio data, encodes each sub-audio data through a corresponding number of encoders created for the sub-audio data, obtains encoded audio, and sends the encoded audio to the vehicle-mounted audio device through a preset transmission channel.

[0288] Since the data contained in the sub-audio data is a single track data (or each data contained in the sub-audio data is different type of audio data), after the single track data is encoded and sent to the vehicle-mounted audio device, the vehicle-mounted audio device obtains a single audio (or each decoded audio obtained is different type of audio) by decoding the encoded audio through a decoder created for the encoded audio. Therefore, the vehicle-mounted audio device can independently play, adjust, etc. each audio, such as adjusting the volume of music, navigation sound and call sound respectively, and playing different type of audio through different speaker, etc. Thus, the audio processing method provided by the present application can realize many optimization scenarios between the electronic device and the vehicle for improving user experience, meet the personalized needs of vehicle audio playing, and improve the driving and riding experience of the driver and passengers of the vehicle.

[0289] Please refer to Figure 7 , Figure 7 Another flowchart of an audio processing method is shown in the embodiments of the present application. The method comprises:

[0290] S201, send an audio shunting instruction and a cross-device audio stream transfer capability start instruction.

[0291] S202, start an audio shunting function and a cross-device audio stream transfer function.

[0292] S203, a user triggers an input operation.

[0293] S204, receive audio data generated by multiple application programs.

[0294] S205, processing the audio data into sub-audio data.

[0295] S206, forwarding the sub-audio data through an audio routing.

[0296] S207, forwarding the sub-audio data through a virtual audio service.

[0297] S208, encoding processing the sub-audio data, and sending encoded audio.

[0298] S209, sending the encoded audio through a transmission channel.

[0299] The specific content of steps S201 to S209 can refer to the method described in the foregoing steps S101 to S109, which will not be described here.

[0300] S210, receiving and sending the encoded audio.

[0301] The application framework layer of the in-vehicle audio device can include a device virtualization service and an audio framework. The device virtualization service can include a transmission channel, a decoder, and an audio playback module.

[0302] For example, the transmission channel of the electronic device sends each encoded audio to the transmission channel of the in-vehicle audio device. After receiving each encoded audio, the transmission channel of the in-vehicle audio device transmits each encoded audio to the corresponding decoder.

[0303] S211, decoding processing the encoded audio to obtain decoded audio.

[0304] In the embodiment of the application, the decoding processing refers to decompressing the encoded audio, which can restore the data volume of the sub-audio data and facilitate subsequent smooth playback of the decoded audio.

[0305] Optionally, the decoding processing can also be converting a network signal into an audio signal.

[0306] For example, the device virtualization service of the in-vehicle audio device creates a corresponding number of decoders for the encoded audio. The created decoders decode the encoded audio transmitted to the decoders to obtain decoded audio. Then, the decoded audio is sent to the audio playback module of the in-vehicle audio device.

[0307] S212, sending the decoded audio.

[0308] For example, the audio playback module temporarily stores each decoded audio sent by the decoder. Every interval of a preset time length, the temporarily stored each decoded audio is sent to the audio framework.

[0309] S213, playing the decoded audio according to the playing policy.

[0310] For example, the playing policy specifically includes increasing the playing volume of the navigation audio and decreasing the playing volume of the music audio at the intersection. For another example, the playing policy specifically includes playing the navigation audio through the driver seat speaker and the co-driver seat speaker. For another example, the playing policy specifically includes playing the music audio through all the speakers in the vehicle, and the like.

[0311] For example, the playing policy specifically includes increasing the playing volume of the navigation audio and decreasing the playing volume of the music audio at the intersection. For another example, the playing policy specifically includes playing the navigation audio through the driver seat speaker and the co-driver seat speaker. For another example, the playing policy specifically includes playing the music audio through all the speakers in the vehicle, and the like.

[0312] For example, the playing policy specifically includes increasing the playing volume of the navigation audio and decreasing the playing volume of the music audio at the intersection. For another example, the playing policy specifically includes playing the navigation audio through the driver seat speaker and the co-driver seat speaker. For another example, the playing policy specifically includes playing the music audio through all the speakers in the vehicle, and the like.

[0313] In the audio processing method provided by the embodiments of the present application, the vehicle-mounted audio device receives not the mixed audio data but the individual encoded audios, and the decoded audios obtained after the decoding processing of the encoded audios are also individual audios (or each decoded audio obtained after the decoding is an audio of different audio type). Therefore, the vehicle-mounted audio device can independently perform the playing, adjusting and the like of each audio, for example, can respectively adjust the volume of the music, the navigation sound and the call sound, can specify different speakers to play different types of audios, and the like. Thus, the audio processing method provided by the present application can realize many optimization scenarios for improving the user experience between the electronic device and the vehicle, can meet the personalized requirements of the vehicle audio playing, and improves the driving and riding experience of the driver and the passenger of the vehicle.

[0314] Please refer to Figure 8 , Figure 8 Another system structure block diagram shown in the embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, the electronic device can include an audio framework, a virtual hardware abstraction layer and a device virtualization service, and the vehicle-mounted audio device can include the device virtualization service and the audio framework. Figure 8 ​

[0315] In the embodiments of the present application, the application program is taken as an example of a music application and a navigation application. The user performs input operations on the music application and the navigation application, the music application generates music audio data, the navigation application generates navigation audio data, and the audio framework identifies the audio types of the music audio data and the navigation audio data, and then processes the music audio data and the navigation audio data into music sub-audio data and navigation sub-audio data, respectively. The audio framework sends the music sub-audio data and the navigation sub-audio data to the virtual hardware abstraction layer, which forwards the music sub-audio data and the navigation sub-audio data to the device virtualization service through the audio routes created for the music sub-audio data and the navigation sub-audio data, respectively.

[0316] The device virtualization service encodes the music sub-audio data through a first encoder created for the music sub-audio data to obtain first encoded audio, and sends the first encoded audio to the transmission channel. The device virtualization service encodes the navigation sub-audio data through a second encoder created for the navigation sub-audio data to obtain second encoded audio, and sends the second encoded audio to the transmission channel.

[0317] In this implementation, the sub-audio data of different audio types is encoded by different encoders, which can improve the encoding efficiency and effectively avoid encoding lag, so that the audio played on the in-vehicle audio device is smoother.

[0318] The first encoded audio and the second encoded audio are sent to the transmission channel of the in-vehicle audio device through the transmission channel in the device virtualization service. In this implementation, the transmission channel transmits encoded audio of different audio types, which realizes multiplexing of the transmission channel, reduces resource consumption under the premise of ensuring performance, and effectively saves the resources of the electronic device.

[0319] The transmission channel of the in-vehicle audio device receives the first encoded audio and the second encoded audio sent by the electronic device, and sends the first encoded audio to the first decoder and the second encoded audio to the second decoder. The first decoder is a decoder created by the device virtualization service of the in-vehicle audio device for media audio type encoded audio, and the second decoder is a decoder created by the device virtualization service of the in-vehicle audio device for navigation audio type encoded audio.

[0320] In this implementation, the transmission channel of the in-vehicle audio device transmits encoded audio of different audio types, which realizes multiplexing of the transmission channel, reduces resource consumption under the premise of ensuring performance, and effectively saves the resources of the in-vehicle audio device.

[0321] The first encoded audio is decoded by a first decoder to obtain first decoded audio, and the first decoded audio is sent to an audio framework of the vehicle-mounted audio device.

[0322] In this implementation, the encoded audio of different audio types is decoded by different decoders, effectively avoiding the occurrence of audio lag, thereby improving the fluency of the finally played audio, and further improving the user experience.

[0323] The audio framework of the vehicle-mounted audio device plays the first decoded audio and the second decoded audio according to a playing strategy. For example, the first decoded audio is played by a first player (such as a sub-driver seat loudspeaker), and the second decoded audio is played by a first player (such as a driver seat loudspeaker).

[0324] Since the vehicle-mounted audio device receives not mixed audio data but individual encoded audio, the decoded audio obtained by decoding the encoded audio is also individual audio (or each decoded audio obtained by decoding is audio of a different audio type). Therefore, the vehicle-mounted audio device can independently play, adjust, etc. each audio, for example, can adjust the volume of music, navigation sound, and call sound respectively, can specify different loudspeakers to play different types of audio, etc. Thus, the audio processing method provided in this application can realize many optimization scenarios between the electronic device and the vehicle for improving user experience, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the driver and passengers of the vehicle.

[0325] Compared with the related art, the application is different in the way of transmitting data and the design of audio routing, realizes multiplexing of the transmission channel, reduces resource consumption under the premise of ensuring performance, effectively saves the resources of the electronic device and the vehicle-mounted audio device, and improves the interactive experience between the electronic device and the vehicle.

[0326] The audio processing method provided in this application is described below mainly from the electronic device side. Please refer to Figure 9 , Figure 9 is another flowchart of an audio processing method provided in this application. The method comprises:

[0327] S301, receiving audio data generated by a plurality of application programs.

[0328] S302, processing the audio data into sub-audio data.

[0329] S303, encode the sub-audio data through the corresponding number of encoders created for the sub-audio data to obtain encoded audio.

[0330] S304, send the encoded audio to the vehicle-mounted audio device through the preset transmission channel.

[0331] The specific content of steps S301 to S304 can be referred to the foregoing description, which will not be repeated here.

[0332] The audio processing method provided by the embodiment of the application is applied to an application framework layer of an electronic device connected with a vehicle-mounted audio device. The application framework layer of the electronic device receives audio data generated by a plurality of applications, processes the audio data into sub-audio data, encodes each sub-audio data through a corresponding number of encoders created for the sub-audio data to obtain encoded audio, and sends the encoded audio to the vehicle-mounted audio device through a preset transmission channel.

[0333] Since the data contained in the sub-audio data is a single audio track data (or each data contained in the sub-audio data is different type of audio data), after the single audio track data is encoded and sent to the vehicle-mounted audio device, the vehicle-mounted audio device obtains a single audio (or each decoded audio obtained after decoding the encoded audio is different type of audio) through a decoder created for the encoded audio. Therefore, the vehicle-mounted audio device can independently play, adjust, etc. each audio, for example, can adjust the volume of music, navigation sound and call sound respectively, and can specify different speakers to play different types of audio. Thus, the audio processing method provided by the application can realize many optimization scenarios between the electronic device and the vehicle for improving user experience, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the driver and passengers of the vehicle.

[0334] The audio processing method provided by the application is described below mainly from the vehicle-mounted audio device side. Please refer to Figure 10 , Figure 10 Another flowchart of an audio processing method provided by an embodiment of the application is shown. The method comprises:

[0335] S401, receive encoded audio sent through a preset transmission channel.

[0336] S402, decode the encoded audio through a decoder created for the encoded audio to obtain decoded audio.

[0337] S403, playing the decoded audio according to the playing policy.

[0338] The specific content of steps S401 to S403 can be referred to the foregoing description, which will not be repeated here.

[0339] The audio processing method provided in the embodiments of the present application can meet the personalized needs of audio playing of electronic devices and improve the driving and riding experience of vehicle drivers and passengers.

[0340] Optionally, the audio processing method provided in the embodiments of the present application can also be applied to an interactive scenario of cross-device audio stream transfer between electronic devices. For example, the interactive scenario between a mobile phone and a tablet computer, a mobile phone and a smart screen, a mobile phone and a computer, a tablet computer and a smart screen, a tablet computer and a computer, etc.

[0341] For example, the mobile phone and the tablet computer establish a communication connection through Bluetooth, and the mobile phone and the tablet computer perform audio data transmission and audio data processing. For specific how to perform audio data transmission and audio data processing, the description in steps S201 to S213 can be referred to, which will not be repeated here.

[0342] It is worth noting that the tablet computer itself has a loudspeaker and can be connected with a wired earphone or a wireless earphone as a loudspeaker. The playing policy corresponding to the tablet computer can be to independently adjust the volume of the decoded audio played by different loudspeakers. For example, when the call audio is played through the wireless earphone, the call volume is increased.

[0343] The playing policy corresponding to the tablet computer can also be to play the decoded audio corresponding to the specified loudspeaker through the specified loudspeaker. For example, the call audio is played through the wireless earphone, and the music audio is played through the loudspeaker of the tablet computer itself.

[0344] The audio processing method provided in the embodiments of the present application can meet the personalized needs of audio playing of electronic devices and improve the driving and riding experience of vehicle drivers and passengers.

[0345] In the foregoing audio processing method, one transmission channel is multiplexed, that is, the electronic device end sends the encoded audios of different audio types to the vehicle-mounted audio device end through one transmission channel. In this case, one encoded audio is usually sent through the transmission channel every time an encoded audio is obtained, resulting in low bandwidth utilization. In order to improve the bandwidth utilization, the present application further provides an audio transmission method, which can effectively improve the bandwidth utilization and improve the transmission speed of the encoded audio.

[0346] The audio transmission method provided by the embodiment of the present application will be described below in combination with the software structure. Please refer to Figure 11 , Figure 11 The software structure block diagram of another electronic device shown in an exemplary embodiment of the present application.

[0347] Figure 11 The software structure block diagram of the electronic device shown is based on Figure 4 The software structure block diagram of the electronic device shown is based on Figure 11 As shown, the device virtualization service of the electronic device can further include a packet combining module. The packet combining module will be mainly described below, and the other software structures can refer to the description of the foregoing Figure 4 .

[0348] Exemplarily, each encoder in the device virtualization service encodes the sub-audio data to obtain a plurality of encoded audios; the packet combining module performs packet combining processing on the plurality of encoded audios to obtain packet combining data; and the packet combining module sends the packet combining data to the transmission channel, and the transmission channel sends the packet combining data to the vehicle-mounted audio device.

[0349] The packet combining processing refers to splicing / combining the plurality of encoded audios, and the data obtained by splicing / combining is the packet combining data. It can be understood in a simple way that a plurality of data packets (each encoded audio corresponds to one data packet) are spliced / combined into one data packet (the packet combining data corresponds to one data packet).

[0350] If the plurality of encoded audios are not subjected to packet combining processing, one encoded audio is sent through the transmission channel every time an encoded audio is obtained, which will result in low bandwidth utilization. For example, the data amount of one encoded audio is 400 bytes, and then 400 bytes of data can be transmitted through the transmission channel at a time, and the next encoded audio needs to wait for the transmission of the previous encoded audio to be completed before being transmitted. However, in the audio transmission method provided by the present application, the plurality of encoded audios are subjected to packet combining processing every time, and then the packet combining data obtained by the packet combining processing is sent to the vehicle-mounted audio device through the transmission channel at a time, thereby increasing the data amount transmitted every time, improving the bandwidth utilization, and improving the transmission speed of the encoded audio.

[0351] Optionally, in one possible implementation, all the encoders (including the talk encoders) send the encoded audio obtained after respective encoding processing to the packetizing module; the packetizing module packetizes all the received encoded audio to obtain packetized data; and the packetizing module sends the packetized data to the transmission channel, and the transmission channel sends the packetized data to the vehicle-mounted audio device.

[0352] In this implementation, the packetizing module packetizes all the encoded audio of all audio types, greatly compresses the data volume of the encoded audio, enables more data volume of encoded audio to be transmitted at one time through the transmission channel, improves the bandwidth utilization rate, and increases the transmission speed of the encoded audio.

[0353] Optionally, in one possible implementation, all the encoders (including the talk encoders) send the encoded audio obtained after respective encoding processing to the packetizing module; the packetizing module packetizes all the received encoded audio to obtain packetized data; and the packetizing module sends the packetized data to the transmission channel, and the transmission channel sends the packetized data to the vehicle-mounted audio device.

[0354] In this implementation, a separate transmission channel is established for the encoded audio corresponding to the talk audio data, which can ensure that the encoded audio corresponding to the talk audio data is not disturbed in the transmission process, thereby improving the talk quality. Meanwhile, the packetizing module packetizes the encoded audio sent by the encoders other than the talk encoders, compresses the data volume of the encoded audio, enables more data volume of encoded audio to be transmitted at one time through the transmission channel, improves the bandwidth utilization rate, and increases the transmission speed of the encoded audio.

[0355] The packetizing process is described in detail below with reference to the accompanying drawings. Figure 12 Figure 12 A packetizing process diagram is shown in an exemplary embodiment of the present application.

[0356] Exemplarily, the packetizing module creates an input buffer (inBuffer) for each encoder, i.e., one inBuffer for one encoder, and the inBuffer is used to receive and store the encoded audio sent by the corresponding encoder. The respective inBuffers are created for different encoders, so that one buffer only needs to store encoded audio of one audio type, which can ensure that the reading, storage, etc. of encoded audio of different audio types are not disturbed.

[0357] For example, the packetizing module creates a first inBuffer for the first encoder and a second inBuffer for the second encoder. ​

[0358] In one example, the packing module creates an input buffer, which can be a corresponding input buffer created in advance for an encoder of different audio type and stored in the packing module for a long time, so as to facilitate the later quick reception and storage of encoded audio through the created input buffer.

[0359] In another example, a period starts from the establishment of a connection between the electronic device and the vehicle-mounted audio device and ends when the connection between the electronic device and the vehicle-mounted audio device is disconnected. In each period, the packing module creates a corresponding input buffer for each encoder created by the device virtualization service. For example, when the packing module initially receives encoded audio of a certain audio type sent by a certain encoder, the packing module creates an input buffer corresponding to the encoder. Thereafter, in the period, the packing module can directly receive and store encoded audio sent by the encoder corresponding to the input buffer through the input buffer.

[0360] Optionally, when it is detected that the electronic device is disconnected from the vehicle-mounted audio device, the packing module destroys the created input buffer. This implementation destroys the created input buffer at the end of a period, effectively avoiding long-term occupation of system resources by the input buffer, improving resource utilization, and improving the performance of the electronic device.

[0361] It can be understood that if the packing module does not receive encoded audio sent by an encoder of a certain audio type in a period, the packing module can not create an input buffer corresponding to the encoder in the period. This implementation creates a corresponding input buffer when needed, which can effectively reduce resource consumption and improve resource utilization.

[0362] Optionally, the packing module can also establish a thread for each encoder. The thread is used to improve the smoothness of the process corresponding to the thread and to ensure that the process corresponding to the thread is not disturbed by the processes corresponding to other threads.

[0363] For example, the packing module establishes a first thread for a first encoder and a second thread for a second encoder. The process corresponding to the first thread is that the first encoder sends (or writes) first encoded audio to a first input buffer, and the process corresponding to the second thread is that the second encoder sends (or writes) second encoded audio to a second input buffer. With the existence of the first thread and the second thread, the process of sending first encoded audio by the first encoder to the first input buffer does not interfere with the process of sending second encoded audio by the second encoder to the second input buffer, thereby improving the smoothness of sending encoded audio.

[0364] Optionally, the packing module can further comprise a packet sender. The packet sender is configured to read the encoded audios from the input buffers, splice / combine the read encoded audios to obtain the packing data, and send the packing data to the transmission channel.

[0365] It is worth mentioning that the packet sender works periodically, for example, every time a sending period (i.e. a preset period) arrives, the packet sender reads the encoded audios from the input buffers, reads all the encoded audios in the current input buffers, splices / combines the read encoded audios to obtain the packing data, and sends the packing data to the transmission channel. Then, the packet sender continues to wait for the arrival of the next sending period, and repeats the process of reading the encoded audios, splicing / combining the encoded audios, and sending the packing data until the electronic device is disconnected from the vehicle audio device.

[0366] It is worth mentioning that the packet sender works periodically, for example, every time a sending period (i.e. a preset period) arrives, the packet sender reads the encoded audios from the input buffers, reads all the encoded audios in the current input buffers, splices / combines the read encoded audios to obtain the packing data, and sends the packing data to the transmission channel. Then, the packet sender continues to wait for the arrival of the next sending period, and repeats the process of reading the encoded audios, splicing / combining the encoded audios, and sending the packing data until the electronic device is disconnected from the vehicle audio device.

[0367] It is worth mentioning that the packet sender works periodically, for example, every time a sending period (i.e. a preset period) arrives, the packet sender reads the encoded audios from the input buffers, reads all the encoded audios in the current input buffers, splices / combines the read encoded audios to obtain the packing data, and sends the packing data to the transmission channel. Then, the packet sender continues to wait for the arrival of the next sending period, and repeats the process of reading the encoded audios, splicing / combining the encoded audios, and sending the packing data until the electronic device is disconnected from the vehicle audio device.

[0368] It is worth mentioning that the packet sender works periodically, for example, every time a sending period (i.e. a preset period) arrives, the packet sender reads the encoded audios from the input buffers, reads all the encoded audios in the current input buffers, splices / combines the read encoded audios to obtain the packing data, and sends the packing data to the transmission channel. Then, the packet sender continues to wait for the arrival of the next sending period, and repeats the process of reading the encoded audios, splicing / combining the encoded audios, and sending the packing data until the electronic device is disconnected from the vehicle audio device.

[0369] Optionally, the packing module can further establish a thread for the packet sender, which is configured to improve the fluency of the process to be performed by the packet sender (e.g. the process of reading the encoded audios, splicing / combining the encoded audios, and sending the packing data) and ensure that the process to be performed by the packet sender is not disturbed by the processes corresponding to other threads. For example, the packing module establishes a third thread for the packet sender.

[0370] Optionally, in one possible implementation, the packet sender obtains the packing data after splicing / combining the read encoded audios. If the packing data is successfully obtained, the packet sender sends the packing data to the transmission channel. Then, the packet sender waits for a sending interval, or in other words, the packet sender continues to wait for the arrival of the next sending period.

[0371] If the combined package data is not obtained, the step of obtaining the combined package data is performed. It should be understood that the packet sender may not read the encoded audio in a certain sending period due to jitter of the encoder, network lag, etc., and thus the combined package data is not obtained.

[0372] In this implementation, the encoded audio in a sending period can be spliced / combined by the packet sender and sent to the vehicle-mounted audio device through the transmission channel, so that a large amount of encoded audio is sent in a sending period, the bandwidth utilization is improved, and the transmission speed of the encoded audio is improved.

[0373] Please refer to Figure 13 , Figure 13 The sending period diagram shown in an exemplary embodiment of the present application. As shown in Figure 13 , the sending period between two sending points is one sending period. For example, the sending period between the first sending point and the second sending point is one sending period, denoted as the first sending period; the sending period between the second sending point and the third sending point is one sending period, denoted as the second sending period; the sending period between the third sending point and the fourth sending point is one sending period, denoted as the third sending period; and the sending period between the fourth sending point and the fifth sending point is one sending period, denoted as the fourth sending period.

[0374] Exemplarily, three encoded audios are received in the first sending period, which are the encoded audio of the media audio type in the first row, the encoded audio of the navigation audio type in the second row, and the encoded audio of the mixed audio type in the third row. The three encoded audios are combined to obtain combined package data, and the combined package data is sent to the transmission channel at the second sending point. One encoded audio is received in the second sending period, which is combined to obtain combined package data, and the combined package data is sent to the transmission channel at the third sending point. Five encoded audios are received in the third sending period, which are combined to obtain combined package data, and the combined package data is sent to the transmission channel at the fourth sending point.

[0375] It should be understood that the encoded audios with the cross symbol received in the third sending period should have been received in the second sending period, and they may be received in the third sending period due to jitter of the encoder, network lag, etc.

[0376] It is worth mentioning that the data amount of the packaged data is not limited. The more the encoded audios received in a sending period, the larger the packaged data generated; correspondingly, the less the encoded audios received in a sending period, the smaller the packaged data generated. For example, the data amount of an encoded audio is 400 bytes, and three encoded audios are received in a sending period, and the data amount of the generated packaged data is 1200 bytes. For another example, the data amount of an encoded audio is 400 bytes, and five encoded audios are received in a sending period, and the data amount of the generated packaged data is 2000 bytes.

[0377] Please refer to Figure 14 , Figure 14 The packaged data shown in the example embodiment of the present application is shown in the figure.

[0378] In an example embodiment, the packaged processing of the multiple encoded audios to obtain the packaged data can include: setting a transmission protocol header, splicing / combining each encoded audio (each encoded audio is a data packet), and setting a data header for each encoded audio in the process of splicing / combining the encoded audios.

[0379] The transmission protocol header is used to indicate that the packaged data is data obtained by packaged processing, or that the packaged data is encapsulated data. After the vehicle-mounted audio device receives the packaged data, it can quickly determine that the packaged data is data obtained by packaged processing by identifying the transmission protocol header in the packaged data, so as to perform unpackaged processing on the packaged data.

[0380] The data header can include attribute information of the encoded audio. For example, the identification of the audio route corresponding to the encoded audio, the audio type corresponding to the encoded audio, the application corresponding to the encoded audio, the data packet length corresponding to the encoded audio, and other attribute information are stored in the data header. Optionally, a reserved field can also be stored in the data header, so as to facilitate subsequent addition of identification, notes and other information to the encoded audio through the reserved field, which is beneficial to improve the compatibility of the electronic device.

[0381] It is worth mentioning that the order of splicing / combining each encoded audio can be determined according to the order of reading each encoded audio. For example, the first encoded audio is read from the first input buffer in sequence, and the second encoded audio is read from the second input buffer. The transmission protocol header is set, the first data header is set for the first encoded audio, and the first data header and the first data packet (the data packet corresponding to the first encoded audio) are spliced / combined; the second data header is set for the second encoded audio, and the second data header and the second data packet (the data packet corresponding to the second encoded audio) are spliced / combined to obtain the packaged data. As shown in Figure 14 The packaged data includes the transmission protocol header, the first data header, the first data packet, the second data header, and the second data packet.

[0382] This implementation adds a data header to the encoded audio during the splicing / combining of encoded audio, making it easier for the in-vehicle audio equipment to quickly repackage the combined data based on the data header and quickly obtain the attribute information of each encoded audio.

[0383] In another example, multiple encoded audio files are combined into a single packet to obtain combined data. This may include setting a transport protocol header and sequentially concatenating / combining each encoded audio file (each encoded audio file corresponds to a data packet).

[0384] This implementation method directly splices / combines encoded audio without adding a data header to the encoded audio, which improves the speed of packet processing and thus the speed of generating packet data.

[0385] The audio transmission method provided in the embodiments of this application has been described above in conjunction with the software structure. The audio transmission method provided in the embodiments of this application will now be described below in conjunction with the system architecture. Please refer to... Figure 15 , Figure 15 This is another system architecture block diagram illustrated for an exemplary embodiment of this application. (See diagram below.) Figure 15 As shown, the system architecture block diagram is in Figure 5 Based on the corresponding system architecture diagram, a package merging module and a package sub-package module have been added. The following mainly describes the package sub-package module for in-vehicle audio equipment; other software structures can be found in the previous sections. Figure 5 as well as Figure 11 The description will not be repeated here.

[0386] For example, the transmission channel of the electronic device sends the bundled data to the transmission channel of the in-vehicle audio device. The transmission channel of the electronic device may include a sending unit, through which the bundled data is sent to the transmission channel of the in-vehicle audio device.

[0387] In this implementation, because multiple encoded audio files are packetized, the packetized data is ultimately sent to the transmission channel of the in-vehicle audio device, rather than sending the multiple encoded audio files directly to the transmission channel of the in-vehicle audio device. Therefore, the number of sending units is greatly reduced (for example, where three encoded audio files would normally require three sending units, only one sending unit is needed here), effectively improving the efficiency of sending packetized data and increasing the bandwidth utilization between the electronic device and the in-vehicle audio device.

[0388] Exemplarily, after the transmission channel of the in-vehicle audio device receives the packaged data, the packaged data is processed by a packet splitting module in the device virtualization service to obtain a plurality of encoded audios; each decoder in the device virtualization service decodes each encoded audio to obtain a plurality of decoded audios. Each decoder sends the decoded audio obtained after decoding to an audio playing module, and the audio playing module temporarily stores each decoded audio. Every interval of a preset time length, the audio playing module sends the temporarily stored decoded audios to the audio framework.

[0389] The audio framework obtains a playing strategy, and after receiving each decoded audio sent by the audio playing module, plays and / or adjusts the decoded audio according to the playing strategy. The playing strategy can include independently adjusting the playing volume of the decoded audio, and / or playing the decoded audio corresponding to a specified speaker through the specified speaker.

[0390] In this implementation, the packet splitting processing refers to splitting the packaged data into one or more encoded audios.

[0391] For the in-vehicle audio device, a large amount of data can be received at one time when the in-vehicle audio device receives the packaged data through the transmission channel, thereby improving the data transmission rate and improving the bandwidth utilization.

[0392] Optionally, in a possible implementation, if all the encoders (including the call encoder) send the encoded audios obtained after respective encoding processing to the packaging module at the electronic device end; the packaging module performs packaged processing on all the received encoded audios to obtain packaged data; and the packaging module sends the packaged data to the transmission channel, and the transmission channel sends the packaged data to the in-vehicle audio device. Correspondingly, the transmission channel of the in-vehicle audio device receives the packaged data in this scenario.

[0393] In this implementation, since the packaging module at the electronic device end performs packaged processing on encoded audios of all audio types, the data amount of the encoded audios is greatly compressed, so that more data amount of encoded audios can be transmitted at one time through the transmission channel of the electronic device, thereby enabling the transmission channel of the in-vehicle audio device to receive more data amount of encoded audios at one time, improving the bandwidth utilization between the electronic device and the in-vehicle audio device, and improving the transmission speed of the encoded audios.

[0394] Optionally, in one possible implementation, on the electronic device side, besides the call encoder, other encoders send their respective encoded audio data to the packet merging module; the packet merging module merges the received encoded audio data to obtain merged data; the packet merging module sends the merged data to the transmission channel, and the transmission channel sends the merged data to the in-vehicle audio device. Accordingly, the transmission channel of the in-vehicle audio device receives the merged data in this scenario. Alternatively, on the electronic device side, the call encoder sends the encoded audio data corresponding to the call audio data separately to the transmission channel, and the transmission channel separately sends the encoded audio data corresponding to the call audio data to the in-vehicle audio device. Accordingly, the transmission channel of the in-vehicle audio device receives the encoded audio data corresponding to the call audio data separately.

[0395] In this implementation, a separate transmission channel is established for the encoded audio corresponding to the call audio data. This ensures that the encoded audio is not interfered with during transmission, thereby improving call quality. Simultaneously, because the packet-merging module on the electronic device merges the encoded audio sent by encoders other than the call encoder, the data volume of the encoded audio is compressed. This allows the electronic device's transmission channel to transmit a larger amount of encoded audio data at once, enabling the in-vehicle audio device's transmission channel to receive a larger amount of encoded audio data simultaneously. This improves bandwidth utilization between the electronic device and the in-vehicle audio device, increasing the transmission speed of the encoded audio.

[0396] The subcontracting process is described in detail below with reference to the accompanying drawings. Please refer to them. Figure 16 , Figure 16 This is a schematic diagram illustrating subpackaging processing as an exemplary embodiment of this application.

[0397] For example, the packet module creates an input buffer for each decoder, i.e., one inBuffer per decoder, which is used to receive and store the encoded audio that is split from the packet data.

[0398] For example, the sub-packet module creates a first input buffer for the first decoder and a second input buffer for the second decoder.

[0399] In one example, the packet module creates input buffers, which can be pre-created for decoders of different audio types and stored in the packet module for a long time, so that encoded audio can be quickly received and stored through the created input buffers later.

[0400] In another example, a cycle starts from the electronic device establishing a connection with the in-vehicle audio device and ends when the electronic device disconnects from the in-vehicle audio device. In each cycle, the packet splitting module creates an input buffer for each decoder created by the device virtualization service. For example, when the packet splitting module splits the encoded audio of a certain audio type for the first time, the packet splitting module creates an input buffer for the decoder corresponding to the audio type. Then, in the cycle, the encoded audio corresponding to the input buffer can be directly received and stored through the input buffer.

[0401] Optionally, when it is detected that the electronic device disconnects from the in-vehicle audio device, the packet splitting module destroys the created input buffer. In this implementation, the created input buffer is destroyed at the end of a cycle, which effectively avoids long-term occupation of system resources by the input buffer, improves resource utilization, and improves the performance of the electronic device.

[0402] It can be understood that if the packet splitting module never splits the encoded audio of a certain audio type in a cycle, the packet splitting module can not create an input buffer for the decoder corresponding to the audio type in the cycle. In this implementation, the corresponding input buffer is created only when needed, which can effectively reduce resource consumption and improve resource utilization.

[0403] Optionally, the packet splitting module can also establish a thread for each decoder. The thread is used to improve the smoothness of the process corresponding to the thread and ensure that the process corresponding to the thread is not disturbed by the processes corresponding to other threads.

[0404] For example, the packet splitting module establishes a first thread for a first decoder and a second thread for a second decoder. The process corresponding to the first thread is that the first decoder reads the first encoded audio from the first input buffer, and the process corresponding to the second thread is that the second decoder reads the second encoded audio from the second input buffer. With the existence of the first thread and the second thread, the process of the first decoder reading the first encoded audio from the first input buffer is not disturbed by the process of the second decoder reading the second encoded audio from the second input buffer, and the smoothness of reading the encoded audio is improved.

[0405] Optionally, the packet splitting module can also establish a thread, such as a third thread, for splitting the packetized data. The third thread is used to ensure that the process of splitting the packetized data is not disturbed by the processes corresponding to other threads.

[0406] For example, after each decoder reads the encoded audio, the decoder decodes the encoded audio to obtain decoded audio. Each decoder sends the decoded audio obtained after decoding to the audio playback module. The audio playback module temporarily stores each decoded audio, and sends the temporarily stored decoded audio to the audio framework every preset time interval.

[0407] Optionally, the audio playing module can further include a plurality of playing interfaces, through which the audio framework can be accessed. Illustratively, each decoder sends the decoded audio obtained after decoding processing to the audio framework through a playing interface. For example, the first decoder sends the decoded audio obtained after decoding processing to the audio framework through a first playing interface; the second decoder sends the decoded audio obtained after decoding processing to the audio framework through a second playing interface.

[0408] The audio framework obtains a playing strategy, and after receiving each decoded audio sent by the audio playing module, plays and / or adjusts the decoded audio according to the playing strategy. In this way, the vehicle-mounted audio device can independently play, adjust, etc. each audio, can realize many optimization scenarios between the electronic device and the vehicle for improving user experience, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the vehicle driver and vehicle passengers.

[0409] In one example, the packetizing processing of the packetized data to obtain the encoded audio can include: identifying a transmission protocol header, parsing a data header corresponding to each encoded audio in the packetized data, and according to the data packet length of the encoded audio in each data header, sequentially splitting out each encoded audio (each encoded audio is one data packet).

[0410] Illustratively, after the vehicle-mounted audio device receives the packetized data, the transmission protocol header in the packetized data is identified, it is quickly determined that the packetized data is data obtained after packetizing processing, and thus the packetized data is parsed.

[0411] For each encoded audio, the attribute information in the data header is parsed to determine the identifier of the audio route corresponding to the encoded audio, the audio type corresponding to the encoded audio, the application program corresponding to the encoded audio, and the data packet length corresponding to the encoded audio, etc.

[0412] The encoded audio is then accurately extracted from the packetized data according to the data packet length corresponding to the encoded audio. For example, the end position of the data header is obtained, the start position of the encoded audio is determined according to the end position of the data header, and the encoded audio is extracted from the packetized data according to the start position of the encoded audio and the data packet length.

[0413] In this implementation, by parsing the data header in the packetized data, each encoded audio can be accurately and quickly parsed, which is conducive to subsequent decoding processing of the encoded audio by the vehicle-mounted audio device, so that each audio can be independently played, adjusted, etc.

[0414] The audio transmission method provided by the embodiments of the present application is described above in combination with the system architecture, and the audio transmission method provided by the embodiments of the present application is described below in combination with the flowchart.

[0415] Please refer to Figure 17 , Figure 17 A flowchart of an audio transmission method is shown in the embodiments of the present application. The method comprises:

[0416] S501, sending an audio splitting instruction and a cross-device audio stream transfer capability starting instruction.

[0417] S502, starting an audio splitting function and a cross-device audio stream transfer function.

[0418] S503, a user triggers an input operation.

[0419] S504, receiving audio data generated by multiple application programs.

[0420] S505, processing the audio data into sub-audio data.

[0421] S506, forwarding the sub-audio data through an audio routing.

[0422] S507, forwarding the sub-audio data through a virtual audio service.

[0423] S508, encoding processing the sub-audio data, and sending encoded audio.

[0424] S509, packetizing multiple encoded audios to obtain packetized data.

[0425] S510, sending the packetized data through a transmission channel.

[0426] The specific content of steps S501 to S510 can be referred to the foregoing description, which will not be repeated here.

[0427] The audio transmission method provided by the embodiments of the present application, on the one hand, since the data contained in the sub-audio data processed by the electronic device is a single audio track data (or each data contained in the sub-audio data is a different type of audio data), the single audio track data is encoded and sent to the vehicle-mounted audio device, and the vehicle-mounted audio device decodes the encoded audio through the decoder created for the encoded audio to obtain decoded audio which is also a single audio (or each decoded audio is a different type of audio). Therefore, the vehicle-mounted audio device can independently play, adjust, etc. each audio, for example, can adjust the volume of music, navigation sound and call sound respectively, can specify different speakers to play different types of audio, etc. Thus, many optimization scenarios for improving user experience between the electronic device and the vehicle can be realized, the individualized needs of vehicle audio playback can be met, and the driving and riding experience of the vehicle driver and passengers is improved.

[0428] On the other hand, the electronic device performs packet combining processing on the encoded audio, greatly compresses the data amount of the encoded audio, enables more data amount of encoded audio to be transmitted at one time through the transmission channel, improves the bandwidth utilization rate, and improves the transmission speed of the encoded audio.

[0429] Please refer to Figure 18 , Figure 18 A flowchart of another audio transmission method is shown in the embodiments of the present application. The method comprises:

[0430] S601, send audio splitting instruction and cross-device audio stream transfer capability start instruction.

[0431] S602, start audio splitting function and cross-device audio stream transfer function.

[0432] S603, the user triggers the input operation.

[0433] S604, receive audio data generated by multiple application programs.

[0434] S605, process the audio data into sub-audio data.

[0435] S606, forward the sub-audio data through the audio routing.

[0436] S607, forward the sub-audio data through the virtual audio service.

[0437] S608, encode the sub-audio data, and send the encoded audio.

[0438] S609, packet combining processing is performed on the encoded audio to obtain packet combining data.

[0439] S610, send the packet combining data through the transmission channel.

[0440] S611, receive and send the packet combining data.

[0441] S612, packet splitting processing is performed on the packet combining data to obtain multiple encoded audios.

[0442] S613, decode the multiple encoded audios to obtain multiple decoded audios.

[0443] S614, send the multiple decoded audios.

[0444] S615, play the multiple decoded audios according to the playing strategy.

[0445] The specific content of steps S601 to S615 can be referred to the foregoing description, which will not be repeated here.

[0446] In this implementation, on the one hand, since the multiple encoded audios are subjected to the packetizing processing, the packetized data is finally transmitted to the transmission channel of the vehicle-mounted audio device, instead of directly transmitting the multiple encoded audios to the transmission channel of the vehicle-mounted audio device. Therefore, the number of transmitting units is greatly reduced (for example, three transmitting units are originally required for transmitting three encoded audios, while only one transmitting unit is required here), which effectively improves the efficiency of transmitting the packetized data and improves the bandwidth utilization rate between the electronic device and the vehicle-mounted audio device.

[0447] On the other hand, since the vehicle-mounted audio device receives not the mixed audio data but the individual encoded audios, after the decoding processing of the encoded audios, the decoded audios obtained are also individual audios (or in other words, each decoded audio obtained by decoding is an audio of a different audio type). Therefore, the vehicle-mounted audio device can independently play, adjust, and the like each audio, for example, can respectively adjust the volume of the music, the navigation sound, and the call sound, and can specify different loudspeakers to play different types of audios. Thus, the audio processing method provided by the present application can realize many optimization scenarios between the electronic device and the vehicle for improving the user experience, can meet the personalized needs of vehicle audio playing, and improves the driving and riding experience of the driver and the passenger of the vehicle.

[0448] Next, the audio transmission method provided by the present application is described mainly from the perspective of the electronic device, please refer to Figure 19 , Figure 19 is another flowchart of an audio transmission method provided by an embodiment of the present application. The method comprises:

[0449] S701, obtaining multiple encoded audios.

[0450] S702, packetizing the multiple encoded audios to obtain packetized data.

[0451] S703, transmitting the packetized data to a vehicle-mounted audio device through a preset transmission channel.

[0452] The specific contents of steps S701 to S703 can be referred to the foregoing description, which will not be repeated here.

[0453] In this implementation, since the electronic device packetizes the encoded audios, the data amount of the encoded audios is greatly compressed, more data amount of encoded audios can be transmitted at one time through the transmission channel, the bandwidth utilization rate is improved, and the transmission speed of the encoded audios is improved.

[0454] Next, the audio transmission method provided by the present application is described mainly from the perspective of the vehicle-mounted audio device, please refer to Figure 20 , Figure 20A flowchart of another audio transmission method is shown in the embodiments of the present application. The method comprises:

[0455] S801, receiving the packet data sent through the preset transmission channel.

[0456] S802, packet processing the packet data to obtain a plurality of encoded audios.

[0457] The specific content of steps S801 to S802 can be referred to the foregoing description, which will not be repeated here.

[0458] In this implementation, since the packet processing is performed on the plurality of encoded audios, the packet data is finally sent to the transmission channel of the vehicle-mounted audio device, instead of directly sending the plurality of encoded audios to the transmission channel of the vehicle-mounted audio device. Therefore, the number of sending units is greatly reduced (for example, three sending units are originally required to send three encoded audios, while only one sending unit is required here), which effectively improves the efficiency of sending the packet data and improves the bandwidth utilization rate between the electronic device and the vehicle-mounted audio device.

[0459] Optionally, the audio transmission method provided by the embodiments of the present application can also be applied to the interactive scenarios of cross-device audio stream transfer between electronic devices. For example, the interactive scenarios of mobile phones and tablet computers, mobile phones and smart screens, mobile phones and computers, tablet computers and smart screens, tablet computers and computers, etc.

[0460] Exemplarily, the interactive scenario between the mobile phone and the tablet computer is taken as an example for description. For example, the mobile phone and the tablet computer establish a communication connection through Bluetooth, and the mobile phone and the tablet computer perform audio data transmission and audio data processing. For specific how to perform audio data transmission and audio data processing, reference can be made to the description in steps S601 to S614, which will not be repeated here.

[0461] The audio transmission method provided by the present application performs packet processing on the transmitted encoded audio, greatly compresses the data amount of the encoded audio, enables more data amount of encoded audio to be transmitted at one time through the transmission channel, improves the bandwidth utilization rate, improves the transmission speed of the encoded audio, and improves the user experience.

[0462] The hardware structure of the electronic device involved in the embodiments of the present application will be briefly introduced below in combination with the drawings.

[0463] Please refer to Figure 21 , Figure 21 The hardware structure of the electronic device shown in an exemplary embodiment of the present application is shown in the following figure.

[0464] As Figure 21As shown, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0465] It can be understood that the structures shown in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown, or the electronic device 100 can include a combination of some of the components shown, or the electronic device 100 can include sub-components of some of the components shown. Figure 21 It can be understood that the structures shown in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown, or the electronic device 100 can include a combination of some of the components shown, or the electronic device 100 can include sub-components of some of the components shown. Figure 21 It can be understood that the structures shown in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown, or the electronic device 100 can include a combination of some of the components shown, or the electronic device 100 can include sub-components of some of the components shown. Figure 21 It can be understood that the structures shown in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown, or the electronic device 100 can include a combination of some of the components shown, or the electronic device 100 can include sub-components of some of the components shown. Figure 21 It can be understood that the structures shown in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown, or the electronic device 100 can include a combination of some of the components shown, or the electronic device 100 can include sub-components of some of the components shown.

[0466] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc.

[0467] The different processing units can be independent devices or can be integrated in one or more processors. The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0468] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0469] In embodiments of the present application, the processor 110 can execute each step of the audio processing method and the audio transmission method. For example, the processor 110 can run the software code of the audio processing method and the audio transmission method provided in embodiments of the present application.

[0470] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI) interface, a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0471] It can be understood that the interface connection relationship between the modules shown in the embodiments is only illustrative and does not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0472] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0473] The wireless communication module 160 can provide a wireless communication solution applied on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 can be one or more devices integrated with at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be sent from the processor 110, perform frequency modulation, amplification, and convert the signals to electromagnetic wave radiation via the antenna 2.

[0474] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with the network and the vehicle audio device through the wireless communication technology.

[0475] The electronic device 100 can implement the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU can also be used to perform mathematical and pose calculations, for graphics rendering, etc. The processor 110 can include one or more GPUs, and the execution of program instructions by the GPU can generate or change display information.

[0476] In the embodiments of the present application, the display screen 194 can be used to display the interface of each application program.

[0477] The display screen 194 in the embodiments of the present application can be a touch screen. The display screen 194 can be integrated with a touch sensor 180K. The touch sensor 180K can also be referred to as a "touch panel". That is, the display screen 194 can include a display panel and a touch panel, and the touch sensor 180K and the display screen 194 form a touch screen, also referred to as a "touch screen". The touch sensor 180K is used to detect a touch operation acting on or near it. After the touch operation detected by the touch sensor 180K, the touch operation can be transmitted to the upper layer by the driver (such as the TP driver) of the kernel layer to determine the touch event type. The visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be arranged on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0478] The audio processing method and the audio transmission method provided in the embodiments of the present application can be implemented in the electronic device 100 with the above hardware structure.

[0479] The above describes the examples of the audio processing method and the audio transmission method provided in the embodiments of the present application in detail. It can be understood that the electronic device includes hardware and / or software modules corresponding to the execution of each function in order to implement the above functions. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed in the present text, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered beyond the scope of the present application.

[0480] The embodiments of the present application can divide the functional modules of the electronic device according to the above method examples. For example, each function can be divided into a functional module, or two or more functions can be integrated into one module. The integrated module can be implemented in the form of hardware or software functional module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division manner.

[0481] It should be noted that all related contents of each step involved in the above method embodiments can be cited to the function description of the corresponding functional module, which will not be repeated here.

[0482] The electronic device provided in the embodiments of the present application is used to execute the above audio processing method and audio transmission method, and thus can achieve the same effect as the above implementation method.

[0483] In the case of employing the integrated unit, the electronic device can further include a processing module, a storage module and a communication module. The processing module can be used to control and manage the actions of the electronic device. The storage module can be used to support the electronic device to execute the storage of program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

[0484] The processing module can be a processor or a controller. It can realize or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor can also be a combination of computing functions, such as one or more microprocessor combinations, digital signal processing (DSP) and microprocessor combinations, etc. The storage module can be a memory. The communication module can be a device for interacting with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a WiFi chip, etc.

[0485] The embodiments of the present application also provide a vehicle-mounted audio device, which includes one or more processors, one or more memories, a module installed with a plurality of application programs, and the memory stores one or more programs. When the one or more programs are executed by the processor, the vehicle-mounted audio device executes the audio processing method and the audio transmission method in the above embodiments.

[0486] The embodiments of the present application also provide a vehicle, which includes one or more processors, one or more memories, and the memory stores one or more programs. When the one or more programs are executed by the processor, the vehicle executes the audio processing method and the audio transmission method in the above embodiments.

[0487] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. When the computer program is executed by the processor, the processor executes the audio processing method and the audio transmission method in any of the above embodiments.

[0488] The embodiments of the present application also provide a computer program product, which makes the computer execute the related steps above when the computer program product runs on the computer, so as to realize the audio processing method and the audio transmission method in the above embodiments.

[0489] The embodiments of the present application also provide a chip. Please refer to Figure 22 , Figure 22 The chip provided by the embodiments of the present application is a structure schematic diagram of a chip. Figure 22 The chip shown in the figure can be a general-purpose processor or a special-purpose processor. The chip includes a processor 310. The processor 310 is used to execute the audio processing method and the audio transmission method in any of the above embodiments.

[0490] Optionally, the chip further comprises a transceiver 320, which is used to accept the control of the processor, and is used to support the communication device to execute the technical solutions shown in the foregoing.

[0491] Optionally, Figure 22 The chip shown can further comprise a storage medium 330.

[0492] It should be noted that, Figure 22 The chip shown can be implemented using one or more of the following circuits or devices: a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a state machine, gate logic, discrete hardware components, any other suitable circuit, or any combination of circuits capable of performing the various functions described throughout this application.

[0493] The electronic device, the vehicle-mounted audio device, the computer readable storage medium, the computer program product or the chip provided in the embodiments are all used to execute the corresponding method provided above, and thus the beneficial effects that can be achieved are referable to the beneficial effects of the corresponding method provided above, which will not be described here again.

[0494] Through the above description of the embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0495] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are only illustrative, for example, the division of the modules or units is only a logical function division, and in actual implementation, another division manner can be used, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0496] The units described as separate components may or may not be physically separate, and the components displayed as units may be a physical unit or multiple physical units, that is, may be located in one place, or also can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0497] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0498] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical scheme of the embodiments of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0499] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An audio processing method, characterized by, An application framework layer applied to an electronic device connected with a vehicle-mounted audio device, the method comprising: receiving audio data generated by a plurality of applications; processing the audio data into sub-audio data, the sub-audio data comprising at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sub-audio data, the mixed sub-audio data being obtained by mixing other sub-audio data except the media sub-audio data, the navigation sub-audio data, and the call sub-audio data; encoding the sub-audio data by a corresponding number of encoders created for the sub-audio data to obtain encoded audio; sending the encoded audio to the vehicle-mounted audio device through a preset transmission channel, the encoded audio being used to trigger the vehicle-mounted audio device to decode the encoded audio by a decoder created for the encoded audio to obtain decoded audio, the decoded audio being used for the vehicle-mounted audio device to play the decoded audio according to a playing strategy, the playing strategy comprising independently adjusting a playing volume corresponding to the decoded audio, and / or playing the decoded audio corresponding to a specified speaker through the specified speaker.

2. The method of claim 1, wherein, The method further comprises: obtaining an audio type of the sub-audio data, the audio type of the media sub-audio data being a media audio type, the audio type of the navigation sub-audio data being a navigation audio type, the audio type of the call sub-audio data being a call audio type, and the audio type of the mixed sub-audio data being a mixed audio type; creating an encoder matching the audio type of the sub-audio data.

3. The method of claim 2, wherein, The electronic device further comprises a virtual hardware abstraction layer, and the method further comprises: forwarding the sub-audio data to the encoder matching the sub-audio data through an audio route created by the virtual hardware abstraction layer, the audio route matching the audio type of the sub-audio data.

4. The method of claim 3, wherein, When the sub-audio data is the call sub-audio data, the method further comprises: transmitting the call sub-audio data to the encoder matching the call sub-audio data through a call channel in the virtual hardware abstraction layer.

5. The method according to any one of claims 1 to 4, characterized in that, The transmission channel comprises a first transmission channel and a second transmission channel, the first transmission channel being used for transmitting encoded audio corresponding to non-call audio sub-data, and the second transmission channel being used for transmitting encoded audio corresponding to the call sub-audio data.

6. The method of claim 2, wherein, The processing of the audio data into sub-audio data comprises: identifying at least two of the media sub-audio data, the navigation sub-audio data, the call sub-audio data, and the other sub-audio data from the audio data according to different audio type identifiers carried by the audio data; if the other sub-audio data is identified, mixing the other sub-audio data to obtain the mixed sub-audio data.

7. The method of claim 6, wherein, The other sub-audio data comprises at least one of notification sub-audio data, warning sub-audio data, and ringtone sub-audio data.

8. The method of claim 1, wherein, Before the receiving of the audio data generated by the plurality of applications, the method further comprises: receive an audio split instruction, the audio split instruction being used to instruct to process the audio data into the sub-audio data.

9. The method of claim 1, wherein, Before the receiving the audio data generated by the plurality of applications, the method further includes: in response to an input operation for the plurality of applications, generate the audio data; the applications include at least one of a media application, a navigation application, and a call application.

10. The method of claim 3, wherein, The method further includes: receive an instruction to disconnect the electronic device from the in-vehicle audio device, destroy the encoder, and instruct the virtual hardware abstraction layer to destroy the audio route.

11. An audio processing method, characterized by, An application framework layer applied to an in-vehicle audio device, the in-vehicle audio device being connected to an electronic device, the method including: receive encoded audio sent through a preset transmission channel; the encoded audio being obtained by processing sub-audio data through an encoder created by the application framework layer of the electronic device; the sub-audio data being obtained by processing audio data through the application framework layer of the electronic device; the audio data being generated by a plurality of applications in the electronic device; the sub-audio data including at least one of media sub-audio data, navigation sub-audio data, call sub-audio data, and mixed sound sub-audio data; the mixed sound sub-audio data being obtained by mixing other sub-audio data except the media sub-audio data, the navigation sub-audio data, and the call sub-audio data through the application framework layer of the electronic device; decode the encoded audio through a decoder created for the encoded audio to obtain decoded audio; play the decoded audio according to a playing strategy; the playing strategy including independently adjusting a playing volume corresponding to the decoded audio, and / or playing decoded audio corresponding to a specified speaker through the specified speaker.

12. The method of claim 11, wherein, The method further includes: obtain an audio type corresponding to the encoded audio; the audio type corresponding to the encoded audio of the media sub-audio data being a media audio type, the audio type corresponding to the encoded audio of the navigation sub-audio data being a navigation audio type, the audio type corresponding to the encoded audio of the call sub-audio data being a call audio type, and the audio type corresponding to the encoded audio of the mixed sound sub-audio data being a mixed sound audio type; create a decoder matching the audio type corresponding to the encoded audio.

13. The method of claim 12, wherein, The specified speaker includes a driver seat speaker, and the audio type of the decoded audio corresponding to the specified speaker is the navigation audio type.

14. An electronic device, comprising: including: one or more processors; one or more memories; The memory stores one or more programs, when the one or more programs are executed by the processor, the electronic device executes the method in any one of claims 1-10.

15. A car audio device, characterized by comprising: including: one or more processors; one or more memories; The memory stores one or more programs, when the one or more programs are executed by the processor, the in-vehicle audio device executes the method in any one of claims 11-13.

16. A vehicle characterized by comprising: including: one or more processors; one or more memories; The memory stores one or more programs, when the one or more programs are executed by the processor, make the vehicle execute the method in any one of claims 11 to 13.

17. A chip, characterized by comprise: The processor is configured to call and run the computer program from the memory, so that the electronic device installed with the chip executes the method in any one of claims 1 to 10, or so that the vehicle audio device installed with the chip executes the method in any one of claims 11 to 13.

18. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, when the computer program is executed by the processor, makes the processor execute the method in any one of claims 1 to 11, or makes the processor execute the method in any one of claims 10 to 13.

Images