Cross-device audio playing method, mobile terminal and computer readable storage medium

Abstract

The invention relates to the technical field of mobile terminals, and provides a cross-device audio playing method, a mobile terminal and a computer readable storage medium, the method is applied to the mobile terminal, and the mobile terminal establishes connection with an audio playing device through an interconnection application; comprising the steps that the mobile terminal responds to audio playing requests of one or more audio applications, and a corresponding audio channel is distributed to each piece of audio data according to the audio types of the audio data corresponding to the audio playing requests. And then, the mobile terminal transmits the audio data to the audio playing equipment through the distributed audio channel. Wherein the audio channels correspond to the audio types, namely, different audio channels are used for transmitting audio data of different audio types. Therefore, the audio playing device can clearly sense the audio type of the transmitted audio data, so that the audio data can be played by using the corresponding playing strategy, and the user-defined playing strategy of the audio playing terminal can be ensured to take effect smoothly.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of mobile terminals, and in particular, to a cross-device audio playback method, a mobile terminal, and a computer-readable storage medium. Background Art

[0002] With the development of terminal technology, most existing mobile terminals can be interconnected with other electronic devices. In the interconnected scenario, audio stream transfer and playback is one of the most commonly used functions by users. For example, in the interconnected scenario of a mobile phone and a car audio system, it is a high-frequency requirement for users to transfer music audio and navigation audio on the mobile phone side to the car audio system for playback.

[0003] However, there are usually multiple types of audio on existing mobile terminals, such as music audio, navigation audio, notification sounds, etc. In this case, these multiple types of audio will be mixed on the mobile terminal and then transmitted to the interconnected electronic device for playback. The electronic device cannot distinguish different audio from the mixed audio data, resulting in the failure of the customized playback strategy for specific audio to take effect. Summary of the Invention

[0004] Embodiments of the present application provide a cross-device audio playback method, a mobile terminal, and a computer-readable storage medium, which are used to solve the problem that the customized playback strategy on the audio playback device side fails to take effect when playing audio across devices.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a cross-device audio playback method is provided. This method is applied to a mobile terminal, and the mobile terminal establishes a connection with an audio playback device through an interconnected application. The method includes: when the mobile terminal receives an audio playback request from one or more audio applications, in response to the audio playback request from these one or more audio applications, according to the audio type of the audio data corresponding to the audio playback request, allocate corresponding audio channels for the audio data requested to be played by these audio applications. Among them, the audio channels correspond to the audio types, and different audio channels are used to transmit audio data of different audio types. Then, the mobile terminal transmits the audio data to the audio playback device through the audio channel corresponding to the audio type.

[0007] In this way, the audio playback device that receives this audio data can clearly perceive the audio type of this audio data. Furthermore, when the audio playback device has a customized playback strategy for this audio type, it can obtain the corresponding playback strategy according to the audio type and play this audio data. Ensure that in the interconnected scenario, the mobile terminal can ensure that the customized playback strategy of the interconnected audio playback terminal takes effect smoothly.

[0008] In a possible implementation of the first aspect, the audio playback device includes a vehicle. The mobile terminal can establish a connection with the in-vehicle unit of the vehicle through an interconnected application, thereby ensuring the smooth implementation of the custom playback policy on the in-vehicle unit side, especially ensuring the smooth implementation of the playback policy for navigation audio that the in-vehicle unit side focuses on.

[0009] In a possible implementation of the first aspect, currently, the audio with business requirements for playback policies is usually navigation audio and music audio. Therefore, in order to meet the actual business requirements while saving access resources as much as possible and ensuring the reasonable utilization of access resources, the audio access paths established by the mobile terminal can include three major categories: music audio access path, navigation audio access path, and other audio access paths. Among them, the music audio access path is used to independently transmit music audio. The navigation audio access path is used to independently transmit navigation audio. And the other audio access paths are used to transmit any one or more audio other than music audio and navigation audio.

[0010] In this way, while the mobile terminal ensures the smooth implementation of the custom playback policy corresponding to the audio playback device, it can reduce the resource consumption of the audio access paths and prevent too many audio access paths from occupying too many resources of the mobile terminal.

[0011] In a possible implementation of the first aspect, in order to facilitate the distinction of audio access paths corresponding to different audio types, different audio access paths can be characterized by unique access identifiers. Therefore, when the mobile terminal allocates corresponding audio access paths to audio data according to the audio type of the audio playback request, it can include: the mobile terminal first determines the access identifier corresponding to the audio type according to the predetermined shunting rule. Then, the mobile terminal establishes an audio access path corresponding to this access identifier, and thus the audio access path corresponding to the audio data can be determined through the access identifier, thereby realizing the allocation of audio data to the corresponding audio access path according to the audio type.

[0012] In a possible implementation of the first aspect, the software architecture of the mobile terminal includes a media framework, which is used to receive audio playback requests from audio applications. At the same time, the media framework in the mobile terminal can determine the access identifier corresponding to the audio type according to the predetermined shunting rule. And the media framework of the mobile terminal can establish an audio access path corresponding to this access identifier and allocate the audio data to the corresponding audio access path according to the audio type.

[0013] In another possible implementation of the first aspect, based on actual different business requirements, there may be corresponding different audio splitting requirements, such as increasing or decreasing audio channels accordingly. Therefore, the predetermined splitting rules may need to be maintained frequently. However, it is usually troublesome to maintain business rules on the media framework. And according to the software architecture characteristics of existing mobile terminals, it is usually not likely to let the media framework undertake too many services. Therefore, the decision-making of audio splitting can be transferred out of the media framework, and an additional splitting service that is convenient for developers to maintain is added to implement the decision-making of audio splitting.

[0014] Based on this, the mobile terminal can include a media framework and a splitting service. At the same time, after the mobile terminal establishes a connection with the audio playback device through the interconnected application, the splitting service registers an audio channel allocation callback function with the media framework.

[0015] Then, according to the predetermined splitting rules, determine the channel identifier corresponding to the audio type; establish an audio channel corresponding to the channel identifier, and allocate audio data to the corresponding audio channel according to the audio type, which may include: after the media framework of the mobile terminal receives an audio playback request from one or more audio applications, the media framework of the mobile terminal executes the audio channel allocation callback function and sends a splitting request carrying the audio type to the splitting service of the mobile terminal. Then, the splitting service of the mobile terminal responds to the splitting request of the media framework, determines the channel identifier corresponding to the audio type according to the predetermined splitting rules, and returns it to the media framework. Then, the media framework, according to the instructions of the splitting service, establishes an audio channel corresponding to the channel identifier, and allocates audio data to the corresponding audio channel according to the audio type. Thus, the configuration and maintenance efficiency of the splitting rules can be improved.

[0016] In a possible implementation of the first aspect, in addition to the media framework, the mobile terminal may further include a hardware virtualization sending service. This hardware virtualization sending service is activated after the mobile terminal establishes a connection with the audio playback device through the interconnected application, and is used to undertake transmitting the audio data transmitted by the media framework to the corresponding audio playback device. Based on this, transmitting the audio data to the audio playback device through the audio channel corresponding to the audio type may include:

[0017] The media framework transmits the audio data to the hardware virtualization sending service through the audio path corresponding to the audio data; the hardware virtualization sending service encapsulates the audio data into audio data packets and transmits them to the audio playback device. Among them, in order for the audio playback device to clearly perceive the audio type of the audio data, each piece of audio data carried in the audio data packet has an audio type identifier corresponding to the audio type of this audio data. Subsequently, the audio playback device can parse the audio data packet to obtain the audio data and the corresponding audio type identifier. Furthermore, the audio playback device can determine the audio type corresponding to the audio data according to the audio type identifier and play the audio data according to the playback strategy corresponding to the audio type to ensure the smooth effectiveness of the playback strategy.

[0018] In a possible implementation manner of the first aspect, the hardware virtualization sending service includes a virtual audio HAL and an audio sending module. The virtual audio HAL is used to receive the audio data shunted and transmitted from the media framework of the mobile terminal. The virtual audio HAL shunts and transmits the audio data shunted and transmitted from the media framework to the audio sending module, and the audio sending module transmits it to the audio playback device.

[0019] Based on this, the hardware virtualization sending service encapsulating the audio data into audio data packets and transmitting them to the audio playback device may include: the audio sending module encapsulating the audio data into audio data packets and transmitting them to the audio playback device; wherein, the audio sending module receives the audio data transmitted by the media framework through the audio path via the virtual audio HAL. At the same time, when the audio sending module encapsulates the audio data packet, it adds the audio type identifier corresponding to the audio data to the audio data packet so that the playback strategy of the audio playback device can take effect smoothly.

[0020] In a possible implementation manner of the first aspect, the mobile terminal establishing a connection with the audio playback device through the interconnected application may be achieved by establishing a P2P transmission channel. Subsequently, the audio data packet can be encrypted and transmitted from the mobile terminal to the audio playback device through the P2P transmission channel.

[0021] In a possible implementation manner of the first aspect, the mobile terminal establishing a connection with the audio playback device through the interconnected application may include: the mobile terminal displays a first connection interface in response to the connection broadcast sent by the audio playback device; wherein, the connection broadcast is generated when the audio playback device responds to the connection operation within the interconnected application. Then, the mobile terminal receives the connection code input by the user on the first connection interface and establishes a connection with the audio playback device; wherein, the connection code is generated by the audio playback device and displayed on the second connection interface of the audio playback device.

[0022] In a possible implementation manner of the first aspect, the connection code used by the mobile terminal to establish a connection with the audio playback device through the interconnected application may be a PIN code.

[0023] In a second aspect, the present application provides a cross-device audio playback method, which is applied to an audio playback device, and the audio playback device establishes a connection with a mobile terminal through an interconnected application. The method includes: the audio playback device receives an audio data packet from the mobile terminal, parses the audio data packet to obtain audio data and an audio type identifier of the audio data. Then, the audio playback device determines the audio type of the audio data according to the audio type identifier, obtains a corresponding playback policy according to the audio type, and plays the audio data according to the playback policy.

[0024] Among them, the audio data packet received by the audio playback device is obtained by encapsulating the audio data and the audio type identifier corresponding to the audio data by the hardware virtualization sending service after the media framework in the mobile terminal splits and transmits the audio data to the hardware virtualization sending service.

[0025] Thus, after the audio playback device receives the audio data packet transmitted from the mobile terminal, it can accurately sense the audio type of the audio data by parsing this audio type identifier. Furthermore, play this audio data according to the custom playback policy corresponding to the audio type to ensure the smooth effectiveness of the playback policy.

[0026] In a possible implementation manner of the second aspect, the audio playback device includes a vehicle, and the in-vehicle computer of the vehicle establishes a connection with the mobile terminal through an interconnected application. In another possible implementation manner of the second aspect, the in-vehicle computer of the vehicle can establish a P2P transmission channel with the mobile terminal through the interconnected application to realize the interconnection between the in-vehicle computer of the vehicle and the mobile terminal.

[0027] In a possible implementation manner of the second aspect, the audio playback device includes a hardware virtualization receiving service. This hardware virtualization receiving service is activated after the audio playback device establishes a connection with the mobile terminal through an interconnected application, and is used to receive the audio data packet from the mobile terminal.

[0028] Based on this, the audio playback device receives the audio data packet from the mobile terminal, parses the audio data packet to obtain the audio data and the audio type identifier of the audio data, which may include: the hardware virtualization receiving service receives the audio data packet from the mobile terminal, and parses the audio data packet to obtain the audio data and the audio type identifier of the audio data. Then, the hardware virtualization receiving service determines the audio type of the audio data according to the audio type identifier, obtains a corresponding playback policy according to the audio type, and plays the audio data according to the playback policy, thereby ensuring the smooth effectiveness of the playback policy.

[0029] In another possible implementation of the second aspect, the hardware virtualization receiving service in the audio playback device can receive audio data packets sent by the hardware virtualization sending service in the mobile terminal. And this audio data packet can be encapsulated by the audio sending module in the hardware virtualization sending service. Specifically, after the audio sending module receives audio data from the media framework through the virtual audio HAL, it encapsulates and packages this audio data and the corresponding audio type identifier.

[0030] In a possible implementation of the second aspect, when the hardware virtualization receiving service in the audio playback device is activated, obtaining the corresponding playback policy according to the audio type may include: the hardware virtualization receiving service executes the playback policy callback function and obtains the corresponding playback policy from the interconnected application according to the audio type; wherein, after the audio playback device establishes a connection with the mobile terminal through the interconnected application, the interconnected application registers the playback policy callback function with the hardware virtualization receiving service through the interconnected service SDK interface. Thus, by registering the playback policy callback function, it can be ensured that after the hardware virtualization receiving service obtains the audio data, it actively calls back to obtain the corresponding playback policy, thereby ensuring the smooth effectiveness of the playback policy.

[0031] In a possible implementation of the second aspect, the audio playback device establishing a connection with the mobile terminal through the interconnected application may include: the audio playback device responds to the user's operation of opening the interconnected application and displays the interconnected application interface. Then, the audio playback device responds to the user's brand selection operation within the interconnected application interface, displays the second connection interface including the connection code, and generates a connection broadcast and sends it to the mobile terminal of the brand corresponding to the brand selection operation. When the mobile terminal corresponding to the selected brand responds to the connection broadcast, displays the first connection interface, and receives the connection code input by the user in the first connection interface, it can establish a connection with the audio playback device, thereby realizing the interconnection between the audio playback device and the mobile terminal.

[0032] In a third aspect, the present application provides a mobile terminal, including: one or more processors and a memory, the memory is coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the mobile terminal is caused to perform the following steps:

[0033] In response to an audio playback request of one or more audio applications, allocate a corresponding audio path for the audio data according to the audio type of the audio data corresponding to the audio playback request; wherein, the audio path corresponds to the audio type, and different audio paths are used to transmit audio data of different audio types; transmit the audio data through the audio path corresponding to the audio type to the audio playback device, and the audio playback device plays the audio data according to the playback policy corresponding to the audio type.

[0034] In a possible implementation of the third aspect, the audio playback device includes a vehicle; when the above computer instructions are executed by a processor, the mobile terminal is further caused to perform the following steps: The mobile terminal establishes a connection with the in-vehicle unit of the vehicle through an interconnection application.

[0035] In a possible implementation of the third aspect, when the above computer instructions are executed by a processor, the mobile terminal is further caused to perform the following steps: Establish a music audio path, a navigation audio path, and other audio paths. Among them, the music audio path is used to independently transmit music audio; the navigation audio path is used to independently transmit navigation audio; the other audio paths are used to transmit any one or more audio other than music audio and navigation audio.

[0036] In a possible implementation of the third aspect, when the above computer instructions are executed by a processor, the mobile terminal is further caused to perform the following steps: According to a predetermined shunting rule, determine the path identifier corresponding to the audio type; establish an audio path corresponding to the path identifier, and allocate audio data to the corresponding audio path according to the audio type.

[0037] In a possible implementation of the third aspect, the mobile terminal includes a media framework and a shunting service; when the above computer instructions are executed by a processor, the mobile terminal is further caused to perform the following steps: The media framework executes an audio path allocation callback function and sends a shunting request carrying the audio type to the shunting service; among them, the audio path allocation callback function is registered by the shunting service to the media framework after the mobile terminal establishes a connection with the audio playback device through an interconnection application; the shunting service responds to the shunting request, determines the path identifier corresponding to the audio type according to a predetermined shunting rule, and returns it to the media framework; the media framework establishes an audio path corresponding to the path identifier and allocates audio data to the corresponding audio path according to the audio type.

[0038] In a possible implementation of the third aspect, the mobile terminal includes a media framework and a hardware virtualization sending service; the hardware virtualization sending service is activated after the mobile terminal establishes a connection with the audio playback device through an interconnection application; when the above computer instructions are executed by a processor, the mobile terminal is further caused to perform the following steps: The media framework transmits the audio data to the hardware virtualization sending service through the audio path corresponding to the audio data; the hardware virtualization sending service encapsulates the audio data into an audio data packet and transmits it to the audio playback device; among them, the audio data packet carries an audio type identifier of the audio type corresponding to the audio data.

[0039] In a possible implementation of the third aspect, the hardware virtualization transmission service of the mobile terminal includes a virtual audio HAL and an audio transmission module; when the above computer instructions are executed by the processor, the mobile terminal further performs the following steps: the audio transmission module encapsulates the audio data into audio data packets and transmits them to the audio playback device; wherein, the audio transmission module receives the audio data transmitted through the audio path by the media framework via the virtual audio HAL.

[0040] In a possible implementation of the third aspect, when the above computer instructions are executed by the processor, the mobile terminal further performs the following steps: the mobile terminal establishes a P2P transmission channel with the audio playback device through the interconnected application.

[0041] In a possible implementation of the third aspect, when the above computer instructions are executed by the processor, the mobile terminal further performs the following steps: in response to the connection broadcast sent by the audio playback device, display a first connection interface; wherein, the connection broadcast is generated in response to the connection operation of the user within the interconnected application by the audio playback device; receive the connection code input by the user on the first connection interface, and establish a connection with the audio playback device; wherein, the connection code is generated by the audio playback device and is displayed on the second connection interface of the audio playback device.

[0042] In a fourth aspect, the present application provides an audio playback device, including: audio hardware, one or more processors, and a memory, the audio hardware and the memory are respectively coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the audio playback device performs the following steps: receive the audio data packet from the mobile terminal, parse the audio data packet to obtain the audio data and the audio type identifier of the audio data; determine the audio type of the audio data according to the audio type identifier, obtain the corresponding playback policy according to the audio type, and play the audio data according to the playback policy.

[0043] In a possible implementation of the fourth aspect, when the above computer instructions are executed by the processor, the audio playback device further performs the following steps: the audio playback device establishes a P2P transmission channel with the mobile terminal through the interconnected application.

[0044] In a possible implementation of the fourth aspect, the audio playback device includes a hardware virtualization receiving service, which is activated after the audio playback device establishes a connection with the mobile terminal through the interconnected application; when the above computer instructions are executed by the processor, the audio playback device further performs the following steps: the hardware virtualization receiving service receives audio data packets from the mobile terminal, and parses the audio data packets to obtain audio data and the audio type identifier of the audio data; the hardware virtualization receiving service determines the audio type of the audio data according to the audio type identifier, obtains the corresponding playback policy according to the audio type, and plays the audio data according to the playback policy.

[0045] In a possible implementation of the fourth aspect, when the above computer instructions are executed by the processor, the audio playback device further performs the following steps: the hardware virtualization receiving service executes a playback policy callback function, and obtains the corresponding playback policy from the interconnected application according to the audio type; among them, the playback policy callback function is registered by the interconnected application to the hardware virtualization receiving service through the interconnected service SDK interface after the audio playback device establishes a connection with the mobile terminal through the interconnected application.

[0046] In a possible implementation of the fourth aspect, when the above computer instructions are executed by the processor, the audio playback device further performs the following steps: in response to the user's operation of opening the interconnected application, display the interconnected application interface; in response to the user's brand selection operation within the interconnected application interface, display a second connection interface including a connection code, and generate a connection broadcast to send to the mobile terminal of the brand corresponding to the brand selection operation; the mobile terminal responds to the connection broadcast to display a first connection interface, and receives the connection code input by the user in the first connection interface to establish a connection with the audio playback device.

[0047] In a possible implementation of the fourth aspect, the above audio hardware includes a speaker.

[0048] In the fifth aspect, a computer-readable storage medium of the present application stores a computer program, which, when executed by a processor in a mobile terminal, causes the mobile terminal to execute the cross-device audio playback method as described in the first aspect and any of its possible implementations. Alternatively, when the computer program is executed by a processor in an audio playback device, the audio playback device is caused to execute the cross-device audio playback method as described in the second aspect and any of its possible implementations.

[0049] In a sixth aspect, the present application provides a computer program product. When the computer program product runs on a computer, it causes the computer to execute the method according to the first aspect and any possible implementation manner thereof. The computer may be the above-mentioned mobile terminal. Alternatively, when the computer program product runs on a computer, it causes the computer to execute the method according to the second aspect and any possible implementation manner thereof. The computer may be the above-mentioned audio playback device.

[0050] It can be understood that for the mobile terminal according to any possible implementation manner of the above-mentioned third aspect, the audio playback device according to any possible implementation manner of the fourth aspect, the computer-readable storage medium according to the fifth aspect, and the beneficial effects that can be achieved by the computer program product according to the sixth aspect, reference may be made to the beneficial effects in the first aspect and any possible implementation manner thereof, or reference may be made to the beneficial effects in the second aspect and any possible implementation manner thereof, which will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 FIG. is a schematic diagram of a scenario where a mobile phone 110 and a vehicle 120 are interconnected according to an embodiment of the present application;

[0052] Figure 2 FIG. is another schematic diagram of a scenario where a mobile phone 110 and a vehicle 120 are interconnected according to an embodiment of the present application;

[0053] Figure 3 FIG. is a schematic diagram of an operation flow for a mobile phone 110 and a vehicle 120 to achieve interconnection according to an embodiment of the present application;

[0054] Figure 4 FIG. is a schematic diagram of an interface where a mobile phone 110 displays a first connection interface according to an embodiment of the present application;

[0055] Figure 5 FIG. is a schematic diagram of an interface of a vehicle head unit settings interface according to an embodiment of the present application;

[0056] Figure 6 FIG. is a block diagram of the software and hardware architecture of a mobile terminal according to an embodiment of the present application;

[0057] Figure 7 FIG. is a process of cross-device audio playback in an interconnected scenario according to an embodiment of the present application Figure 1 ;

[0058] Figure 8 FIG. is a process of cross-device audio playback in an interconnected scenario according to an embodiment of the present application Figure 2 ;

[0059] Figure 9 FIG. is a schematic diagram of an audio data packet format according to an embodiment of the present application;

[0060] Figure 10 A process for cross-device audio playback in an interconnected scenario provided by an embodiment of the present application Figure 3 ;

[0061] Figure 11 An interaction process of a cross-device audio playback method provided by an embodiment of the present application Figure 1 ;

[0062] Figure 12 An interaction process of a cross-device audio playback method provided by an embodiment of the present application Figure 2 ;

[0063] Figure 13 A schematic structural diagram of a mobile terminal 1300 provided by an embodiment of the present application;

[0064] Figure 14 A schematic structural diagram of a chip system provided by an embodiment of the present application. Detailed implementation manners

[0065] Next, the technical solutions of the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first" and "second" do not limit the quantity and execution order, and the terms "first" and "second" do not necessarily mean different. Also, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" means two or more.

[0066] With the development of terminal technology, most existing mobile terminals can be interconnected with other electronic devices. Through interconnection, the mobile terminal can transfer the audio stream to other electronic devices for playback. Through interconnection, the control right of the mobile terminal can also be opened to other electronic devices, so that the user can remotely control the mobile terminal on other electronic devices.

[0067] Specifically, the mobile terminal can be interconnected with other electronic devices through a near-field communication module such as Bluetooth, wireless network (such as Wi-Fi), or physical data cable. Alternatively, the user can also install an interconnection application on the mobile terminal and the electronic device respectively, and connect the mobile terminal and this electronic device through Bluetooth, wireless network, etc.

[0068] For example, mobile terminals such as mobile phones, foldable mobile phones, and tablet computers can be interconnected with large-screen electronic devices such as TVs and computers. Mobile terminals such as mobile phones, foldable mobile phones, and tablet computers can also be interconnected with the in-vehicle infotainment system (IVI) in a vehicle to play audio such as music and navigation inside the vehicle.

[0069] Hereinafter, taking a mobile phone and the IVI in a vehicle as an example, the interconnection scenarios of the above-mentioned mobile terminals will be described.

[0070] The IVI is an interactive control device configured in a vehicle. Through the IVI, the interaction between the vehicle and the user can be realized. The IVI can receive the input operations of the user and perform intelligent control on the vehicle according to the input operations of the user. For example, the IVI configured in the vehicle may include components such as a touch screen, a rotary knob, buttons, and speakers (audio systems). Through the touch screen, the rotary knob, and the buttons, the IVI can receive the operations of the user, and then display text, pictures on the touch screen in response to the operations of the user, or control the speakers (audio systems) to play media audio such as radio, music, and navigation.

[0071] As Figure 1 shown, a schematic diagram of a scenario where a mobile phone 110 and a vehicle 120 are interconnected is shown. The mobile phone 110 can be interconnected with the IVI in the vehicle 120 through a near-field communication module such as in-vehicle Bluetooth, a wireless network, or a physical data cable. Through the interconnection between the mobile phone and the IVI, media audio such as music audio and navigation audio played on the mobile phone 110 side can be streamed to the vehicle 120 for playback.

[0072] As Figure 2 shown, a schematic diagram of another scenario where a mobile phone 110 and a vehicle 120 are interconnected is shown. The mobile phone 110 can be interconnected with the IVI in the vehicle 120 through an interconnection application. Through the interconnection application, media audio such as music audio and navigation audio played on the mobile phone 110 side can be streamed to the vehicle 120 for playback. At the same time, through the interconnection application, the control right of the mobile phone 110 can also be opened to the IVI of the vehicle 120, and then the user can operate the applications in the mobile phone 110 on the touch screen of the IVI. In this way, the user can control the mobile phone 110 in the vehicle 120.

[0073] For example, the user can pre-set operable applications in the interconnected application interface 111 of the mobile phone 110 (i.e., the user pre-selects the applications in the interconnected application of the mobile phone 110 that can open the control right to the in-vehicle computer). For example, the user can set applications such as mail, music, settings, calls, chats, and navigation in the mobile phone 110 as operable applications. At the same time, the user can also store information such as music playlists and common navigation destinations in the interconnected application. Then, after the user operates the interconnected application through the touch screen 121 corresponding to the in-vehicle computer of the vehicle 120 and interconnects the mobile phone 110 with the in-vehicle computer of the vehicle 120 through the interconnected application, the user can directly remotely operate these pre-set operable applications on this touch screen 121.

[0074] Exemplarily, after the mobile phone 110 is interconnected with the in-vehicle computer on the vehicle 120 through the interconnected application, the user can directly open the music application on the mobile phone 110 on the touch screen 121 of the in-vehicle computer and transfer the music audio stream on the mobile phone 110 side to play in the vehicle 120. At the same time, the user can also directly open the navigation application on the mobile phone 110 on the touch screen 121 of the in-vehicle computer and transfer the navigation audio stream on the mobile phone 110 side to play in the vehicle 120. In this way, the user can operate the applications on the mobile phone 110 without operating the mobile phone 110 in the vehicle 120, which is convenient for the user to operate.

[0075] Of course, it can be understood that after the mobile phone 110 is interconnected with the in-vehicle computer on the vehicle 120 through the interconnected application, in addition to operating on the touch screen 121 of the in-vehicle computer, the user can also operate on the mobile phone 110 to start the music application to play music audio. Or operate on the mobile phone 110 to start the navigation application to play navigation audio, etc. At this time, since the mobile phone 110 and the in-vehicle computer of the vehicle 120 are interconnected, even if the user operates the music touch screen and navigation touch screen on the mobile phone 110 side, the music audio and navigation audio will be transmitted from the mobile phone 110 to the in-vehicle computer and played in the vehicle 120. That is, the music and navigation are played through the in-vehicle computer speaker.

[0076] As Figure 3 shown, a schematic diagram of the scenario where the user operates the interconnected application on the touch screen 121 of the in-vehicle computer of the vehicle 120 and interconnects the mobile phone 110 with the in-vehicle computer of the vehicle 120 is shown.

[0077] First, after the in-vehicle computer on the vehicle 120 is started, the touch screen 121 displays the main interface 122 of the in-vehicle computer. The main interface 122 includes an application list icon 123.

[0078] It can be understood that this application list icon 123 can be in any area of the main interface 122. Refer to Figure 3, the application list icon 123 is set in the menu bar below the main interface 122. Of course, according to the actual interface appearance design, the application list icon 123 can also be in other areas. This embodiment of the present application Figure 3 is only for illustrative purposes and does not impose any limitation on the setting area of the application list icon 123.

[0079] The in-vehicle computer of the vehicle 120 responds to the user's click operation on the application list icon 123 in the main interface 122 and enters the application list interface 124. The application list interface 124 may include a "search bar" and applications such as "weather", "gallery", "settings",... "clock", "file management", "interconnection", etc.

[0080] The in-vehicle computer of the vehicle 120 responds to the user's click operation on the "interconnection" application in the application list interface 124 and enters the interconnection application interface 125 (that is, in response to the user's operation of opening the interconnection application, the interconnection application interface 125 is displayed). The interconnection application interface 125 displays the mobile phone brands currently supported by the in-vehicle computer of the vehicle 120.

[0081] The in-vehicle computer of the vehicle 120 responds to the user's click operation on the mobile phone brand in the interconnection application interface 125. For example, in response to the user's click operation on the mobile phone brand corresponding to the mobile phone 110 (that is, the user's brand selection operation in the interconnection application interface 125), it enters the second connection interface 126.

[0082] The second connection interface 126 displays the connection code for connecting to the mobile phone and is accompanied by a connection instruction: "Turn on Bluetooth on the mobile phone → Approach the in-vehicle computer → Enter the above connection code on the mobile phone".

[0083] Finally, the mobile phone 110 turns on Bluetooth in response to the user's Bluetooth turn-on operation. After detecting that it is currently close to the in-vehicle computer of the vehicle 120, the mobile phone 110 displays the first connection interface. As Figure 4 shown, it shows a schematic diagram of the interface of the mobile phone 110 displaying the first connection interface.

[0084] The first connection interface 112 displayed on the mobile phone 110 includes an input box for entering the connection code. Further, the mobile phone 110 receives the connection code input by the user through the input box. If the connection code received by the mobile phone 110 is the same as the connection code displayed on the second connection interface 126 on the in-vehicle computer, the interconnection with the in-vehicle computer of the vehicle 120 is completed through this connection code.

[0085] In some embodiments, the connection code may be a personal identification number (PIN code).

[0086] In some embodiments, after the mobile phone 110 approaches the in-vehicle unit, it can detect the connection broadcast sent by the in-vehicle unit, and then display the first connection interface 112 as shown in Figure 4 by responding to the connection broadcast sent by the in-vehicle unit. This connection broadcast can be generated after the in-vehicle unit generates a connection code or synchronously generated.

[0087] The embodiments of the present application do not impose any limitation on the type of the connection broadcast, and it can be any existing broadcast that can be used to establish interconnection. In some embodiments, the connection broadcast sent by the in-vehicle unit can be a Bluetooth Low Energy (BLE) broadcast, which can be specifically set according to actual needs, and the embodiments of the present application do not impose any limitation on this.

[0088] For the convenience of understanding the interconnection operation process, the above process will be described from the user's perspective as follows.

[0089] That is, the user performs a connection operation on the interconnection application on the touch screen 121 of the in-vehicle unit according to the Figure 3 shown connection operation process, triggers the touch screen 121 of the in-vehicle unit to display a second connection interface 126 including a PIN connection code and a connection indication, and triggers the in-vehicle unit to generate a connection broadcast corresponding to the connection code for broadcasting. After that, the user refers to the connection indication on the second connection interface 126, operates the mobile phone 110 to turn on the Bluetooth of the mobile phone 110, and holds the mobile phone 110 close to the in-vehicle unit. After the mobile phone 110 detects the connection broadcast broadcast by the in-vehicle unit, it displays the first connection interface 112 in response to the connection broadcast. If the user inputs the PIN connection code displayed on the second connection interface 126 on the first connection interface 112 of the mobile phone 110, the mobile phone 110 receives the connection code and can complete the connection between the mobile phone 110 and the in-vehicle unit on the vehicle 120 based on the connection.

[0090] It should be noted that Figure 3 and Figure 4 the shown interconnection application operation scenarios are used as examples in the embodiments of the present application, and the embodiments of the present application do not constitute any limitation on this. That is, when connecting through the interconnection application, the connection operations of the user in the interconnection application are not limited to the Figure 3 and Figure 4 shown scenarios, which specifically depend on the actual user interface (UI) design of the interconnection application. The Figure 3 and Figure 4 of the embodiments of the present application do not constitute a limitation on the interconnection application operation scenarios.

[0091] Currently, in the interconnection scenario, audio stream transfer and playback is one of the most frequently used functions by users. Especially in the interconnection scenario between a mobile phone and an in-vehicle unit, transferring the navigation audio stream on the mobile phone side to the in-vehicle unit for playback is a high-frequency demand of users.

[0092] However, there are usually multiple types of audio on existing mobile terminals. In addition to music and navigation, there are also system notification sounds, application notification sounds, alarm sounds, etc. According to the audio playback strategies of various audio applications on existing mobile terminals, if there are multiple audio at the same time, the audio application may not stop playing a certain audio, but mix and play multiple audios.

[0093] For example, when the mobile terminal is playing music audio and the navigation application needs to play a navigation announcement sound, the music application will not pause playing the music audio, but choose to lower the volume of the music audio, so that the mobile terminal mixes and plays the music audio and the currently needed navigation announcement sound together. That is, the music audio will be mixed and played with the navigation announcement sound, but in order to highlight the currently urgently needed navigation announcement sound, the volume of the music audio will be lower than that of the navigation announcement sound.

[0094] In this way, if the mobile terminal is interconnected with other electronic devices, these multiple types of audio will be mixed on the mobile terminal and then transmitted to the interconnected electronic devices for playback.

[0095] However, according to actual business requirements, different playback strategies are configured for different types of audio on existing electronic devices. Therefore, if the mobile terminal transmits mixed audio to the interconnected electronic device, it will cause the interconnected electronic device to be unable to recognize a specific type of audio, and thus cannot adopt a specific playback strategy for that specific type of audio, affecting the effectiveness of the interconnected electronic device's custom playback strategy for audio types.

[0096] Exemplarily, taking the interconnection between a mobile phone and a car head unit as an example, after the mobile phone and the car head unit are interconnected, multiple audio on the mobile phone side will be mixed and transmitted to the car head unit for playback by the car head unit speaker. However, there may be multiple car head unit speakers configured on some vehicles, and the car manufacturer can specify different car head unit speakers for different types of audio to play.

[0097] For example, in order to enable the driver to clearly hear the navigation audio, the car manufacturer can configure the car head unit speaker closest to the driver to play the navigation audio at a high volume for the navigation audio. At the same time, in order to meet the personalized needs of users, the vehicle can also provide a sound setting function on the car head unit to allow users to define the playback strategy by themselves.

[0098] As Figure 5 shown, a schematic diagram of an interface of a car head unit setting interface is shown. The left side of the setting interface includes setting categories such as "Network", "Display", "Sound", "General", "Original Vehicle", "Reverse", "Voice", etc. In response to the user's click operation on the setting category in the setting interface, the car head unit can display the corresponding setting interface for each setting category on the right side of the setting interface. Figure 5As shown, in response to the user's click operation on the "Sound" major setting category in the in-vehicle device settings interface, the corresponding sound settings interface is displayed on the right side of the settings interface.

[0099] In this sound settings interface, the in-vehicle device can set the sound priority in response to the user's operation, and can also, in response to the user's operation, separately set whether the navigation audio is played in a mixed mode or in an exclusive mode. Among them, if the in-vehicle device sets the navigation audio to be played in a mixed mode, the in-vehicle device can also, in response to the user's operation, correspondingly set the volume of the navigation audio and other audio, so as to ensure that the user can hear the navigation audio during mixed playback.

[0100] If the in-vehicle device sets the navigation audio to be played in an exclusive mode, it means that the in-vehicle device will not play any other audio except the navigation audio. That is, the in-vehicle device will only play the navigation audio transmitted from the mobile phone, and will choose not to play other music audio, notification sounds, etc. transmitted from the mobile phone. The navigation audio will exclusively occupy the in-vehicle device's speaker and only play the navigation audio.

[0101] Then, if the in-vehicle device does not know that the audio currently transmitted from the mobile phone contains navigation audio, the playback strategy set by the in-vehicle device for the navigation audio will not take effect, resulting in a poor user experience.

[0102] It should be noted that Figure 5 The in-vehicle device settings interface shown is only used as an example in the embodiments of the present application, and does not impose any limitations on the in-vehicle device settings interface.

[0103] Based on this, in an interconnected scenario, in order to ensure that an electronic device interconnected with a mobile terminal can distinguish different types of audio and implement a specific playback strategy for a specific type of audio for a custom audio playback strategy, the embodiments of the present application provide a cross-device audio playback method.

[0104] The cross-device audio playback method provided by the embodiments of the present application is applied to a mobile terminal and an audio playback device interconnected with this mobile terminal. Among them, the mobile terminal and the audio playback device establish a connection through an interconnected application.

[0105] In the embodiments of the present application, the mobile terminal can be any type of device such as a mobile phone, a foldable mobile phone, a tablet computer, etc. The audio playback device can be an in-vehicle device or any large-screen electronic device, such as a TV, a computer, etc.

[0106] It can be understood that the embodiments of the present application do not specifically limit the type of the mobile terminal and the type of the audio playback device interconnected with the mobile terminal.

[0107] After the mobile terminal establishes a connection with the audio playback device through the interconnected application, if the mobile terminal receives an audio playback request from one or more audio applications. For example, it receives an audio playback request from any one or more of the navigation application, music application, and system application. The mobile terminal establishes different audio channels for different types of audio data requested to be played by the audio application according to the audio type of the audio data.

[0108] Then, the mobile terminal allocates the audio data requested to be played by these audio applications to the audio channels corresponding to the audio types. The different types of audio data are transmitted to the audio playback device through the allocated audio channels for playback. That is, the audio channels correspond to the audio types, and the mobile terminal allocates the audio data to the corresponding audio channels for transmission according to the audio type, so that the audio data can be transmitted to the audio playback device through different channels.

[0109] In this way, since the mobile terminal transfers the audio data stream to the audio playback device through different audio channels according to the audio type. Therefore, when the audio playback device receives the audio data, it can determine the audio type of the audio data. Therefore, if the audio playback device has a specific playback strategy for this type of audio, it can obtain the corresponding playback strategy according to the audio type to play the transmitted audio data.

[0110] Thus, for the audio data requested to be played by the audio application, the mobile terminal shunts and transmits each audio playback device according to the audio type. Whether the transmitted audio data is one or more, it can enable the audio playback device to directly determine the audio type of the received audio data. Furthermore, the audio playback device can play the audio according to the defined playback strategy for different types of audio, thus ensuring the smooth effectiveness of the custom playback strategy on the audio playback device.

[0111] That is to say, the cross-device audio playback method provided by the embodiments of the present application mainly lies in that the mobile terminal does not mix the audio data of different audio types, but independently transfers them to the audio playback device according to the audio type, so as to ensure that the audio playback device can play the audio according to the custom playback strategy.

[0112] Hereinafter, taking the interconnection between a mobile phone and a car stereo as an example, the cross-device audio playback method of the embodiments of the present application will be explained.

[0113] The mobile phone is interconnected with the car stereo through the interconnected application (the interconnected operation scenario can refer to Figure 3 and Figure 4 ). After that, if the mobile phone receives an audio playback request from one or more audio applications. The mobile phone then establishes and allocates a corresponding audio channel for each type of audio data requested to be played by each audio application according to the audio type of the audio data.

[0114] Then, the mobile phone transmits audio data of different audio types to the vehicle head unit through the audio path corresponding to the audio type. In this way, since the audio data transmitted by the mobile phone is shunted to the vehicle head unit through different audio paths according to the audio type, after receiving the audio data, the vehicle head unit can clearly know the audio type of each piece of audio data transmitted by the mobile phone. Furthermore, the vehicle head unit can directly obtain the playback strategy defined for this type of audio based on the audio type of the received audio data, and thus play the transmitted audio data using the corresponding playback strategy.

[0115] Exemplarily, if the mobile phone transmits audio data to the vehicle head unit via the navigation audio path, then when the vehicle head unit receives this audio data, it can determine that the audio type of this audio data is navigation audio. Furthermore, if the vehicle head unit has a customized playback strategy for navigation audio, it can obtain the customized playback strategy for navigation audio to play the received audio data.

[0116] Thus, even in the interconnected scenario, the customized playback strategy for navigation audio on the vehicle head unit can take effect smoothly, thereby ensuring the user experience.

[0117] It can be understood that the customized playback strategy can be the playback strategy fixedly configured at the factory of the vehicle head unit, or the playback strategy configured by the user on the vehicle head unit (the scenario of the user configuring the playback strategy can be referred to Figure 5 as shown).

[0118] In some embodiments, the audio types of the audio data may include music audio, navigation audio, system notification sound, application notification sound, alarm sound, etc. Optionally, the mobile terminal can establish corresponding audio paths for each audio type one by one. That is, for how many audio types of audio data there are on the mobile terminal, the mobile terminal establishes the same number of audio paths to transmit the audio data corresponding to these audio types respectively.

[0119] Of course, in order to save path resources, the mobile terminal can also configure the number of audio paths according to actual service requirements. For example, the number of audio paths can be set according to the audio data that needs to be focused on in actual services.

[0120] In some embodiments, taking the application scenario of the interconnection between the mobile phone and the vehicle head unit as an example, since the vehicle head unit usually pays more attention to music audio and navigation audio, the vehicle head unit may only have specific playback strategies for music audio and navigation audio. Therefore, the mobile phone can only focus on shunting music audio and navigation audio, and thus the mobile phone can only establish three audio paths, namely the music audio path, the navigation audio path, and the other audio path.

[0121] Accordingly, the music audio path is dedicated to independently transmitting music audio, and the navigation audio path is dedicated to independently transmitting navigation audio. Other audio paths can transmit all other types of audio except music audio and navigation audio. For example, other audio paths can transmit system notification sounds, application notification sounds, alarm sounds, etc.

[0122] It can be understood that since other audio paths can transmit multiple different types of audio, if there are multiple audio data in the mobile terminal that need to be transmitted via other audio paths simultaneously, then the multiple audio data on this other audio path are still transmitted to the audio playback device after being mixed. However, it should be noted that because the audio path is set according to the actual playback strategy of the in-vehicle unit, even if the mixed audio is transmitted through other audio paths, it will not affect the effectiveness of the playback strategy on the in-vehicle unit side.

[0123] It should be noted that the number of audio paths can be set according to the actual situation, and the embodiments of the present application do not make any limitations in this regard. For example, a separate notification audio path can also be established for the system notification sound, which is dedicated to transmitting the system notification sound.

[0124] In the Android operating system, in order to manage access to audio hardware (such as speakers), there is usually a corresponding audio HAL in the Hardware Abstraction Layer (HAL). This audio HAL is used to provide an interface between the upper-layer application and the audio hardware. By using the audio HAL, the application can access and control the functions of the audio hardware, including audio input, output, encoding, decoding, and mixing, etc. It can be simply understood that the main purpose of the audio HAL is to provide a set of standard audio interfaces for the upper-layer application to manage the input, output, configuration, and control of the audio stream.

[0125] Therefore, when the mobile terminal is not interconnected with any audio playback device, the audio data requested to be played by the audio application in the mobile terminal usually passes through the audio HAL and is transmitted to the audio hardware of the mobile terminal to play the audio. For example, the mobile phone transmits the audio data to the mobile phone speaker for playback via the audio HAL.

[0126] As Figure 6 shown, taking the mobile phone as an example, a block diagram of the software and hardware architecture of a mobile terminal is shown. Hereinafter, the audio playback process of the mobile terminal will be described in detail in conjunction with Figure 6 this.

[0127] Refer to Figure 6, the software and hardware structure of the mobile terminal includes several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In the embodiments of the present application, the software and hardware system of the mobile terminal includes an application layer, an application framework layer, a HAL layer, a kernel layer, and a hardware layer from top to bottom.

[0128] Among them, the application layer includes audio applications such as music applications and navigation applications. When the mobile terminal needs to connect to an audio playback device through an interconnected application, the application layer may also include an interconnected application. The application framework layer includes a media framework. The HAL layer includes an audio HAL. The kernel layer includes an audio driver. The hardware layer includes the audio hardware of the mobile phone, such as the mobile phone speaker.

[0129] It can be understood that according to actual needs, the application layer may also include applications such as cameras, photo galleries, calendars, calls, maps, navigation, WLAN, Bluetooth, videos, and text messages. The application framework layer may also include a window manager, a content provider, a view system, a resource manager, a notification manager, an activity manager, an input manager, etc.

[0130] The HAL layer may also include a display HAL, a camera HAL, a sensor HAL, etc. Correspondingly, the kernel layer may also include a display driver, a camera driver, a sensor driver, etc. The hardware layer may also include hardware such as a display screen, a camera, and a sensor. It can be understood that the embodiments of the present application do not make any limitations in this regard.

[0131] Specifically, audio applications such as music applications and navigation applications in the mobile phone application layer will send audio data requests to the media framework to play audio. After the media framework receives the audio data and audio playback requests of the audio application, it sends the audio data corresponding to the audio playback request to the audio HAL and transmits it to the mobile phone speaker for playback via the audio driver.

[0132] Thus, it can be seen that the audio hardware of the mobile terminal can play the audio data requested by the upper-layer audio application, which needs to be realized via the audio HAL and audio driver corresponding to the audio hardware.

[0133] However, after the mobile terminal is interconnected with the audio playback device, the audio hardware for playing audio will change from the mobile terminal's own audio hardware to this interconnected audio playback device. It can be simply understood that after establishing the interconnection, this interconnected audio playback device can be regarded as a peripheral device of the mobile terminal for playing audio. Taking a mobile phone and a car head unit as an example, that is, the audio hardware for playing audio will change from the mobile phone speaker to the car head unit speaker.

[0134] However, in the software architecture of a mobile terminal, there is usually no corresponding audio HAL and audio driver for this interconnected audio playback device. Therefore, after the mobile terminal is interconnected with the audio playback device, in order to ensure that the audio data of the mobile terminal is not played from its own speaker but can be smoothly transmitted to the interconnected audio playback device to realize the playback of audio data on the interconnected peripheral device. After the mobile terminal and the audio playback device are interconnected, a hardware virtualization service needs to be established accordingly. The mobile terminal and the audio playback device realize the transfer and playback of audio data through the established hardware virtualization service.

[0135] It can be understood that the mobile terminal is the sender of the hardware virtualization service, and the audio playback device is the receiver of the hardware virtualization service. For example, a mobile phone is the sender of the hardware virtualization service, and a car head unit is the receiver of the hardware virtualization service.

[0136] Hereinafter, for the convenience of distinction, the hardware virtualization service on the mobile terminal side is called the hardware virtualization sending service, and the hardware virtualization service on the audio playback device side is called the hardware virtualization receiving service.

[0137] At the same time, in the embodiments of the present application, after the mobile terminal is interconnected with the audio playback device, the audio stream splitting is implemented by the media framework in the application framework layer. That is, in order to ensure that the custom playback policy of the audio playback device can take effect, after the media framework receives the audio playback request of the audio application, it allocates a corresponding audio path for each audio data according to the audio type of the audio data to achieve audio stream splitting. For example, music audio is allocated to the music audio path, and navigation audio is allocated to the navigation audio path. Then, the media framework sends the split audio data to the interconnected audio playback device via the hardware virtualization sending service.

[0138] That is to say, the hardware virtualization sending service, as the sender of the audio data, is responsible for obtaining the split audio data from the local media framework and sending it to the hardware virtualization receiving service. And the hardware virtualization receiving service, as the receiver and player of the audio data, is responsible for receiving and processing the split audio data transmitted and playing the transferred audio data according to the custom playback policy.

[0139] In this way, after the mobile terminal is interconnected with the audio playback device, the media framework can transmit the split audio data from the mobile terminal to the audio playback device via the hardware virtualization sending service, so as to realize the splitting and transfer of audio data from the mobile terminal to the audio playback device for playback.

[0140] As Figure 7 shown, taking a mobile phone and a car head unit as an example, a flowchart of cross-device audio playback in an interconnected scenario is shown.

[0141] Refer to Figure 7, on the mobile phone side, audio applications such as music applications and navigation applications still send audio data requests to the media framework to play audio. Different from Figure 6 , after the media framework of the mobile phone receives the audio playback request, it no longer directly transmits the audio data to the mobile phone speaker via the audio HAL and audio driver of the mobile phone. Instead, after splitting the audio data, it transmits the split audio data to the in-vehicle infotainment system (IVI) interconnected with the mobile phone via the hardware virtualization sending service and the interconnection service. Among them, the interconnection service corresponds to the interconnection application and is the service activated after the mobile phone and the IVI are connected through the interconnection application.

[0142] , on the IVI side, after receiving the audio data split and transmitted from the mobile phone through the hardware virtualization receiving service, it sends an audio data request to the media framework of the IVI to play audio. The media framework of the IVI can then send the audio data to the audio hardware of the IVI, that is, to the IVI speaker, via the audio HAL and audio driver on the IVI. The IVI speaker plays the music audio and navigation audio.

[0143] Additionally, if there are playback policies customized by the vehicle manufacturer or the user on the IVI side, the hardware virtualization receiving service can obtain the customized playback policy from the interconnection application based on the audio type. Then, it sends the playback policy along with the audio data to the media framework, and subsequently, the media framework drives the IVI speaker to play the audio data transferred from the mobile phone based on this playback policy.

[0144] In some embodiments, the hardware virtualization service established between the mobile terminal and the audio playback device may include a virtual audio HAL, an audio sending module, and an audio receiving module. The virtual audio HAL and the audio sending module are on the mobile terminal side, and the audio receiving module is on the audio playback device side. That is, the hardware virtualization sending service includes the virtual audio HAL and the audio sending module, and the hardware virtualization receiving service includes the audio receiving module.

[0145] Among them, the virtual audio HAL is responsible for splitting and receiving audio data from the established audio path of the media framework, and then splitting and forwarding the audio data to the upper-layer audio sending module. After receiving the audio data from the virtual audio HAL, the audio sending module encapsulates the audio data into audio data packets and adds an audio type identifier to each audio data packet.

[0146] Then, the audio sending module sends the audio data packets to the audio playback device, that is, to the corresponding audio receiving module, through the transmission channel established by the interconnection service.

[0147] After the audio receiving module receives the audio data packet sent by the audio sending module, it parses the audio data packet and determines the audio type of the audio data according to the audio type identifier carried in the audio data packet. Then, the audio receiving module obtains the custom playback policy for this audio type from the interconnected application, configures the player, and plays the received audio data according to the custom playback policy.

[0148] That is to say, in the embodiment of this application, after the mobile terminal shunts and transmits the audio data to the audio sending module according to the audio type via the media framework and the virtual audio HAL. Since the audio sending module receives different audio data from different audio channels, and different audio channels correspond to different audio types, the audio sending module can clearly know the audio type of each audio data. Furthermore, in order to enable the audio playback device to also clearly perceive the audio type of each audio data, when the audio sending module encapsulates and packs the audio data for transmission, an audio type identifier is correspondingly added to the audio data packet. In this way, each audio data packet corresponding to each piece of audio data carries an audio type identifier representing the audio type corresponding to this audio data. Thus, after the audio receiving module receives the audio data packet transmitted by the audio sending module, it can determine the audio type of this audio data through the audio type identifier in the audio data packet.

[0149] Such as Figure 8 shown, taking a mobile phone and a car radio as an example, a flowchart of cross-device audio playback in another interconnected scenario is shown. Hereinafter, the audio shunting in the cross-device audio playback method will be explained in detail in combination with Figure 8 this.

[0150] When the media framework receives audio playback requests from a music application, a navigation application, and other applications (such as system applications), the media framework establishes a corresponding number of audio channels according to the audio type, and respectively allocates corresponding audio channels for music audio, navigation audio, and other audio.

[0151] Referring to Figure 8 , the audio channels established by the media framework are Channel 1, Channel 2, and Channel 3. Exemplarily, music audio can be allocated to Channel 1, navigation audio can be allocated to Channel 2, and other audio can all be allocated to Channel 3.

[0152] Then, the media framework shunts and transmits the music audio, navigation audio, and other audio to the audio sending module of the hardware virtualization sending service via the virtual audio HAL through the audio channels specified for the music audio, navigation audio, and other audio. It can be understood that the virtual audio HAL monitors Channel 1, Channel 2, and Channel 3 established by the media framework, and obtains the audio data from these three channels and transmits it to the audio sending module.

[0153] After the audio sending module receives the audio data shunted and sent by the virtual audio HAL, it encapsulates the audio data into data packets and adds an audio type identifier to the data packets to obtain audio data packets carrying the audio type identifier. Then, it transmits the audio data packets to the audio receiving module on the in-vehicle side through the interconnection service.

[0154] After the audio receiving module on the in-vehicle side receives the audio data, it parses the audio data packets, extracts the audio data in each audio data packet and the audio type identifier corresponding to this audio data. After the audio receiving module determines the audio type according to the audio type identifier, it obtains the corresponding playback policy from the interconnection application. Then, it sends the playback policy and the parsed audio data to the media framework to request audio playback.

[0155] After the media framework on the in-vehicle side receives the playback policy and the audio data, it can determine how to play the audio data specifically. For example, the media framework can determine the volume of audio playback, the speaker for playing the audio, etc. Furthermore, the media framework transmits the audio to the corresponding in-vehicle speaker for playback through the audio HAL and the audio driver in the in-vehicle system. For example, Figure 8 is shown as transmitting the audio to the in-vehicle speaker 1 for playback.

[0156] In some embodiments, the encapsulation of the audio data packets can follow any existing audio transmission protocol. For example, the audio data can be encapsulated into audio data packets following the Real-time Transport Protocol (RTP).

[0157] Currently, RTP audio packets usually pack 20 ms of audio data. Taking the RTP protocol as an example, Figure 9 shows a schematic diagram of an audio data packet format. Among them, type(0), type(1), and type(2) are audio type identifiers, representing the audio types of the audio data. For example, 0 can represent other audio except music audio and navigation audio, 1 can represent navigation audio, and 2 can represent music audio.

[0158] In addition, in the embodiments of the present application, the connection between the mobile terminal and the audio playback device is established through the interconnection application, mainly by establishing a peer to peer lending (P2P) transmission channel for interconnection. Therefore, the audio sending module can encrypt and transmit the audio data packets to the in-vehicle system through the established P2P transmission channel.

[0159] In some embodiments, when the mobile terminal and the audio playback device are disconnected from each other, in order to ensure that the audio of the mobile terminal can be played on its own audio hardware, it is necessary to enable the hardware virtualization service. That is, the mobile terminal turns off the hardware virtualization service, so that the media framework follows the Figure 6 process shown, and transmits the audio data to the speaker for playback via the audio HAL and the audio driver.

[0160] In some embodiments, based on different service requirements, there may be different audio splitting strategies. For example, it may be necessary to increase or decrease the audio path based on actual service requirements. Therefore, the configured audio splitting strategy may need to be frequently maintained. Thus, in order to facilitate the management and configuration of different audio splitting strategies. The decision-making for audio splitting can be transferred out of the media framework, and an additional splitting service that is convenient for developers to maintain is added to implement the decision-making for audio splitting.

[0161] As Figure 10 shown, a flowchart of cross-device audio playback in another interconnection scenario is shown. Hereinafter, the audio splitting in the embodiments of the present application will be explained in detail in combination with Figure 10 this.

[0162] After the media framework receives an audio playback request from an audio application, the media framework requests the splitting service to allocate an audio path for this audio data according to the audio type of the audio data. After receiving the splitting request from the media framework, the splitting service determines the path identifier (Identity document, ID) of this audio data according to the predetermined splitting rule (audio splitting strategy) according to the audio type of the audio data, and returns this path ID to the media framework.

[0163] Then, based on the indication of the splitting service, the media framework establishes an audio path corresponding to the path ID specified by the splitting service, and transmits the audio data to the virtual audio HAL via this audio path. The virtual audio HAL then transmits the audio data to the audio sending module. Then, the audio sending module encapsulates and packages the audio data, and adds an audio type identifier before transmitting it to the interconnected audio playback device.

[0164] For example, for music audio, the splitting service can specify that the media framework establish path 1 for transmission according to the predetermined splitting rule. Similarly, for navigation audio, the splitting service can specify that the media framework establish path 2 for transmission according to the predetermined splitting rule. For other audio, the splitting service can specify that the media framework establish path 3 for transmission.

[0165] Of course, it can be understood that if the audio path corresponding to the path ID indicated by the splitting service has been established by the media framework, the media framework can omit the process of establishing the path and directly transmit the audio data through the audio path.

[0166] Therefore, when there is a need to change the audio splitting strategy in actual business requirements, developers can directly reconfigure the splitting rules within the splitting service without having to modify the media framework, thereby improving the configuration efficiency. Subsequently, the splitting service allocates audio channels for audio data based on the updated splitting rules to achieve the splitting transmission and playback of different types of audio.

[0167] As Figure 11 shown, an interaction flowchart of a cross-device audio playback method is illustrated. Figure 11 The shown interaction flowchart is mainly the interaction process of each module in the mobile terminal, which is mainly divided into an interconnection process, an audio splitting transmission process, and a disconnection process.

[0168] Interconnection process: During the process of establishing an interconnection between the mobile terminal and the audio playback device, the splitting service first activates the hardware virtualization sending service. That is, it activates the virtual audio HAL and the audio sending module. On the side of the audio playback device, the hardware virtualization receiving service is also activated, that is, the audio receiving module is activated. Then, the audio sending module and the audio receiving module perform session negotiation to prepare for audio transmission. The specific process of session negotiation can refer to any existing session negotiation process, and the embodiments of the present application do not make any limitations on this.

[0169] Meanwhile, the splitting service registers an audio channel allocation callback function and a virtual audio HAL with the media framework. Registering the virtual audio HAL is to inform the media framework that a logical channel can be established with the virtual audio HAL so that audio data can be sent to the virtual audio HAL through this channel. After establishing the interconnection, there will be both an audio HAL and a virtual audio HAL in the mobile terminal. Therefore, in order to let the media framework clearly know whether the audio data should be sent to the audio HAL or the virtual audio HAL, the splitting service registers an audio channel allocation callback function with the media framework.

[0170] In the case where the splitting service has registered an audio channel allocation callback function with the media framework, if the media framework receives an audio playback request from an audio application, the media framework will first execute this audio channel allocation callback function and request splitting from the splitting service. This means that the media framework will transmit the received audio data to the virtual audio HAL for transfer to the interconnected audio playback device for playback.

[0171] In the case where the media framework has not registered an audio channel allocation callback function, if the media framework receives an audio playback request, it will not request splitting from the splitting service. The media framework will directly transmit the audio data to the audio HAL and play it through the audio driver to the speaker of the mobile terminal.

[0172] Audio Splitting and Transmission Process: After the mobile terminal establishes a connection with the audio playback device, if the audio application requests to play audio from the media framework, it triggers the media framework to execute the audio path allocation callback function and request splitting from the splitting service. In the embodiments of the present application, when the media framework requests splitting from the splitting service, in order to enable the splitting service to specify the audio path based on the audio type, parameters such as the user ID (UID) of the audio application, the process ID (PID), and the audio type (stream_type) of the audio data can be carried in the splitting request. Then, the splitting service determines the path ID corresponding to the audio data of the splitting request according to the predetermined splitting rules. The splitting service returns the path ID to the media framework.

[0173] After the media framework receives the path ID feedback from the splitting service, it establishes an audio path corresponding to the path ID. For example, it can establish Path 1, Path 2, or Path 3. The media framework transmits the audio data to the audio sending module via the established audio path through the virtual audio HAL. After the audio sending module receives the audio data transmitted through different audio paths, it can determine the audio type of this audio data according to the transmitted audio path. Then, the audio sending module encapsulates the audio data into audio data packets and adds the corresponding audio type identifier to the packet header. After adding type(0), type(1), or type(2), for example, it sends the packets to the audio receiving module.

[0174] In addition, it can be understood that since the mobile terminal and the audio playback device are connected through an interconnected application, when the audio sending module sends audio data to the audio receiving module, it is transmitted through the P2P channel corresponding to the interconnected service. This step Figure 11 is not shown.

[0175] Disconnection Process: After the mobile terminal disconnects from the audio playback device, the established hardware virtualization service is no longer useful. Therefore, the splitting service can close the hardware virtualization sending service on its own side. That is, it closes the virtual audio HAL and the audio sending module. On the audio playback device side, it will also close the hardware virtualization receiving service, that is, close the audio receiving module. At the same time, the splitting service unregisters the registered audio path allocation callback function and the virtual audio HAL from the media framework.

[0176] Subsequently, if the media framework receives an audio playback request, it will directly transmit it to the audio HAL to drive the local audio hardware to play the audio. Thus, the mobile terminal in the embodiments of the present application transmits audio data to the interconnected audio playback device by splitting the audio data according to the audio type, ensuring that the interconnected audio playback device can know the audio type of the audio data, and then using the playback strategy corresponding to this audio type to play the audio data, guaranteeing the smooth effectiveness of the playback strategy.

[0177] Such asFigure 12 As shown, an interaction flowchart of another cross-device audio playback method is presented. Figure 12 The interaction process shown is mainly the interaction process of each module in the audio playback device.

[0178] When the audio playback device establishes an interconnection with the mobile terminal through the interconnection application, the interconnection application in the audio playback device needs to first initialize the interface of the interconnection service software development kit (SDK) corresponding to the mobile terminal.

[0179] This interconnection service SDK interface is integrated in the audio playback device. According to the actual interconnection requirements of the audio playback device, it can generally integrate the interconnection service SDK interfaces provided by multiple different brands of mobile terminals. Furthermore, when the audio playback device selects to establish an interconnection with a certain brand of mobile terminal, it initializes the SDK interface corresponding to this brand of mobile terminal, and through this interconnection service SDK interface, it can call the interconnection-related services provided by the connected mobile terminal.

[0180] It can be simply understood that the interconnection service SDK interface is the interface provided by the mobile terminal for the audio playback device to call the interconnection service. Conversely, it means that the audio playback device can call the interconnection service provided by the mobile terminal through the interconnection service SDK interface.

[0181] Generally speaking, the interconnection service SDK interface is a set of library functions that do not run actively and need to be loaded and run by other modules. And the interconnection service SDK interface is mainly used to call the interfaces related to the interconnection service, and in the embodiments of this application, the device interconnection is achieved through the interconnection application, so the interconnection application can be used to initialize this interconnection service SDK interface.

[0182] Therefore, in order to enable the audio playback device to play audio according to the customized playback strategy. After the interconnection application on the audio playback device initializes the interconnection service SDK interface, it registers a playback strategy callback function with the audio receiving module in the hardware virtualization service through this interconnection service SDK interface. Through the registered playback strategy callback function, after the audio receiving module obtains the audio data, it triggers the audio receiving module to obtain the customized playback strategy on the audio playback device side.

[0183] Furthermore, when the audio receiving module receives the audio data packet sent by the audio sending module, the audio receiving module first parses this data packet to obtain the audio data and determines the audio type of this audio data according to the audio data identifier in the audio data packet. Then, the audio receiving module executes the playback strategy callback function and obtains the playback strategy defined by the audio playback device for this audio type from the interconnection application through this interconnection service SDK interface.

[0184] After the audio receiving module obtains the playback strategy of the audio data, it further sends the audio data and the playback strategy to the media framework to request audio playback. The media framework then transmits the audio data to the speaker for audio playback according to the playback strategy. For example, the media framework can determine the playback mode of the audio data according to the playback strategy, including the playback volume, the specified speaker for playback, etc., and then drive the specified speaker to play the audio at the specified playback volume.

[0185] Thus, the audio playback device can implement playing the specified type of audio according to the custom playback strategy, thereby ensuring the smooth effectiveness of the custom playback strategy.

[0186] As Figure 13 shown, it is a schematic structural diagram of a mobile terminal 1300 provided by an embodiment of the present application.

[0187] The mobile terminal 1300 may include a processor 1310, an external memory interface 1320, an internal memory 1321, a universal serial bus (USB) connector 1330, a charging management module 1340, a power management module 1341, a battery 1342, an antenna 1, an antenna 2, a mobile communication module 1350, a wireless communication module 1360, an audio module 1370, a speaker 1370A, a receiver 1370B, a microphone 1370C, a headphone jack 1370D, a sensor module 1380, a button 1390, a motor 1391, an indicator 1392, a camera module 1393, a display screen 1394, and a subscriber identification module (SIM) card interface 1395, etc. The sensor module 1380 may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0188] It can be understood that the structure schematically shown in the embodiment of the present application does not constitute a specific limitation on the mobile terminal 1300. In other embodiments of the present application, the mobile terminal 1300 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0189] The processor 1310 may include one or more processing units. For example, the processor 1310 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. The processor 1310 may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions. Specifically, in the embodiments of the present application, the processor 1310 may implement the cross-device audio playback method described in any one of the above embodiments.

[0190] A memory may also be provided in the processor 1310 for storing instructions and data. In some embodiments, the memory in the processor 1310 may be a cache memory. This memory may save the instructions or data that have been used by the processor 1310 or are used frequently. If the processor 1310 needs to use this instruction or data, it can be directly called from this memory. This avoids repeated accesses, reduces the waiting time of the processor 1310, and thus improves the efficiency of the system.

[0191] In some embodiments, the processor 1310 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. The processor 1310 may be connected to modules such as a touch sensor, an audio module, a wireless communication module, a display screen, a camera module, etc. through at least one of the above interfaces.

[0192] It can be understood that the interface connection relationships between the modules illustrated in the embodiments of the present application are only illustrative descriptions and do not constitute a structural limitation on the mobile terminal 1300. In other embodiments of the present application, the mobile terminal 1300 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0193] The external memory interface 1320 may be used to connect an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the mobile terminal 1300. The external memory card communicates with the processor 1310 through the external memory interface 1320 to implement the data storage function. For example, files such as music and videos are saved in the external memory card. Or files such as music and videos are transferred from the mobile terminal to the external memory card.

[0194] The internal memory 1321 can be used to store computer-executable program codes, which include instructions. The internal memory 1321 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the mobile terminal 1300 (such as audio data, phone book, etc.). In addition, the internal memory 1321 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 1310 executes various functional methods or data processing of the mobile terminal 1300 by running the instructions stored in the internal memory 1321 and / or the instructions stored in the memory provided in the processor.

[0195] The USB connector 1330 is an interface that conforms to the USB standard specification and can be used to connect the mobile terminal 1300 and peripheral devices, specifically, it can be a Mini USB connector, a Micro USB connector, a USB Type C connector, etc. For example, it can be connected to a charger to charge the mobile terminal 1300. Connect other electronic devices for data transmission. It can also be connected to headphones to output the audio in the mobile terminal 1300 through the headphones.

[0196] The charging management module 1340 is used to receive the charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. The power management module 1341 is used to connect the battery 1342, the charging management module 1340, and the processor 1310. The power management module 1341 receives the inputs from the battery 1342 and / or the charging management module 1340 and supplies power to the processor 1310, the internal memory 1321, the display screen 1394, the camera module 1393, the wireless communication module 1360, etc.

[0197] The wireless communication function of the mobile terminal 1300 can be implemented through antenna 1, antenna 2, the mobile communication module 1350, the wireless communication module 1360, the modulation and demodulation processor, and the baseband processor, etc. Among them, antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 1350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the mobile terminal 1300. The modulation and demodulation processor can include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing.

[0198] The wireless communication module 1360 can provide wireless communication solutions applied to the mobile terminal 1300, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Bluetooth low energy (BLE), ultra wide band (UWB), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0199] The mobile terminal 1300 can implement the display function through the GPU, the display screen 1394, and the application processor, etc. The GPU is a microprocessor for image processing, connecting the display screen 1394 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 1310 may include one or more GPUs, which execute program instructions to generate or change display information. Among them, the display screen 1394 is used to display images, videos, etc. The display screen 1394 includes a display panel. In some embodiments, the mobile terminal 1300 may include one or more display screens 1394.

[0200] The mobile terminal 1300 can implement the camera function through the camera module 1393, ISP, video codec, GPU, display screen 1394, application processor AP, neural network processor NPU, etc. Among them, the camera module 1393 can be used to collect color image data and depth data of the shooting object. In some embodiments, the camera module 1393 may be composed of a color camera module and a 3D sensing module. The mobile terminal 1300 may include one or more camera modules 1393. The camera module 1393 may also be composed of two or more cameras.

[0201] The mobile terminal 1300 can implement the audio function through the audio module 1370, speaker 1370A, receiver 1370B, microphone 1370C, headphone jack 1370D, and application processor, etc. Such as music playback, recording, etc.

[0202] The audio module 1370 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 1370 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 1370 can be disposed in the processor 1310, or some functional modules of the audio module 1370 can be disposed in the processor 1310.

[0203] The speaker 1370A, also known as the "horn", is used to convert an audio electrical signal into a sound signal. The mobile terminal 1300 can listen to music through the speaker 1370A, or output the audio signal of a hands-free call.

[0204] The receiver 1370B, also known as the "earpiece", is used to convert an audio electrical signal into a sound signal. When the mobile terminal 1300 answers a call or a voice message, the user can listen to the voice by bringing the receiver 1370B close to the ear.

[0205] The microphone 1370C, also known as the "microphone" or "transmitter", is used to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak by bringing the mouth close to the microphone 1370C to input the sound signal into the microphone 1370C. The mobile terminal 1300 can be provided with at least one microphone 1370C. In some other embodiments, the mobile terminal 1300 can be provided with two microphones 1370C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the mobile terminal 1300 can also be provided with three, four or more microphones 1370C, which can collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording.

[0206] In addition, the headphone jack 1370D is used to connect a wired headphone. The keys 1390 can include a power-on key, volume keys, etc. The keys 1390 can be mechanical keys or touch keys. The motor 1391 can generate a vibration prompt for an incoming call vibration prompt or for touch vibration feedback. The indicator 1392 can be an indicator light, which can be used to indicate the charging state, power change, or can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 1395 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 1395 to achieve contact and separation from the mobile terminal 1300. The mobile terminal 1300 can support one or more SIM card interfaces.

[0207] Another embodiment of the present application provides a mobile terminal, including: one or more processors and a memory, the memory being coupled to the processors; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processors execute the computer instructions, the mobile terminal implements the cross-device audio playback method described in the above embodiments.

[0208] Another embodiment of the present application provides an audio playback device, including: audio hardware, one or more processors and a memory, the audio hardware and the memory are respectively coupled to the processors; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processors execute the computer instructions, the audio playback device implements the cross-device audio playback method described in the above embodiments.

[0209] In some embodiments, the audio hardware of the audio playback device includes a speaker.

[0210] Another embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor in a mobile terminal or an audio playback device, the mobile terminal or the audio playback device implements the cross-device audio playback method described in the above embodiments.

[0211] The embodiment of the present application also provides a computer program product. When the computer program product runs on a computer, the computer executes each function or step in the above method embodiments.

[0212] The embodiment of the present application also provides a chip system, as Figure 14 shown. The chip system 140 includes at least one processor 1401 and at least one interface circuit 1402. The processor 1401 and the interface circuit 1402 can be interconnected by a line. For example, the interface circuit 1402 can be used to receive signals from other devices (such as the memory of a computer). For another example, the interface circuit 1402 can be used to send signals to other devices (such as the processor 1401).

[0213] Exemplarily, the interface circuit 1402 can read the instructions stored in the memory and send the instructions to the processor 1401. When the instructions are executed by the processor 1401, the computer can execute each step in the above embodiments. Of course, the chip system may also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0214] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0215] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0216] The unit described as a separated component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, that is, it can be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0218] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: USB flash drive, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk or optical disc and other various media that can store program codes.

[0219] The above content is only a specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application shall be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

1. A cross-device audio playback method, characterized in that, Applied to a mobile terminal, the mobile terminal establishes a connection with an audio playback device through an interconnected application; the method includes: In response to an audio playback request of one or more audio applications, according to the audio type of the audio data corresponding to the audio playback request, allocate a corresponding audio path for the audio data; wherein, the audio path corresponds to the audio type, and different audio paths are used to transmit audio data of different audio types; Transmit the audio data to the audio playback device through the audio path corresponding to the audio type, and the audio playback device plays the audio data according to the playback strategy corresponding to the audio type.

2. The method according to claim 1, characterized in that The audio playback device includes a vehicle, and the mobile terminal establishes a connection with the in-vehicle unit of the vehicle through the interconnected application.

3. The method according to claim 1 or 2, characterized in that, The audio paths include a music audio path, a navigation audio path, and other audio paths; wherein, the music audio path is used to independently transmit music audio; the navigation audio path is used to independently transmit navigation audio; the other audio paths are used to transmit any one or more audio other than the music audio and the navigation audio.

4. The method according to any one of claims 1 to 3, characterized in that The allocating a corresponding audio path for the audio data according to the audio type of the audio data corresponding to the audio playback request includes: Determine the path identifier corresponding to the audio type according to a predetermined shunting rule; Establish an audio path corresponding to the path identifier, and allocate the audio data to the corresponding audio path according to the audio type.

5. The method according to claim 4, wherein The mobile terminal includes a media framework and a shunting service; the determining the path identifier corresponding to the audio type according to a predetermined shunting rule; The establishing an audio path corresponding to the path identifier and allocating the audio data to the corresponding audio path according to the audio type includes: The media framework executes an audio path allocation callback function and sends a shunting request carrying the audio type to the shunting service; wherein, the audio path allocation callback function is registered by the shunting service to the media framework after the mobile terminal establishes a connection with the audio playback device through the interconnected application; The shunting service responds to the shunting request, determines the path identifier corresponding to the audio type according to a predetermined shunting rule, and returns it to the media framework; The media framework establishes an audio path corresponding to the path identifier, and allocates the audio data to the corresponding audio path according to the audio type.

6. The method according to any one of claims 1-5, characterized in that, The mobile terminal includes a media framework and a hardware virtualization sending service; wherein, the hardware virtualization sending service is activated after the mobile terminal establishes a connection with the audio playback device through the interconnected application; the transmitting the audio data to the audio playback device through the audio path corresponding to the audio type includes: The media framework transmits the audio data to the hardware virtualization sending service through the audio path corresponding to the audio data; The hardware virtualization sending service encapsulates the audio data into audio data packets and transmits them to the audio playback device; wherein, the audio data packets carry audio type identifiers corresponding to the audio types of the audio data; the audio playback device parses the audio data packets, determines the audio type corresponding to the audio data according to the audio type identifiers, and plays the audio data according to the playback strategy corresponding to the audio type.

7. The method according to claim 6, wherein The hardware virtualization sending service includes a virtual audio HAL and an audio sending module; The hardware virtualization sending service encapsulates the audio data into audio data packets and transmits them to the audio playback device, including: The audio sending module encapsulates the audio data into audio data packets and transmits them to the audio playback device; wherein, the audio sending module receives the audio data transmitted by the media framework through the audio path via the virtual audio HAL.

8. The method according to any one of claims 1 to 7, characterized in that The mobile terminal establishes a connection with the audio playback device through an interconnected application, including: the mobile terminal establishes a P2P transmission channel with the audio playback device through the interconnected application.

9. The method according to any one of claims 1 to 8, characterized in that, The mobile terminal establishes a connection with the audio playback device through an interconnected application, including: In response to a connection broadcast sent by the audio playback device, display a first connection interface; wherein, the connection broadcast is generated by the audio playback device in response to a connection operation by the user within the interconnected application; Receive a connection code input by the user on the first connection interface and establish a connection with the audio playback device; wherein, the connection code is generated by the audio playback device and displayed on a second connection interface of the audio playback device.

10. A cross-device audio playback method, characterized in that, Applied to an audio playback device, the audio playback device establishes a connection with a mobile terminal through an interconnected application; the method includes: Receive an audio data packet from the mobile terminal, and parse the audio data packet to obtain the audio data and the audio type identifier of the audio data; Determine the audio type of the audio data according to the audio type identifier, obtain the corresponding playback strategy according to the audio type, and play the audio data according to the playback strategy.

11. The method according to claim 10, wherein The audio playback device includes a vehicle, and the in-vehicle computer of the vehicle establishes a connection with the mobile terminal through the interconnected application; the in-vehicle computer of the vehicle establishes a P2P transmission channel with the mobile terminal through the interconnected application.

12. The method according to claim 10 or 11, characterized in that The audio playback device includes a hardware virtualization receiving service, and the hardware virtualization receiving service is activated after the audio playback device establishes a connection with the mobile terminal through the interconnected application; The receiving the audio data packet from the mobile terminal and parsing the audio data packet to obtain the audio data and the audio type identifier of the audio data, including: The hardware virtualization receiving service receives the audio data packet from the mobile terminal and parses the audio data packet to obtain the audio data and the audio type identifier of the audio data; The hardware virtualization receiving service determines the audio type of the audio data according to the audio type identifier, obtains the corresponding playback strategy according to the audio type, and plays the audio data according to the playback strategy.

13. The method according to claim 12, wherein Obtaining a corresponding playback policy according to the audio type includes: The hardware virtualization receiving service executes a playback policy callback function to obtain a corresponding playback policy from the interconnected application according to the audio type; wherein, the playback policy callback function is registered by the interconnected application to the hardware virtualization receiving service through the interconnected service SDK interface after the audio playback device establishes a connection with the mobile terminal through the interconnected application.

14. The method according to any one of claims 10-13, characterized in that, The audio playback device establishing a connection with the mobile terminal through the interconnected application includes: Responding to a user's operation of opening the interconnected application, displaying an interconnected application interface; Responding to a user's brand selection operation within the interconnected application interface, displaying a second connection interface including a connection code, and generating a connection broadcast to be sent to the mobile terminal of the brand corresponding to the brand selection operation; the mobile terminal responds to the connection broadcast to display a first connection interface, and receives the connection code input by the user in the first connection interface to establish a connection with the audio playback device.

15. A mobile terminal, characterized in that, Including: One or more processors and a memory, the memory being coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the mobile terminal executes the cross-device audio playback method according to any one of claims 1-9.

16. An audio playback device, characterized in that, Including: Audio hardware, one or more processors and a memory, the audio hardware and the memory are respectively coupled to the processor; one or more computer program codes are stored in the memory, and the computer program codes include computer instructions; when the processor executes the computer instructions, the audio playback device executes the cross-device audio playback method according to any one of claims 10-14.

17. The audio playback device according to claim 16, wherein, The audio hardware includes a speaker.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the mobile terminal, the mobile terminal executes the cross-device audio playback method according to any one of claims 1-9; when the computer program is executed by the processor of the audio playback device, the audio playback device executes the cross-device audio playback method according to any one of claims 10-14.

Images