Audio playing method, electronic equipment, readable storage medium and chip system

Abstract

The invention discloses an audio playing method, electronic equipment, a readable storage medium and a chip system, and the method can comprise the steps: receiving a first request from first equipment based on an established first screen projection connection, and receiving a second request from second equipment based on an established second screen projection connection; creating a first audio processing module and a second audio processing module through a virtual speaker service of the third device in response to the first request and the second request; processing the audio data from the first equipment based on the first audio processing module to obtain first audio data, and processing the audio data from the second equipment based on the second audio processing module to obtain second audio data; and calling a loudspeaker of the third device to play the first audio data and the second audio data. By adopting the method and the device, the audio data of the plurality of slave devices can be flexibly played by using the loudspeaker of the master device in a multi-device screen projection scene.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and in particular, to an audio playing method, an electronic device, a readable storage medium, and a chip system. Background Art

[0002] With the rapid development of intelligent terminal technologies, screen mirroring technologies have emerged. Screen mirroring technologies support mirror display of the screen of one device (such as a mobile phone) in the interface of another device (such as a computer). Currently, multi-device screen mirroring technologies can support mirror display of the screens of multiple slave devices in the interface of a master device respectively. For example, the screens of a first device (such as mobile phone A) and a second device (such as mobile phone B) can both be mirror-displayed in the interface of a third device (such as a computer). In such a multi-device screen mirroring scenario, slave devices often have a need to play their own audio data through the master device. Therefore, in a multi-device screen mirroring scenario, how the master device plays the audio data of multiple slave devices is a technical problem worthy of study. Summary of the Invention

[0003] Embodiments of this application provide an audio playing method, an electronic device, a readable storage medium, and a chip system, which can flexibly play the audio data of multiple slave devices by using the speaker of a master device in a multi-device screen mirroring scenario.

[0004] In a first aspect, this application provides an audio playing method, which can be applied to a third device. The method includes: receiving a first request from a first device based on an established first screen mirroring connection, and receiving a second request from a second device based on an established second screen mirroring connection; the first request is used to request the third device to play audio data from the first device; the second request is used to request the third device to play audio data from the second device; in response to the first request and the second request, creating a first audio processing module and a second audio processing module through the virtual speaker service of the third device; processing the audio data from the first device based on the first audio processing module to obtain first audio data, and processing the audio data from the second device based on the second audio processing module to obtain second audio data; calling the speaker of the third device to play the first audio data and the second audio data; where the first screen mirroring connection is a screen mirroring connection between the third device and the first device; the second screen mirroring connection is a screen mirroring connection between the third device and the second device.

[0005] Among them, the third device can be understood as the master device, and the first device and the second device can be understood as slave devices.

[0006] In the above embodiments, in a multi-device screen mirroring scenario, the third device can respectively create a first audio processing module for processing the audio data of the first device and a second audio processing module for processing the audio data of the second device through the virtual speaker service, so as to independently process the audio data of different devices simultaneously, avoid resource conflicts, and can call the speaker to play the audio data of different devices at the same time. In this way, it is beneficial to improve the flexibility and intelligence of audio playback.

[0007] In combination with the first aspect, in a possible way, creating the first audio processing module and the second audio processing module through the virtual speaker service of the third device includes: calling the audio manager through the virtual speaker service of the third device to create the first audio processing module and the second audio processing module; the audio manager is used to create and call at least two audio processing modules. It can be seen that by calling the audio manager, efficient management of the audio processing of different devices can be achieved.

[0008] In combination with the first aspect, in a possible way, creating the first audio processing module and the second audio processing module by calling the audio manager through the audio processing interface of the virtual speaker service of the third device includes: calling the audio manager through the audio processing interface of the virtual speaker service of the third device to create the first audio processing module and the second audio processing module. It can be seen that the audio manager can be called more conveniently through the audio processing interface.

[0009] In combination with the first aspect, in a possible way, the first request may carry first identification information, and the second request may carry second identification information; creating the first audio processing module and the second audio processing module by calling the audio manager through the audio processing interface of the virtual speaker service of the third device includes: based on the first identification information, calling the audio manager through the audio processing interface of the virtual speaker service of the third device to create the first audio processing module; and based on the second identification information, calling the audio manager through the audio processing interface of the virtual speaker service of the third device to create the second audio processing module. It can be seen that based on the first identification information and the second identification information, the first audio processing module and the second audio processing module can be created more accurately.

[0010] In combination with the first aspect, in a possible manner, the first audio processing module may include a first decoding module, and the second audio processing module may include a second decoding module; the above-mentioned processing of the audio data from the first device by the first audio processing module to obtain first audio data, and the processing of the audio data from the second device by the second audio processing module to obtain second audio data includes: decoding the audio data from the first device by the first decoding module to obtain first audio data, and decoding the audio data from the second device by the second decoding module to obtain second audio data. It can be seen that the first decoding module and the second decoding module can simultaneously decode the audio data from two different devices, which is beneficial to improving the efficiency of audio processing.

[0011] In combination with the first aspect, in a possible manner, the above-mentioned decoding the audio data from the first device by the first decoding module to obtain first audio data, and decoding the audio data from the second device by the second decoding module to obtain second audio data includes: calling the audio manager through the virtual speaker service of the third device to write the audio data of the first device into the first decoding queue, obtaining the audio data of the first device from the first decoding queue based on the first decoding module, decoding the audio data of the first device to obtain first audio data, and calling the audio manager to write the audio data of the second device into the second decoding queue, obtaining the audio data of the second device from the second decoding queue based on the second decoding module, and decoding the audio data of the second device to obtain second audio data. It can be seen that calling the audio manager to write the audio data of the first device into the first decoding queue and the audio data of the second device into the second decoding queue respectively can enable the audio manager not to wait for decoding to complete, thus avoiding audio stuttering.

[0012] In combination with the first aspect, in a possible manner, the above-mentioned calling the audio manager to write the audio data of the first device into the first decoding queue includes: calling the audio manager through the audio processing interface of the virtual speaker service of the third device to write the audio data of the first device into the first decoding queue; the above-mentioned calling the audio manager to write the audio data of the second device into the second decoding queue includes: calling the audio manager through the audio processing interface of the virtual speaker service of the third device to write the audio data of the second device into the second decoding queue. It can be seen that through the audio processing interface, it is more convenient to call the audio manager to write the audio data of the first device into the first decoding queue and the audio data of the second device into the second decoding queue.

[0013] In combination with the first aspect, in a possible manner, the first audio processing module may further include a first playback module, and the second audio processing module may further include a second playback module; the above-mentioned calling the speaker of the third device to play the first audio data and the second audio data includes: calling the speaker of the third device to play the first audio data through the first playback module, and calling the speaker of the third device to play the second audio data through the second playback module. It can be seen that the first audio data and the second audio data can be played on the speaker of the third device simultaneously through the first playback module and the second playback module, which is beneficial to improving the flexibility of audio playback.

[0014] In combination with the first aspect, in a possible manner, the above-mentioned playing the first audio data by calling the speaker of the third device based on the first playback module and playing the second audio data by calling the speaker of the third device based on the second playback module includes: calling the speaker of the third device to play the first audio data at a first volume based on the first playback module, and calling the speaker of the third device to play the second audio data at a second volume based on the second playback module; wherein, the first volume is higher than the second volume, and the reception time of the first request is earlier than the reception time of the second request. It can be seen that controlling the volume of audio data based on the received request time is beneficial to improving the intelligence of audio playback.

[0015] In combination with the first aspect, in a possible manner, the above-mentioned method further includes: displaying the audio playback interface of the first device and the audio playback interface of the second device on the screen mirroring interface of the third device. It can be seen that by displaying the audio playback interfaces of the first device and the second device, it is convenient to more intuitively display the audio playback effects of the first device and the second device.

[0016] In combination with the first aspect, in a possible manner, the above-mentioned method further includes: receiving a close request from the first device; the close request is used to request the third device to stop playing the first audio data; based on the close request, deleting the first audio processing module through the virtual speaker service and stopping calling the speaker of the third device to play the first audio data. It can be seen that deleting the first audio processing module through the virtual speaker service when receiving the close request is beneficial to saving memory overhead.

[0017] Second aspect, an embodiment of the present application provides an electronic device, which includes: one or more processors, a display screen, and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, and the computer program code includes computer instructions. The one or more processors call the computer instructions to cause the electronic device to perform: based on an established first screen mirroring connection, receive a first request from a first device, and based on an established second screen mirroring connection, receive a second request from a second device; the first request is used to request a third device to play audio data from the first device; the second request is used to request the third device to play audio data from the second device; in response to the first request and the second request, create a first audio processing module and a second audio processing module through the virtual speaker service of the third device; process the audio data from the first device based on the first audio processing module to obtain first audio data, and process the audio data from the second device based on the second audio processing module to obtain second audio data; call the speaker of the third device to play the first audio data and the second audio data; wherein, the first screen mirroring connection is a screen mirroring connection between the third device and the first device; the second screen mirroring connection is a screen mirroring connection between the third device and the second device.

[0018] In combination with the second aspect, in a possible manner, the one or more processors call the computer instructions to cause the electronic device to perform: call an audio manager through the virtual speaker service of the third device to create a first audio processing module and a second audio processing module; the audio manager is used to create and call at least two audio processing modules.

[0019] In combination with the second aspect, in a possible manner, the one or more processors call the computer instructions to cause the electronic device to perform: call an audio manager through the audio processing interface of the virtual speaker service of the third device to create a first audio processing module and a second audio processing module.

[0020] In combination with the second aspect, in a possible manner, the first request may carry first identification information, and the second request may carry second identification information; the one or more processors call the computer instructions to cause the electronic device to perform: based on the first identification information, call an audio manager through the audio processing interface of the virtual speaker service of the third device to create a first audio processing module; and based on the second identification information, call an audio manager through the audio processing interface of the virtual speaker service of the third device to create a second audio processing module. It can be seen that based on the first identification information and the second identification information, the first audio processing module and the second audio processing module can be created more accurately.

[0021] In combination with the second aspect, in a possible manner, the first audio processing module may include a first decoding module, and the second audio processing module may include a second decoding module; the one or more processors call the computer instructions to cause the electronic device to perform: decoding and processing the audio data from the first device based on the first decoding module to obtain first audio data, and decoding and processing the audio data from the second device based on the second decoding module to obtain second audio data.

[0022] In combination with the second aspect, in a possible manner, the one or more processors call the computer instructions to cause the electronic device to perform: calling the audio manager through the virtual speaker service of the third device to write the audio data of the first device into the first decoding queue, obtaining the audio data of the first device from the first decoding queue based on the first decoding module, decoding and processing the audio data of the first device to obtain first audio data, and calling the audio manager to write the audio data of the second device into the second decoding queue, obtaining the audio data of the second device from the second decoding queue based on the second decoding module, and decoding and processing the audio data of the second device to obtain second audio data.

[0023] In combination with the second aspect, in a possible manner, the one or more processors call the computer instructions to cause the electronic device to perform: calling the audio manager through the audio processing interface of the virtual speaker service of the third device to write the audio data of the first device into the first decoding queue; calling the audio manager through the audio processing interface of the virtual speaker service of the third device to write the audio data of the second device into the second decoding queue.

[0024] In combination with the second aspect, in a possible manner, the first audio processing module may further include a first playback module, and the second audio processing module may further include a second playback module; the one or more processors call the computer instructions to cause the electronic device to perform: calling the speaker of the third device through the first playback module to play the first audio data, and calling the speaker of the third device through the second playback module to play the second audio data.

[0025] In combination with the second aspect, in a possible manner, the one or more processors call the computer instructions to cause the electronic device to perform: calling the speaker of the third device through the first playback module to play the first audio data at a first volume, and calling the speaker of the third device through the second playback module to play the second audio data at a second volume; wherein, the first volume is higher than the second volume, and the reception time of the first request is earlier than the reception time of the second request.

[0026] In combination with the second aspect, in a possible way, the one or more processors call the computer instructions to cause the electronic device to execute: display the audio playing interface of the first device and the audio playing interface of the second device in the screen mirroring interface of the third device.

[0027] In combination with the second aspect, in a possible way, the one or more processors call the computer instructions to cause the electronic device to execute: receive a close request from the first device; the close request is used to request the third device to stop playing the first audio data; based on the close request, delete the first audio processing module through the virtual speaker service and stop calling the speaker of the third device to play the first audio data.

[0028] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions to cause the electronic device to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides a computer program product containing instructions. When the computer program product runs on an electronic device, it causes the electronic device to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0030] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions. When the instructions run on an electronic device, it causes the electronic device to execute the method described in the first aspect or any possible implementation manner of the first aspect. Description of the Drawings

[0031] Figure 1A is a schematic diagram of a multi-device screen mirroring scenario provided by an embodiment of the present application;

[0032] Figure 1B is a schematic diagram of another multi-device screen mirroring scenario provided by an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0034] Figure 3 is a schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

[0035] Figure 4 is a schematic diagram of a system architecture applied to an embodiment of the present application;

[0036] Figure 5 is a schematic diagram of the flow of an audio playing method provided by an embodiment of the present application;

[0037] Figure 6 It is a schematic diagram of communication between a first device and a third device through the DMSDP service provided by an embodiment of the present application;

[0038] Figure 7 It is a schematic diagram of a third device processing audio data of different devices provided by an embodiment of the present application;

[0039] Figure 8 It is a schematic diagram of a screen mirroring interface of a third device provided by an embodiment of the present application;

[0040] Figure 9 It is a schematic flowchart of another audio playback method provided by an embodiment of the present application. Detailed implementation manners

[0041] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0042] The terms "first", "second", "third", etc. in the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a series of steps or units are included, or optionally, steps or units not listed are also included, or optionally, other steps or units inherent to these processes, methods, products or devices are also included.

[0043] In addition, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or similar expressions thereof refer to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, and c can represent: a, or b, or c, or a and b, or a and c, or b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0044] Next, the multi-device screen mirroring scenario involved in the embodiments of the present application will be described.

[0045] The multi-device screen mirroring scenario in the embodiments of this application is a scenario where the screens of multiple slave devices are mirrored and displayed in the interface of a master device. Among them, the method of mirroring multiple slave devices to the master device may include a collaborative screen mirroring method and / or a heterogeneous source screen mirroring method.

[0046] Please refer to Figure 1A , Figure 1A which is a schematic diagram of a multi-device screen mirroring scenario provided by the embodiments of this application. As Figure 1A shown, the screen of mobile phone 101 and the screen of mobile phone 102 can be respectively mirrored and displayed in the interface of computer 103. Among them, a part of the interface of computer 103 can be used to display the screen of mobile phone 101, and another part of the area can be used to display the screen of mobile phone 102. The screen displayed in a part of the interface of computer 103 can be the same as the screen of mobile phone 101, and the screen displayed in another part of the area can be the same as the screen of mobile phone 102. Computer 103 can be understood as the master device, and mobile phones 101 and 102 can be understood as slave devices. This screen mirroring method in which the screen displayed by the master device is always the same as the screen displayed by the slave device can be called the collaborative screen mirroring method.

[0047] For example, assume that mobile phone 101 uses the collaborative screen mirroring method to mirror its screen to computer 103, and mobile phone 102 also uses the collaborative screen mirroring method to mirror its screen to computer 103. If mobile phone 101 displays the screen of a shopping application and mobile phone 102 displays the screen of a conference application, then the interface of computer 103 displays the screen of the shopping application in mobile phone 101 and the screen of the conference application in mobile phone 102. If mobile phone 101 switches from displaying the screen of the shopping application to displaying the screen of a video application, then computer 103 also switches from displaying the screen of the shopping application in mobile phone 101 to displaying the screen of the video application in mobile phone 101. If mobile phone 102 switches from displaying the screen of the conference application to displaying the screen of a game application, then computer 103 also switches from displaying the screen of the conference application in mobile phone 102 to displaying the screen of the game application in mobile phone 102.

[0048] Please refer to Figure 1B , Figure 1B which is a schematic diagram of another multi-device screen mirroring scenario provided by the embodiments of this application. As Figure 1BAs shown in the figure, the mobile phone 101 can project the screen of a certain application (such as Application A) onto the interface of the computer 103 for display, and the mobile phone 102 can also project the screen of a certain application (such as Application B) onto the interface of the computer 103 for display. Among them, a part of the interface of the computer 103 can be used to display the screen of Application A in the mobile phone 101, and another part of the area can be used to display the screen of Application B in the mobile phone 102. The screen displayed in a part of the interface of the computer 103 can be the same as or different from the screen of the mobile phone 101, and the screen displayed in another part of the area can be the same as or different from the screen of the mobile phone 102. This way of projecting the screen of a certain application of the slave device onto the master device for display, that is, the master device can display a screen different from that of the slave device, can be called the heterogeneous screen projection method.

[0049] For example, assume that the mobile phone 101 projects the screen of the conference application onto the computer 103 using the heterogeneous screen projection method, and the mobile phone 102 projects the screen of the shopping application onto the computer 103 using the heterogeneous screen projection method. Then the computer 103 can display the screen of the conference application in the mobile phone 101 and the screen of the shopping application in the mobile phone 102. If the mobile phone 101 switches from displaying the screen of the conference application to displaying the screen of the call application, then the computer 103 keeps displaying the screen of the conference application in the mobile phone 101 without switching. Similarly, if the mobile phone 102 switches from displaying the screen of the shopping application to displaying the screen of the chat application, then the computer 103 keeps displaying the screen of the shopping application in the mobile phone 102 without switching.

[0050] It can be understood that the above two multi-device screen projection scenarios are only for illustration and do not constitute a limitation to this application. In some other multi-device screen projection scenarios, it can also be that some slave devices adopt the collaborative screen projection method and some slave devices adopt the heterogeneous screen projection method. For example, the above mobile phone 101 can adopt the collaborative screen projection method, and the mobile phone 102 can adopt the heterogeneous screen projection method. This application is not limited to this.

[0051] Currently, in this multi-device screen projection scenario, multiple slave devices often have the need to use the speaker of the master device to play audio data. For example, Figure 1B when the mobile phone 101 projects the screen of the music application onto the computer 103 and the mobile phone 102 projects the screen of the video application onto the computer 103, in this case, the mobile phone 101 has the need to use the speaker of the computer 103 to play the audio data of the music application, and the mobile phone 102 has the need to use the speaker of the computer 103 to play the audio data of the video application.

[0052] Based on this, the embodiments of this application provide an audio playback method, which can flexibly play the audio data of multiple slave devices using the speaker of the master device in a multi-device screen projection scenario.

[0053] Embodiments of the present application can be applied to electronic devices with audio playback functions. In some embodiments, the electronic device can be a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), a wearable electronic device, a smart watch, or the like.

[0054] Exemplarily, please refer to Figure 2 , Figure 2 which is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application. As Figure 2 shown, the electronic device 100 may include: 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna, a sensor module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a key 180, a display 190, etc.

[0055] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0056] Among them, the CPU is the final execution unit for information processing and program running. Its main tasks include processing instructions, executing operations, controlling time, and processing data, etc. The CPU may include a controller, an arithmetic unit, a cache memory, and a bus for connecting these components. The controller is the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of instruction fetching and execution. The arithmetic unit refers to the component that performs various arithmetic and logical operation operations. The cache memory can store the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the cache memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0057] The GPU is also known as the display core, visual processor, or display chip, and is a microprocessor for graphics and image processing. The GPU is connected to the display 190, the AP, and the CPU. The GPU can be used to execute complex mathematical and geometric calculations, floating-point operations, parallel calculations, and can also be used for graphics rendering. The GPU reduces the dependence of the graphics card on the CPU. Especially when processing 3D graphics, the GPU adopts core technologies such as cubic environment texture mapping and vertex blending. Therefore, the GPU performs some of the work that was originally done by the CPU.

[0058] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, the acquired audio data is saved in the external memory card.

[0059] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store the operating system and at least one APP required for a function, such as a screen mirroring APP, etc.

[0060] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared technology (IR), and so on. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via an antenna, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna for radiation. In the embodiments of the present application, through the wireless communication module 160, the electronic device 100 may establish a screen mirroring connection with other devices.

[0061] The electronic device 100 may implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the AP, etc. For example, playing audio data and so on.

[0062] Among them, the audio module 170 is used to convert digital audio information into analog audio signals for output, and is also used to convert analog audio inputs into digital audio signals. The audio module 170 may also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110. In the embodiments of the present application, the audio module 170 may be used to play the audio data of at least one device that has established a screen mirroring connection with the electronic device 100.

[0063] The display 190 is used to display images, videos, etc. The display 190 may include a signal processing module and a display panel. The signal processing module may periodically report a tearing effect signal (TE) to the CPU / GPU driver, triggering the CPU / GPU to render the user interface (UI). For example, the display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N displays 190, where N is a positive integer greater than 1. In the embodiments of the present application, the display 190 may be used to display the screens of at least one device that has established a screen mirroring connection with the electronic device 100.

[0064] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0065] The software architecture of the electronic device 100 will be described below.

[0066] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the software system of the electronic device 100 is exemplarily described by taking the Windows system with a layered architecture as an example.

[0067] Please refer to Figure 3 , Figure 3It is a schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application. The layered architecture divides the software into several layers, and each layer has clear roles and divisions of labor. Communication between layers is carried out through software interfaces. In some embodiments, the Windows system is divided into user mode and kernel mode. Among them, the user mode includes an application layer and subsystem dynamic link libraries. The kernel mode is divided into a firmware layer, a hardware abstraction layer (HAL), a kernel, a driver layer, and an executive body from bottom to top.

[0068] The application layer may include a series of application programs. Such as Figure 3 As shown, the application programs may include screen mirroring applications, camera applications, maps, music, videos, conferencing and other application programs. In the embodiment of the present application, the device screen mirroring application program is used to enable the device screen mirroring function. Among them, only some application programs are shown in the figure, and the application layer may also include other application programs, such as shopping applications, browsers, etc., which are not limited in the embodiment of the present application.

[0069] The subsystem dynamic link libraries include an application programming interface (API) module, and the API module includes Windows (window) API, Windows native API, etc. Among them, both Windows API and Windows native API can provide system call entry and internal function support for application programs. In the embodiment of the present application, the subsystem dynamic link libraries may include a distributed mobile sensing development platform (DMSDP) service. The DMSDP service can be used to virtualize the device, for example, virtualize the audio device and provide virtual speaker services. In the present application, the DMSDP service can be used to call the audio manager to create a decoding module and a playback module for each slave device respectively in a multi-device screen mirroring scenario, and accordingly implement the simultaneous playback of audio data of multiple slave devices.

[0070] The executive body includes a process manager, an I / O manager, an operating system event driver (OsEventDriver) node, a Windows management instrumentation (WMI) plug-in, a power manager, a virtual memory manager, an operating system to System on chip (OS2SOC) node, etc.

[0071] The kernel and driver layer include the kernel and device drivers. The kernel is an abstraction of the processor architecture, isolating the executive body from the differences in the processor architecture to ensure the portability of the system. The kernel can perform thread scheduling and dispatching, trap handling and exception dispatching, interrupt handling and dispatching, etc. The device driver runs in kernel mode and is the interface between the I / O system and related hardware. The device driver may include a GPU driver (also called a graphics card driver), a CPU driver (such as an Intel DTT driver), a mouse driver, an audio driver, a camera driver, a keyboard driver, etc. For example, a GPU driver can drive the GPU to run, and a CPU driver can drive the CPU to run.

[0072] HAL is a kernel-state module that hides various hardware-related details, such as I / O interfaces, interrupt controllers, and multi-processor communication mechanisms, and provides a unified service interface for different hardware platforms running Windows, achieving portability on multiple hardware platforms.

[0073] The firmware layer may include the basic input output system (BIOS). BIOS is a set of programs fixed to a read only memory (ROM) chip on the computer motherboard. It stores the most important basic input and output programs of the computer, the self-test program after power-on, and the system self-starting program. It can read and write specific information of system settings from the complementary metal oxide semiconductor (CMOS).

[0074] The above software architecture runs on the hardware layer, which includes GPU, CPU, memory, speaker, mouse and keyboard, etc. In an embodiment of the present application, the speaker can be used to play audio data of at least one slave device that establishes a screen projection connection with the master device.

[0075] It should be noted that the functional modules included in the above software architecture are only exemplary and do not constitute a specific limitation on the software architecture of the mobile phone of the present application. In other embodiments, the functional modules included in the above software architecture may be more or less, and the present application does not limit this. Although the embodiments of the present application are described using the Windows system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Android.

[0076] The following describes the system architecture used in the embodiments of the present application.

[0077] See also Figure 4 , Figure 4It is a schematic diagram of a system architecture applied to the embodiments of the present application. As Figure 4 shown, the system architecture may include a first device 401, a second device 402, and a third device 403. Among them, the first device 401 and the third device 403 may establish communication in a wired or wireless manner, and the second device 402 and the third device 403 may also establish communication in a wired or wireless manner. It can be understood that Figure 4 the number and form of the devices shown are only for illustration and do not constitute a limitation on the embodiments of the present application. In practical applications, there may be devices with a number different from Figure 4 that shown.

[0078] In the embodiments of the present application, the first device 401, the second device 402, and the third device 403 may all be the above-mentioned Figure 2 and Figure 3 electronic device 100. For example, the electronic device 100 may be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, a personal computer, a super mobile personal computer, a netbook, a personal digital assistant, a television, etc. The embodiments of the present application do not make any limitations in this regard.

[0079] In some embodiments, the screen of the first device 401 may be cast to the interface of the third device 403 for display. At the same time, the screen of the second device 402 may also be cast to the interface of the third device 403 for display. The first device 401, the second device 402, and the third device 403 may be of the same type or different types of devices. For example, the first device 401, the second device 402, and the third device 403 may all be mobile phones, or the first device 401 and the second device 402 may be mobile phones, and the third device 403 may be a laptop computer.

[0080] The audio playback method provided by the embodiments of the present application may be applied to a scenario of multi-device screen casting, such as the scenario where the first device 401 and the second device 402 are cast to the third device 403. Among them, by executing the audio playback method provided by the embodiments of the present application, the third device 403 can flexibly play the audio data of the first device 401 and the audio data of the second device 402, which is beneficial to improving the intelligence of audio playback.

[0081] Next, the audio playback method provided by the embodiments of the present application will be described.

[0082] Please refer to Figure 5 , Figure 5 which is a flowchart of an audio playback method provided by the embodiments of the present application. As Figure 5As shown, the method may include but is not limited to the following steps:

[0083] S501. The first device establishes a first screen mirroring connection with the third device.

[0084] S502. The second device establishes a second screen mirroring connection with the third device.

[0085] In the embodiments of the present application, the first device can establish a first screen mirroring connection with the third device, and the second device can establish a second screen mirroring connection with the third device. Here, step S501 and step S502 can be executed sequentially. For example, step S501 is executed first, and then step S502, or step S502 is executed first and then step S501. Step S501 and step S502 can also be executed simultaneously, that is, when the third device establishes a first screen mirroring connection with the first device, it also establishes a second screen mirroring connection with the second device.

[0086] In some embodiments, after the first device establishes a first screen mirroring connection with the third device, the screen of the first device can be projected and displayed on the interface of the third device. After the second device establishes a second screen mirroring connection with the third device, the screen of the second device can also be projected and displayed on the interface of the third device. Among them, the first device and the second device can be described as slave devices, and the third device can be described as a master device. Optionally, by operating each function control included in the screen of the first device displayed on the third device, the first device can be controlled to execute corresponding functions; by operating each function control included in the screen of the second device displayed on the third device, the second device can be controlled to execute corresponding functions.

[0087] In some embodiments, the first device establishing a first screen mirroring connection with the third device can be in a wired connection or a wireless connection manner, and the second device establishing a second screen mirroring connection with the third device can also be in a wired connection or a wireless connection manner. Among them, the wired connection manner can be, for example, a universal serial bus (USB) connection, and the wireless connection manner can be, for example, a Wi-Fi connection. The present application is not limited thereto.

[0088] In some embodiments, before the first device establishes a first screen mirroring connection with the third device, the first device may send a first screen mirroring connection request to the third device. Correspondingly, the third device may receive the first screen mirroring connection request from the first device. The second device may send a second screen mirroring connection request to the third device. Correspondingly, the third device may receive the screen mirroring connection request from the second device. Optionally, after receiving the first screen mirroring connection request and / or the second screen mirroring connection request, the third device may output a connection prompt message, which may be used to prompt whether to agree to establish a screen mirroring connection with the first device and / or the second device. For example, when the third device receives the first screen mirroring connection request, it may output a prompt message to prompt whether to agree to establish the first screen mirroring connection with the first device. Another example is that when the third device receives the first screen mirroring connection request and the second screen mirroring connection request at the same time, it may output a prompt message for prompting whether to agree to establish the first screen mirroring connection with the first device and the second screen mirroring connection with the second device.

[0089] In some embodiments, the first screen mirroring connection and the second screen mirroring connection may be of the same type of screen mirroring connection. For example, both the first screen mirroring connection and the second screen mirroring connection may be collaborative screen mirroring connections, or both the first screen mirroring connection and the second screen mirroring connection may be heterogeneous screen mirroring connections. The first screen mirroring connection and the second screen mirroring connection may be of different types of screen mirroring connections. For example, the first screen mirroring connection may be a collaborative screen mirroring connection, and the second screen mirroring connection may be a heterogeneous screen mirroring connection.

[0090] S503, based on the established first screen mirroring connection, the first device sends a first request to the third device. Correspondingly, based on the established first screen mirroring connection, the third device receives the first request from the first device.

[0091] In the embodiments of the present application, after the first device establishes a first screen mirroring connection with the third device, it may send a first request to the third device based on the established first screen mirroring connection. Among them, the first request may be used to request the third device to play the audio data of the first device. Based on the established first screen mirroring connection, the first device can project the screen onto the interface of the third device for mirror display. For a screen with an audio playback requirement, such as the screen of an audio application in the first device, or the screen of a video application, or the screen of a game application, etc., the first device may request the third device to play the audio data of the corresponding screen.

[0092] In some embodiments, the first request may carry first identification information, and the first identification information may be used to uniquely identify the first device. Optionally, the first identification information may be the account information logged in by the first device, or the digital identification information corresponding to the first device. For example, the first identification information may be the digital identification 1, or may be the serial number of the first device.

[0093] In some embodiments, after the third device receives a first request from the first device, it may send a first response to the first device. Correspondingly, the first device may receive the first response from the third device. The first response may include the result of whether the first device is allowed to use the speaker of the third device to play audio data. In response to being allowed to use the speaker of the third device to play audio data, the first device may send audio data to the third device. In response to not being allowed to use the speaker of the third device to play audio data, the first device may output a prompt message indicating that it is not allowed to use the speaker of the third device to play audio data.

[0094] In some embodiments, before the first device sends audio data to the third device, it may send first audio parameters to the third device. Correspondingly, the third device may receive the first audio parameters from the first device. Optionally, the first audio parameters may include parameters such as the audio encoding method, audio bit rate, number of audio channels, and audio sampling rate of the first device.

[0095] In some embodiments, the first device and the third device may communicate through the DMSDP service. The first device may send data (such as the first request, the audio data of the first device, the first audio parameters, etc.) to the third device through the DMSDP service. The third device may receive data from the first device through the DMSDP service. Among them, the DMSDP service may be a service included in the application framework layer. Exemplarily, please refer to Figure 6 , Figure 6 is a schematic diagram of communication between a first device and a third device through the DMSDP service provided by an embodiment of the present application. As Figure 6 shown, the application framework layer of the first device includes the DMSDP service, and the application framework layer of the third device also includes the DMSDP service. Correspondingly, the application layer of the first device may include a DMSDP interface layer, and the application layer of the third device may include a DMSDP interface layer. The first device and the third device may call the DMSDP service based on the DMSDP interface layer, and then implement data interaction based on the DMSDP service. Optionally, the DMSDP service may further specifically include a concurrent state cache module, a virtualization capability module, a concurrent management module, and a concurrent information interaction module.

[0096] S504, the second device sends a second request to the third device based on the established second screen mirroring connection. Correspondingly, the third device receives the second request from the second device based on the established second screen mirroring connection.

[0097] In the embodiments of the present application, after the second device and the third device establish a second screen mirroring connection, they may send a second request to the third device based on the established second screen mirroring connection. Among them, the second request may be used to request the third device to play the audio data of the second device.

[0098] Optionally, step S503 and step S504 can be executed successively or simultaneously. That is to say, when the first device has established a first screen mirroring connection with the third device and the second device has established a second screen mirroring connection with the third device, the time when the first device sends a first request to the third device and the time when the second device sends a second request to the third device can be the same or different. Correspondingly, the time when the third device receives the first request and the time when it receives the second request can also be the same or different. Based on the established second screen mirroring connection, the second device can mirror and display the screen on the interface of the third device. For a screen with an audio playback requirement, such as the screen of an audio application in the second device, or the screen of a video application, or the screen of a game application, etc., the second device can request the third device to play the audio data of the corresponding screen.

[0099] In some embodiments, the second device and the third device can communicate through the DMSDP service, and the second device can send a second request to the third device through the DMSDP service. The third device can receive the first request from the second device through the DMSDP service.

[0100] In some embodiments, the second request can carry second identification information, and the second identification information can be used to uniquely identify the second device. Optionally, the second identification information can be the account information logged in by the second device, or the digital identification information corresponding to the second device. For example, the second identification information can be the digital identification 2, or it can be the serial number of the second device.

[0101] In some embodiments, after the third device receives the second request from the second device, it can send a second response to the second device. Correspondingly, the second device can receive the second response from the third device. The second response can include the result of whether to allow the second device to use the speaker of the third device to play audio data. In response to being allowed to use the speaker of the third device to play audio data, the second device can send audio data to the third device. In response to not being allowed to use the speaker of the third device to play audio data, the second device can output a prompt message indicating that it is not allowed to use the speaker of the third device to play audio data.

[0102] In some embodiments, before the second device sends audio data to the third device, it can send second audio parameters to the third device. Correspondingly, the third device can receive the second audio parameters from the second device. Optionally, the second audio parameters can include parameters such as the audio encoding method, audio bit rate, number of audio channels, and audio sampling rate of the second device.

[0103] S505, The third device responds to the first request and the second request, and creates a first audio processing module and a second audio processing module through the virtual speaker service of the third device.

[0104] In an embodiment of the present application, in response to the received first request and second request, the third device may create a first audio processing module and a second audio processing module through a virtual speaker server. Among them, the virtual speaker service can be used for speaker virtualization of the third device. The first audio processing module can be used to process the audio data of the first device. The second audio processing module can be used to process the audio data of the second device.

[0105] In some embodiments, the virtual speaker service may be established based on the DMSDP service. When the third device creates the first audio processing module through the virtual speaker service, the state of the virtual speaker service can be set to not occupied, so that the third device can continue to create the second audio processing module through the virtual speaker service.

[0106] In some embodiments, the third device may be configured with multiple speakers, and one speaker may correspond to one virtual speaker service. That is to say, one virtual speaker service can be used to virtualize one speaker. In this case, different virtual speaker services may correspond to different service identifiers. Optionally, the first request may carry a first service identifier of the first virtual speaker service, and the second device may carry a second service identifier of the second virtual speaker service. In response to the first request and the second request, the third device may create a first audio processing module through the first virtual speaker service based on the first service identifier, and create a second audio processing module through the second virtual speaker service based on the second service identifier. For ease of understanding, the embodiments of the present application are described by taking the third device configured with one speaker as an example, but this does not limit the present application.

[0107] In some embodiments, the third device may call an audio manager through the virtual speaker service to create a first audio processing module and a second audio processing module. Among them, the audio manager can be used to create and call at least two audio processing modules. For example, when the third device establishes a third screen mirroring connection with the fourth device, the third device may also call the audio manager through the virtual speaker service to create a third audio processing module. In this way, the third device can manage the audio processing modules of different devices separately through the audio manager, so that the third device can process the audio data from different devices simultaneously.

[0108] In some embodiments, the audio manager can be invoked through the audio processing interface (AudioHandler) of the virtual speaker service, that is, the third device can invoke the audio manager through the audio processing interface of the virtual speaker service to create a first audio processing module and a second audio processing module. Optionally, the first request can carry first identification information, and the second request can carry second identification information. The third device can, based on the first identification information, invoke the audio manager through the audio processing interface of the virtual speaker service to create a first audio processing module, and based on the second identification information, invoke the audio manager through the audio processing interface of the virtual speaker service to create a second processing module. For example, assuming the first identification information is the digital identifier 1 and the second identification information is the digital 2, the third device can invoke the audio manager through the audio processing interface of the virtual speaker service to create two audio processing modules, such as audio processing module 1 and audio processing module 2.

[0109] In some embodiments, before the third device invokes the audio manager through the audio processing interface of the virtual speaker service to create a first audio processing module and a second audio processing module, it can determine whether there already exists an audio processing module for processing the audio data of the first device and / or an audio processing module for processing the audio data of the second device. If there already exists an audio processing module for processing the audio data of the first device, the existing audio processing module for processing the audio data of the first device can be used as the first audio processing module. Similarly, if there already exists an audio processing module for processing the audio data of the second device, the existing audio processing module for processing the audio data of the second device can be used as the second audio processing module.

[0110] S506. The third device processes the audio data from the first device based on the first audio processing module to obtain first audio data, and processes the audio data from the second device based on the second audio processing module to obtain second audio data.

[0111] In the embodiments of the present application, after the third device creates the first audio processing module and the second audio processing module, it can process the audio data from the first device based on the first audio processing module to obtain first audio data, and process the audio data from the second device based on the second audio processing module to obtain second audio data. Among them, the first audio data can be understood as the decoded audio data of the first device. The second audio data can be understood as the decoded audio data of the second device. By processing the audio data of different devices through different audio processing modules, the processing of the audio data of multiple devices can be carried out simultaneously, which is conducive to improving the efficiency and flexibility of audio processing.

[0112] Optionally, the third device may establish a first data channel between the DMSDP service of the third device and the DMSDP service of the first device, and establish a second data channel between the DMSDP service of the third device and the DMSDP service of the second device. The first device may send audio data to the third device through the first data channel. Correspondingly, the third device may receive the audio data from the first device through the first data channel. The second device may send audio data to the third device through the second data channel. Correspondingly, the third device may receive the audio data from the second device through the second data channel.

[0113] In some embodiments, the first audio processing module may include a first decoding module, and the second audio processing module may include a second decoding module. The third device may perform decoding processing on the audio data from the first device based on the first decoding module to obtain first audio data. Optionally, the third device may perform decoding processing on the audio data from the first device based on the first audio parameter through the first decoding module. Correspondingly, the third device may perform decoding processing on the audio data from the second device based on the second decoding module to obtain second audio data. Optionally, the third device may perform decoding processing on the audio data from the second device based on the second audio parameter through the second decoding module.

[0114] In some embodiments, the third device may call the audio manager to write the audio data of the first device into the first decoding queue, and then obtain the audio data of the first device from the first decoding queue based on the first decoding module, and perform decoding processing on the audio data of the first device to obtain first audio data. Correspondingly, the third device may call the audio manager to write the audio data of the second device into the second decoding queue, and then obtain the audio data of the second device from the second decoding queue based on the second decoding module, and perform decoding processing on the audio data of the second device to obtain second audio data. After the audio manager writes the audio data of the first device into the first decoding queue, it does not need to wait for the first decoding module to finish decoding the audio data of the first device, and may immediately write the audio data of the second device into the second decoding queue. This can avoid the delay caused by waiting for decoding for too long and is beneficial to improving the smoothness of audio playback.

[0115] In some embodiments, the third device may call the audio manager through the audio processing interface of the virtual speaker service to write the audio data of the first device into the first decoding queue, and call the audio manager through the audio processing interface of the virtual speaker service of the third device to write the audio data of the second device into the second decoding queue. After the audio processing interface of the virtual speaker service calls the audio manager to write the audio data of the first device into the first decoding queue, it does not need to wait for the first decoding module to finish decoding the audio data of the first device, and can return to call the audio manager again to write the audio data of the second device into the second decoding queue. In this way, a smooth audio playback effect can be achieved.

[0116] S507. The third device calls the speaker of the third device to play the first audio data and the second audio data.

[0117] In the embodiments of the present application, after obtaining the first audio data and the second audio data, the third device may call the speaker to play the first audio data and the second audio data. Among them, the first audio data and the second audio data may be played simultaneously. That is to say, the first device and the second device may simultaneously use the speaker of the third device to play their respective audio data. For example, assuming that the first device projects the screen of a music application onto the interface of the third device for display, and the second device projects the screen of a video application onto the interface of the third device for display, then the third device calls the speaker to simultaneously play the first audio data of the music application of the first device and the second audio data of the video application of the second device.

[0118] In some embodiments, the first audio processing module may further include a first playback module, and the second audio processing module may further include a second playback module. The first playback module may be used to call the speaker of the third device to play the first audio data. The second playback module may be used to call the speaker of the third device to play the second audio data. After the third device obtains the first audio data and the second audio data, it may put the first audio data into the first playback queue and put the second audio data into the second playback queue. Then, the third device may obtain the first audio data from the first playback queue through the first playback module, and call the speaker through the first playback module to play the first audio data. Similarly, the third device may obtain the second audio data from the second playback queue through the second playback module, and call the speaker through the second playback module to play the second audio data.

[0119] In some embodiments, the third device may also receive a shutdown request from the first device. The shutdown request may be used to request the third device to stop playing the first audio data. Based on the shutdown request, the third device may delete the first audio processing module through the virtual speaker service and stop invoking the speaker of the third device to play the first audio data. Optionally, the third device may also delete the first audio parameters through the virtual speaker service. In this way, it is beneficial to save the memory resources of the third device.

[0120] Exemplarily, please refer to Figure 7 , Figure 7 which is a schematic diagram of a third device processing audio data of different devices provided by an embodiment of the present application. As Figure 7 shown, for the audio data from the first device and the audio data from the second device, the third device may create a first audio processing module and a second audio processing module by invoking the audio manager through the audio processing interface of the virtual speaker service. The first audio processing module may include a first decoding module and a first playback module, and the second audio processing module may include a second decoding module and a second playback module. The audio data from the first device may be decoded by the first decoding module to obtain the first audio data. The audio data from the second device may be decoded by the second decoding module to obtain the second audio data. Then, the speaker of the third device may be invoked by the first playback module to play the first audio data, and the speaker of the third device may be invoked by the second playback module to play the second audio data. When the third device receives a shutdown request from the first device, the first audio processing module may be deleted by invoking the audio manager through the audio processing interface of the virtual speaker service, and the speaker of the third device may be stopped from playing the first audio data. When the third device receives a shutdown request from the second device, the second audio processing module may be deleted by invoking the audio manager through the audio processing interface of the virtual speaker service, and the speaker of the third device may be stopped from playing the second audio data.

[0121] In some embodiments, the third device may control the volume of the speaker playing the audio data according to the order of receiving the requests. Optionally, when the receiving time of the first request is earlier than the receiving time of the second request, the third device may invoke the speaker through the first playback module to play the first audio data at a first volume and invoke the speaker through the second playback module to play the second audio data at a second volume. The first volume may be higher than the second volume. That is to say, the volume of the audio data of the device that receives the request first may be higher than the volume of the audio data of the device that receives the request later.

[0122] In some embodiments, the third device can also control the volume of the speaker playing the audio data according to the priority of the device. Optionally, the third device can obtain the priorities of different devices, and the volume of the audio data of the device with a higher priority can be higher than that of the device with a lower priority. For example, assuming that the priority of the second device is higher than that of the first device, the third device can call the speaker based on the first playback module to play the first audio data at a third volume, and call the speaker based on the second playback module to play the second audio data at a fourth volume, where the third volume can be lower than the fourth volume.

[0123] In some embodiments, the third device can also control the volume of the speaker playing the audio data according to the priority of the virtual speaker service. Optionally, the first request can be used to request the third device to play the audio data of the first device through the media virtual speaker service, and the second request can be used to request the third device to play the audio data of the second device through the Modem (modulation / demodulation processor) virtual speaker service. Then, the third device can call the speaker based on the first playback module to play the first audio data at a fifth volume, and call the speaker based on the second playback module to play the second audio data at a sixth volume. Among them, the priority of the Modem virtual speaker service can be higher than that of the media virtual speaker service, the sixth volume can be higher than the fifth volume, the first audio data can be, for example, the audio data of a video application, and the second audio data can be, for example, the audio data of a call.

[0124] In some embodiments, the third device can display the audio playback interface of the first device and the audio playback interface of the second device on the screen mirroring interface. Optionally, the audio playback interface of the first device can include a first volume control, and the audio playback interface of the second device can include a second volume control. In response to the first selection operation received for the first volume control, the third device can play the first audio data at the volume corresponding to the first selection operation. In response to the second selection operation received for the second volume control, the third device can play the second audio data at the volume corresponding to the second selection operation. Optionally, the audio playback interface of the first device can further include a first pause / play control, and the audio playback interface of the second device can further include a second pause / play control. In response to the third selection operation received for the first pause / play control, the third device can pause playing the first audio data. In response to the fourth selection operation received for the second pause / play control, the third device can pause playing the second audio data.

[0125] Exemplarily, please refer to Figure 8 , Figure 8 which is a schematic diagram of the screen mirroring interface of a third device provided by an embodiment of the present application. As Figure 8As shown, the screen mirroring interface 80 of the third device can display the audio playback interface 801 of the first device and the audio playback interface 802 of the second device. Among them, the audio playback interface 801 includes a first volume control 8011 and a first pause / play control 8012, and the audio playback interface 802 includes a second volume control 8021 and a second pause / play control 8022. In response to a first selection operation on the second volume control 8021 received by the third device, such as selecting a volume level of 20, the third device can play the first audio data (such as Song A) at the volume level of 20. In response to a second selection operation on the second volume control 8021 received by the third device, such as selecting a volume level of 10, the third device can play the second audio data (such as Song B) at the volume level of 10. Optionally, in response to a third selection operation on the first pause / play control 8012 received by the third device, the third device can pause the playback of the first audio data. In response to a fourth selection operation on the second pause / play control 8022 received by the third device, the third device can pause the playback of the second audio data.

[0126] By implementing this application, in a multi-device screen mirroring scenario, the third device can respectively create a first audio processing module for processing the audio data of the first device and a second audio processing module for processing the audio data of the second device through the virtual speaker service, so as to independently process the audio data of different devices simultaneously, avoid resource conflicts, and can call the speaker to play the audio data of different devices at the same time. In this way, it is beneficial to improve the flexibility and intelligence of audio playback.

[0127] In some embodiments, the first device and the second device can transmit audio-related data to the third device based on the DMSDP service, and the third device can provide a virtual speaker service based on the DMSDP service, and then call the audio manager through the virtual speaker service to process the audio data of the first device and the audio data of the second device respectively. Please refer to Figure 9 , Figure 9 is a schematic flowchart of another audio playback method provided by an embodiment of this application. As shown in Figure 9 , the method includes but is not limited to the following steps:

[0128] S901, the screen mirroring application of the first device calls the DMSDP service to send a first request to the DMSDP service of the third device. Correspondingly, the DMSDP service of the third device receives the first request from the DMSDP service of the first device.

[0129] In an embodiment of the present application, the first device may be installed with a screen mirroring application. The first device may establish a first screen mirroring connection with the third device through the screen mirroring application. Furthermore, the screen of the first device may be mirrored and displayed on the interface of the third device. After the first device and the third device establish the first screen mirroring connection, a first request may be sent to the DMSDP service of the third device by calling the DMSDP service of the first device. Among them, the first request may be used to request the third device to play the audio data of the first device. Or it may also be described as that the first request may be used to request to use the audio playback capability of the third device.

[0130] In some embodiments, the first device and the third device may establish a data transmission channel based on the DMSDP service. The first device may send first audio parameters to the DMSDP service of the third device by calling the DMSDP service. The third device may process the audio data of the first device according to the first audio parameters.

[0131] S902, the DMSDP service of the third device calls the audio manager through the audio interface of the virtual speaker service to create a first decoding module and a first playback module.

[0132] In an embodiment of the present application, after the third device receives the first request, the DMSDP service of the third device may be associated with the virtual speaker service, and then call the audio manager through the audio interface of the virtual speaker service to create a first decoding module and a first playback module. Among them, the first decoding module may be used to decode the audio data from the first device, and the first playback module may be used to play the decoded audio data of the first device.

[0133] S903, the DMSDP service of the first device sends the audio data of the first device to the DMSDP service of the third device. Correspondingly, the DMSDP service of the third device receives the audio data of the first device from the DMSDP service of the first device.

[0134] In an embodiment of the present application, when the screen of the first device mirrored on the third device needs to play audio, the DMSDP service of the first device may send the audio data of the first device to the DMSDP service of the third device.

[0135] S904, the DMSDP service of the third device calls the audio manager through the audio processing interface of the virtual speaker service to send the audio data of the first device to the first decoding module.

[0136] In the embodiment of the present application, after the third device receives the audio data of the first device, the DMSDP service of the third device can send the audio data of the first device to the first decoding module by calling the audio manager through the audio processing interface of the virtual speaker service. Among them, after the audio processing interface of the virtual speaker service calls the audio manager to send the audio data of the first device to the first decoding module, it can immediately return to wait for the newly obtained third audio data without waiting for the first decoding module to decode the audio data of the first device.

[0137] Optionally, the third audio data can be from the first device or from the second device. For the audio data from the first device, the DMSDP service of the third device can send it to the first decoding module by calling the audio manager through the audio processing interface of the virtual speaker service. For the audio data from the second device, the third device can first determine whether there is a second decoding module for decoding the audio data of the second device. If there is a second decoding module, the DMSDP service of the third device can send the audio data to the second decoding module by calling the audio manager through the audio processing interface of the virtual speaker service; if there is no second decoding module, the DMSDP service of the third device can first call the audio manager through the audio processing interface of the virtual speaker service to create a second decoding module, and then send the audio data to the second decoding module.

[0138] S905, the first decoding module of the third device decodes the audio data of the first device to obtain the first audio data.

[0139] In the embodiment of the present application, the first decoding module of the third device can decode the audio data of the first device to obtain the first audio data. Here, the first audio data can be understood as the decoded audio data of the first device. Optionally, the first decoding module can decode the audio data of the first device based on the first audio parameter.

[0140] S906, the first decoding module of the third device sends the first audio data to the first playback module of the third device.

[0141] In the embodiment of the present application, after the first decoding module of the third device obtains the first audio data, it can send the first audio data to the first playback module of the third device. Optionally, the first decoding module can send the first audio data to the first playback queue, and the first playback can obtain the first audio data from the first playback queue.

[0142] S907, the first playback module of the third device calls the speaker to play the first audio data.

[0143] In the embodiment of the present application, the first playback module of the third device may call a speaker to play the first audio data. Optionally, the first audio data may be a complete audio, or a certain segment of the complete audio.

[0144] S908, the screen mirroring application of the second device calls the DMSDP service to send a second request to the DMSDP service of the third device. Correspondingly, the DMSDP service of the third device receives the second request from the DMSDP service of the second device.

[0145] In the embodiment of the present application, step S908 and step S901 may be executed sequentially or simultaneously. The implementation principles of step S908 and step S901 are similar and will not be elaborated here.

[0146] S909, the DMSDP service of the third device calls the audio manager through the audio interface of the virtual speaker service to create a second decoding module and a second playback module.

[0147] In the embodiment of the present application, after the third device receives the second request, the DMSDP service of the third device may call the audio manager through the audio interface of the virtual speaker service to create a second decoding module and a second playback module.

[0148] In some embodiments, if the third device receives the first request and the second request simultaneously, then the DMSDP service of the third device may call the audio manager through the audio interface of the virtual speaker service to first create a first decoding module and a first playback module, and then create a second decoding module and a second playback module.

[0149] S910, the DMSDP service of the second device sends the audio data of the second device to the DMSDP service of the third device. Correspondingly, the DMSDP service of the third device receives the audio data of the second device from the DMSDP service of the second device.

[0150] In the embodiment of the present application, when the screen mirroring of the second device in the third device needs to play audio, the DMSDP service of the second device may send the audio data of the second device to the DMSDP service of the third device.

[0151] S911, the DMSDP service of the third device calls the audio manager through the audio processing interface of the virtual speaker service to send the audio data of the second device to the second decoding module.

[0152] In the embodiments of the present application, after the third device receives the audio data of the second device, the DMSDP service of the third device can call the audio manager through the audio processing interface of the virtual speaker service to send the audio data of the second device to the second decoding module. Here, if the third device receives the audio data of the first device and the audio data of the second device at the same time, or, the third device receives the audio data of the second device during the process of calling the audio manager through the audio processing interface of the virtual speaker service to send the audio data of the first device to the first decoding module, then after the DMSDP service of the third device calls the audio manager through the audio processing interface of the virtual speaker service to send the audio data of the first device to the first decoding module, it can send the audio data of the second device to the second decoding module.

[0153] In some embodiments, for the case where the third device has multiple speakers, different speakers can correspond to different virtual speaker services. Optionally, the third device can send the audio data of the first device to the first decoding module through the first virtual speaker service, and send the audio data of the second device to the second decoding module through the second virtual speaker service. In this case, the third device can send the audio data of the first device to the first decoding module and send the audio data of the second device to the second decoding module at the same time.

[0154] S912, the second decoding module of the third device decodes the audio data of the second device to obtain second audio data.

[0155] S913, the second decoding module of the third device sends the second audio data to the second playback module of the third device.

[0156] S914, the second playback module of the third device calls the speaker to play the second audio data.

[0157] In the embodiments of the present application, steps S912 - S914 can be executed simultaneously with steps S905 - S907. It can be understood that the audio data of the first device and the audio data of the second device can be processed independently, and the speakers of the third device can play the first audio data and the second audio data simultaneously.

[0158] By implementing the embodiments of the present application, in the multi-device screen mirroring scenario, the third device can create different decoding modules and playback modules for the audio data of different devices based on the audio manager, so that the decoding and playback of the audio data of multiple devices can be carried out simultaneously, which is beneficial to improving the flexibility and intelligence of audio playback.

[0159] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk).

[0160] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0161] The steps in the method embodiments of this application can be adjusted, combined, and deleted according to actual needs.

[0162] The modules in the device embodiments of this application can be combined, divided, and deleted according to actual needs.

[0163] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The computer-readable storage medium can include: a flash drive, ROM, RAM, a magnetic disk, or an optical disc, etc.

[0164] What is disclosed above is only a preferred embodiment of this application, and it is only a part of the embodiments of this application. It cannot be used to limit the scope of rights of this application.

Claims

1. An audio playing method, characterized in that, The method is applied to a third device, and the method includes: Based on an established first screen mirroring connection, receiving a first request from a first device, and based on an established second screen mirroring connection, receiving a second request from a second device; the first request is used to request the third device to play audio data from the first device; the second request is used to request the third device to play audio data from the second device; In response to the first request and the second request, creating a first audio processing module and a second audio processing module through the virtual speaker service of the third device; Processing the audio data from the first device based on the first audio processing module to obtain first audio data, and processing the audio data from the second device based on the second audio processing module to obtain second audio data; Invoking the speaker of the third device to play the first audio data and the second audio data; Wherein, the first screen mirroring connection is a screen mirroring connection between the third device and the first device; the second screen mirroring connection is a screen mirroring connection between the third device and the second device.

2. The method according to claim 1, characterized in that, The creating a first audio processing module and a second audio processing module through the virtual speaker service of the third device includes: Invoking an audio manager through the virtual speaker service of the third device to create a first audio processing module and a second audio processing module; the audio manager is used to create and invoke at least two audio processing modules.

3. The method according to claim 2, wherein The invoking an audio manager through the virtual speaker service of the third device to create a first audio processing module and a second audio processing module includes: Invoking an audio manager through the audio processing interface of the virtual speaker service of the third device to create a first audio processing module and a second audio processing module.

4. The method according to claim 3, characterized in that The first request carries first identification information, and the second request carries second identification information; The invoking an audio manager through the audio processing interface of the virtual speaker service of the third device to create a first audio processing module and a second audio processing module includes: Based on the first identification information, invoking an audio manager through the audio processing interface of the virtual speaker service of the third device to create a first audio processing module; and based on the second identification information, invoking an audio manager through the audio processing interface of the virtual speaker service of the third device to create a second audio processing module.

5. The method according to claim 1, wherein The first audio processing module includes a first decoding module, and the second audio processing module includes a second decoding module; The processing the audio data from the first device based on the first audio processing module to obtain first audio data, and processing the audio data from the second device based on the second audio processing module to obtain second audio data includes: Decoding the audio data from the first device based on the first decoding module to obtain first audio data, and decoding the audio data from the second device based on the second decoding module to obtain second audio data.

6. The method according to claim 5, characterized in that, Performing decoding processing on the audio data from the first device based on the first decoding module to obtain first audio data, and performing decoding processing on the audio data from the second device based on the second decoding module to obtain second audio data, includes: Invoking the audio manager through the virtual speaker service of the third device to write the audio data of the first device into a first decoding queue, obtaining the audio data of the first device from the first decoding queue based on the first decoding module, performing decoding processing on the audio data of the first device to obtain first audio data, and invoking the audio manager to write the audio data of the second device into a second decoding queue, obtaining the audio data of the second device from the second decoding queue based on the second decoding module, and performing decoding processing on the audio data of the second device to obtain second audio data.

7. The method according to claim 6, characterized in that, The invoking the audio manager to write the audio data of the first device into the first decoding queue includes: Invoking the audio manager through the audio processing interface of the virtual speaker service of the third device to write the audio data of the first device into the first decoding queue; The invoking the audio manager to write the audio data of the second device into the second decoding queue includes: Invoking the audio manager through the audio processing interface of the virtual speaker service of the third device to write the audio data of the second device into the second decoding queue.

8. The method according to claim 5, wherein The first audio processing module further includes a first playback module, and the second audio processing module further includes a second playback module; The invoking the speaker of the third device to play the first audio data and the second audio data includes: Invoking the speaker of the third device to play the first audio data through the first playback module, and invoking the speaker of the third device to play the second audio data through the second playback module.

9. The method according to claim 8, wherein The based on the first playback module invoking the speaker of the third device to play the first audio data and based on the second playback module invoking the speaker of the third device to play the second audio data includes: Based on the first playback module, invoking the speaker of the third device to play the first audio data at a first volume, and based on the second playback module, invoking the speaker of the third device to play the second audio data at a second volume; wherein, the first volume is higher than the second volume, and the reception time of the first request is earlier than the reception time of the second request.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: Displaying the audio playback interface of the first device and the audio playback interface of the second device on the screen mirroring interface of the third device.

11. The method according to any one of claims 1-9, characterized in that, The method further includes: Receiving a shutdown request from the first device; the shutdown request is used to request the third device to stop playing the first audio data; Based on the shutdown request, deleting the first audio processing module through the virtual speaker service and stopping invoking the speaker of the third device to play the first audio data.

12. An electronic device, characterized in that, Comprising a memory and one or more processors; the memory is coupled to the one or more processors and is used for storing a computer program, the computer program comprising program instructions; the one or more processors call the program instructions so that the electronic device executes the method according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program which, when executed by a processor, implements the method according to any one of claims 1-11.

14. A chip system, which is applied to an electronic device, is characterized in that Comprising one or more processors, when the one or more processors execute computer instructions, the electronic device is caused to execute the method according to any one of claims 1-11.

Images