Audio playing method and device and electronic equipment

Abstract

The invention relates to the technical field of audio and video, in particular to an audio playing method and device and electronic equipment. The audio playing method comprises the following steps: under the condition that a playing instruction for audio data is received in an application program controlled by a first operating system, sending the audio data to a pulse audio module; controlling the pulse audio module to transmit the audio data to a rear-end module of a Weston system; and controlling a rear-end module of the Weston system to transmit the audio data to a client of a second operating system so as to play the audio data through the client of the second operating system. According to the invention, the feasibility of audio playing can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of audio and video technology, and in particular to an audio playback method, device and electronic equipment. Background Art

[0002] With the development of science and technology, electronic devices are developing more and more rapidly, which greatly facilitates the lives of users. For example, electronic devices can run Android systems. Applications can create audio tracks. Audio data can be transmitted from the framework layer, hal layer, kernel layer, and adsp layer through the Android framework, and audio playback can be achieved through the hardware sound card. However, when the system included in the electronic device does not have a hardware sound card, the audio data cannot be played. Summary of the invention

[0003] The present disclosure provides an audio playback method, device and electronic device to at least solve the problem that audio data cannot be played in the related art. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an audio playback method, comprising:

[0005] When a play instruction for audio data is received in an application controlled by the first operating system, the audio data is sent to a pulse audio module;

[0006] Control the pulse audio module to transmit the audio data to a back-end module of the Weston system;

[0007] The back-end module of the Weston system is controlled to transmit the audio data to the client of the second operating system so as to play the audio data through the client of the second operating system.

[0008] According to some embodiments, the pulse audio module includes a pulse audio service submodule and a remote display protocol sink submodule, and the controlling the pulse audio module to transmit the audio data to the back-end module of the Weston system includes:

[0009] Controlling the Pulse audio service submodule to receive the audio data through the remote display protocol sink submodule;

[0010] Through the remote display protocol sink submodule, the Pulse audio service submodule is controlled to transmit the unix socket to the back-end module of the Weston system, wherein the unix socket includes the audio data.

[0011] According to some embodiments, the method further comprises:

[0012] When the pulse audio module is started, the unix related modules are loaded according to the preset parameters, wherein the unix related modules include the remote display protocol sink submodule and the unix socket.

[0013] According to some embodiments, the backend module controlling the Weston system to transmit the audio data to a client of a second operating system includes:

[0014] When the Weston system is started, a thread corresponding to the remote display protocol sink submodule is created to receive the audio data;

[0015] When the backend module of the Weston system receives the audio data through the thread, the backend module of the Weston system is controlled to transmit the audio data to the client of the second operating system.

[0016] According to some embodiments, before the back-end module controlling the Weston system transmits the audio data to the client of the second operating system, the method further includes:

[0017] The backend module controlling the Weston system reads the audio data from the unix socket.

[0018] According to some embodiments, wherein the client of the second operating system is a remote display protocol-client, the back-end module controlling the Weston system transmits the audio data to the client of the second operating system, including:

[0019] Control the back-end module of the Weston system to transmit the audio data to the remote display protocol-server;

[0020] The remote display protocol-server is controlled to transmit the audio data to the remote display protocol-client through a remote display protocol connection.

[0021] According to some embodiments, the method further comprises:

[0022] Control the remote display protocol-server and the remote display protocol-client to perform at least one of the following operations through the header.command prompt channel:

[0023] negotiate audio codec;

[0024] Negotiate audio delay information;

[0025] Negotiate audio playback packets.

[0026] According to a second aspect of an embodiment of the present disclosure, there is provided an audio playback device, characterized in that it includes:

[0027] A data sending unit, configured to send the audio data to the pulse audio module when receiving a play instruction for the audio data in the application controlled by the first operating system;

[0028] A data transmission unit, used for controlling the pulse audio module to transmit the audio data to a back-end module of a Weston system;

[0029] The audio playback unit is used to control the back-end module of the Weston system to transmit the audio data to the client of the second operating system so as to play the audio data through the client of the second operating system.

[0030] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0031] processor;

[0032] a memory for storing instructions executable by the processor;

[0033] The processor is configured to execute the instructions to implement the audio playback method described in any one of the first aspects above.

[0034] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the audio playback method described in any one of the first aspects above.

[0035] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the audio playback method described in any one of the first aspects is implemented.

[0036] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0037] In some or related embodiments, when a play instruction for audio data is received in an application controlled by a first operating system, the audio data is sent to a pulse audio module; the pulse audio module is controlled to transmit the audio data to a back-end module of a Weston system; the back-end module of the Weston system is controlled to transmit the audio data to a client of a second operating system, so as to play the audio data through the client of the second operating system. Therefore, the audio data of the first operating system can be transmitted to the second operating system so as to be played through the hardware sound card of the second operating system, so that the audio data can be played across systems, reducing the situation where the first operating system does not have a hardware sound card and cannot play the audio data, resulting in poor executability of the audio data playback, improving the executability of the audio data playback, and expanding the application scenarios of desktop-level applications.

[0038] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0040] Figure 1 is a flow chart of an audio playing method according to an exemplary embodiment;

[0041] Figure 2 is a flow chart of an audio playing method according to an exemplary embodiment;

[0042] Figure 3 is a schematic diagram showing an example of an audio playback system according to an exemplary embodiment;

[0043] Figure 4 is a schematic diagram showing an example of an audio playback system according to an exemplary embodiment;

[0044] Figure 5 is a schematic diagram showing an example of an audio playback system according to an exemplary embodiment;

[0045] Figure 6 is a schematic diagram showing an example of an audio playback system according to an exemplary embodiment;

[0046] Figure 7 is a block diagram of an audio playback device according to an exemplary embodiment;

[0047] Figure 8It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0048] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0050] Figure 1 is a flowchart of an audio playing method according to an exemplary embodiment. Figure 1 As shown, the audio playback method can be used in a cross-platform (cross-system) audio playback scenario, including the following steps:

[0051] In step S101, when a play instruction for audio data is received in an application controlled by a first operating system, the audio data is sent to a pulse audio module;

[0052] According to some embodiments, the technical solution of the embodiments of the present disclosure may be, for example, an electronic device, which may include multiple operating systems, wherein the multiple operating systems may include, for example, a desktop operating system and an Android operating system. Among the multiple operating systems, there is a desktop operating system that does not have a hardware sound card, that is, the operating system cannot play audio data. Among the multiple operating systems, there may also be an Android operating system, which includes a hardware sound card, and audio data can be played through the hardware sound card.

[0053] In some embodiments, the electronic device may be referred to as a "terminal" or a "terminal device", and may also be referred to as a "user equipment (UE)", "user terminal (user terminal)", "mobile station (MS)", "mobile terminal (MT)", subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device (mobile device), wireless device (wireless device), wireless communication device (wireless communication device), remote device (remote device), mobile subscriber station, access terminal (access terminal), mobile terminal (mobile terminal), wireless terminal (wireless terminal), remote terminal (remote terminal), handheld device (handset), user agent (user agent), mobile client (mobile client), client (client), etc.

[0054] In some embodiments, the first operating system may be, for example, an operating system that cannot play audio data. The first in the first operating system is only used to distinguish it from other operating systems, and the first operating system does not specifically refer to a fixed operating system. For example, when the system name corresponding to the first operating system changes, the first operating system may also change accordingly.

[0055] According to some embodiments, the application may be, for example, an application controlled by the first operating system. Audio data may be inserted into the application, and a play instruction may be received on a display interface corresponding to the application. The play instruction is used to play the audio data. The play instruction includes, but is not limited to, a voice play instruction, a click play instruction, a timed play instruction, and the like.

[0056] In some embodiments, the audio data may be, for example, audio data that needs to be played in an application. The audio data does not specifically refer to a fixed data. For example, when a modification instruction for the audio data is received, the audio data may be modified. For example, when the duration corresponding to the audio data changes, the audio data may also change accordingly. For example, the audio data may be, for example, separate audio data, or audio data included in audio and video data.

[0057] In some embodiments, the Pulse audio module can be used as a middleware between an application and a hardware device. The Pulse audio module of the embodiment of the present disclosure can be used as a middleware between an application and a hardware sound card.

[0058] According to some embodiments, when a play instruction for audio data is received in an application controlled by the first operating system, the audio data is sent to the Pulse audio module.

[0059] In step S102, the pulse audio module is controlled to transmit the audio data to the back-end module of the Weston system;

[0060] According to some embodiments, the backend module of the Weston system can, for example, interact with the client of the second operating system and transmit the audio data to the client of the second operating system. The client of the second operating system can play the audio data.

[0061] According to some embodiments, when the Pulse audio module receives audio data, the Pulse audio module may be controlled to transmit the audio data to a backend module of the Weston system.

[0062] In step S103, the backend module of the Weston system is controlled to transmit the audio data to the client of the second operating system so as to play the audio data through the client of the second operating system.

[0063] For example, in one embodiment of the present disclosure, the second operating system refers to a system that can control a hardware sound card to play audio, and the second operating system can be, for example, an Android operating system.

[0064] According to some embodiments, when the backend module of the Weston system receives audio data, the backend module of the Weston system can be controlled to transmit the audio data to the client of the second operating system so that the audio data can be played through the client of the second operating system.

[0065] In some or related embodiments, when a play instruction for audio data is received in an application controlled by a first operating system, audio data is sent to a pulse audio module; the pulse audio module is controlled to transmit the audio data to a back-end module of a Weston system; and the back-end module of a Weston system is controlled to transmit the audio data to a client of a second operating system, so as to play the audio data through the client of the second operating system. Therefore, the audio data of the first operating system can be transmitted to the second operating system so as to be played through the hardware sound card of the second operating system, so that the audio data can be played across systems, reducing the situation where the first operating system does not have a hardware sound card and cannot play audio data, resulting in poor executability of audio data playback, improving the executability of audio data playback, and expanding the application scenarios of desktop-level applications.

[0066] Figure 2 is a flowchart of an audio playing method according to an exemplary embodiment. Figure 2 As shown, the audio playback method can be used in a cross-platform (cross-system) audio playback scenario, including the following steps:

[0067] In step S201, when a play instruction for audio data is received in an application controlled by a first operating system, the audio data is sent to a pulse audio module;

[0068] The specific process is as described above and will not be repeated here.

[0069] Among them, in one embodiment of the present disclosure, Figure 3 This is an example schematic diagram of an audio playback system according to an exemplary embodiment. All operating systems and related modules involved in this solution can be set in the Linux kernel, for example.

[0070] According to some embodiments, the execution subject of the embodiments of the present disclosure may be, for example, an electronic device. The screen size of the electronic device may be, for example, greater than a size threshold. The size threshold may be, for example, 10 inches. The architecture of the electronic device may be, for example, a WSL (Windows Subsystem for Linux) architecture.

[0071] In one embodiment of the present disclosure, the pulse audio module can, for example, run on a desktop-level system. The pulse audio module includes a remote display protocol sink submodule, a pulse audio service submodule, and a remote display protocol source submodule. Among them, in the technical solution of the embodiment of the present disclosure, when transmitting audio data from the first operating system to the second operating system, for example, the remote display protocol sink submodule and the pulse audio service submodule can be used. For example, when transmitting audio data from the second operating system to the first operating system, for example, the remote display protocol source submodule and the pulse audio service submodule can be used. For example, the remote display protocol sink submodule, the pulse audio service submodule, and the remote display protocol source submodule can realize two-way audio data transmission.

[0072] According to some embodiments, Figure 4 : is an example schematic diagram of an audio playback system according to an exemplary embodiment. The left side of the vertical line may be, for example, a client of the second operating system. The client of the second operating system may be, for example, an RDP-client. The right side of the vertical line may be, for example, a container system. The container system may include, for example, a backend module of a Weston system, an RDP server, and a Wayland APP. The Wayland APP may be, for example, an application of the first operating system of the embodiment of the present disclosure. The RDP server may be, for example, FreeRDP.

[0073] In some embodiments, for example, the Wayland protocol and the RDP protocol can be used to complete the transmission of audio data. Wherein, Wayland can be, for example, a communication protocol between a display server and a client. The display server is also called a Wayland compositor. Wherein, Weston is a specific implementation of the Wayland compositor. From the perspective of the internal architecture, Weston can, for example, include a window management module (shell), a synthesizer module (compositor) and an input management module. Wherein the RDP backend module (RDP backend) as the Weston backend can, for example, implement the instantiation of the input device (keyboard, mouse) and output device (graphics processing unit (GPU) GPU, display) interface. Figure 5The figure is an example schematic diagram of an audio playback system according to an exemplary embodiment. The RDP-backend can be a headless system, for example, which uses pixman as a renderer and uses memory as a buffer. When the RDP backend detects the connection requirement of FreeRDP, it instantiates each peer input and output device.

[0074] According to some embodiments, FreeRDP is an implementation of a Remote Desktop Protocol that complies with the Apache open source protocol, supports 3D (3D) functions, has a higher refresh rate, and also supports desktop virtualization technology (RemoteFX), highly compressed digital video codec standard encoding and decoding of H264, user management, audio, and peripheral redirection, etc. FreeRDP mainly implements encoding and decoding of RDP-backend backend communications.

[0075] According to some embodiments, the first operating system may be, for example, a desktop operating system. The desktop operating system includes, but is not limited to, Ubuntu, Debian, Windows, etc. Ubuntu is a Linux operating system based on desktop applications. The kernel of the Debian system may be, for example, a Linux kernel or a FreeBSD kernel. The Windows operating system may be, for example, an operating system developed based on a graphical user interface. The second operating system may be, for example, an Android operating system.

[0076] In some embodiments, the image rootfs can be constructed using an application container engine (docker), for example Figure 3 The container system may include, for example, a first operating system, Weston, and a PulseAudio service module. After the container system is started, the Weston and PulseAudio service modules may be started with the daemon process MSLGd, and an RDP connection may be established in silent mode through freeRDP.

[0077] In some embodiments, the first operating system may be, for example, an Ubuntu operating system. For example, a WPS application and an A browser may be run in the Ubuntu operating system. The first application may be, for example, a WPS application. For example, an audio or video file may be inserted into the WPS application. For example, a play instruction for the audio or video file may be received. The play instruction may be, for example, a click instruction. The click instruction may be, for example, a direct click on the display interface of the audio or video, and the click instruction may also be, for example, a click operation of detecting a play control. When a play instruction for audio data is received in the WPS application controlled by the Ubuntu operating system, the audio data may be sent to the Pulse audio module. Since there is no hardware sound card in the Ubuntu operating system, the Ubuntu operating system cannot play the audio data.

[0078] Among them, in one embodiment of the present disclosure, the pulseaudio service module can, for example, provide a resident system service process, which receives audio-related operation request commands, manages audio devices, and implements data interaction with audio devices, etc. Various applications that access the pulse audio library and access audio devices through the pulse audio API interface are clients of the pulse audio system service. They write data to the audio playback device through the pulse audio system service to play the sound, and read the recording device through the pulse audio system service to obtain the collected audio data.

[0079] In step S202, the Pulse audio service submodule is controlled to receive audio data through the remote display protocol sink submodule;

[0080] The specific process is as described above and will not be repeated here.

[0081] In some embodiments, Figure 6 1 is a flow chart of an audio playback method according to an exemplary embodiment. The pulse audio module includes a pulse audio service submodule and a remote display protocol sink submodule. The electronic device can, for example, control the pulse audio service submodule to receive audio data through the remote display protocol sink submodule.

[0082] According to some embodiments, when the pulse audio module is started, the unix-related modules are loaded according to pre-set parameters, wherein the unix-related modules include a remote display protocol sink submodule and a unix socket, which can improve the accuracy of module loading, reduce the process of inaccurate module loading, and make the audio data unable to be transmitted, and improve the convenience of audio data transmission.

[0083] Among them, in the technical solution of the embodiment of the present disclosure, the Unix-related modules loaded according to the preset parameters can be, for example, the remote display protocol sink submodule and the Unix socket.

[0084] Different preset parameters can be used to load different modules. For example, the container system can pre-compile the Pulse audio service submodule, and use the Pulse audio service submodule as a server to receive audio data sent by the first operating system.

[0085] For example, in one embodiment of the present disclosure, the relevant modules may be started in the following manner:

[0086] "--load=\"module-rdp-sink sink_name=RDPSink\"", which is the RDP Sink plug-in in the pulse audio service in the figure above;

[0087] "--load=\"module-rdp-source source_name=RDPSource\"", which is the RDP Source plug-in in the pulse audio service in the figure above;

[0088] "--load=\"module-native-protocol-unix socket="SHARE_PATH" / mnt / PulseServer auth-anonymous=true\"", that is, the unix socket used by pulseaudio to connect with weston.

[0089] In step S203, the Pulse audio service submodule is controlled to transmit the unix socket to the backend module of the Weston system through the remote display protocol sink submodule, wherein the unix socket includes audio data;

[0090] According to some embodiments, the Pulse audio module can communicate with the back-end module of the Weston system through a unix socket, for example. The unix socket is then sent to the back-end module of the Weston system through the remote display protocol sink submodule. Since the unix socket includes audio data, the Pulse audio service submodule can be controlled to send the audio data to the back-end module of the Weston system through the remote display protocol sink submodule.

[0091] According to some embodiments, when the Weston system is started, a thread corresponding to the remote display protocol sink submodule is created to receive audio data;

[0092] When the back-end module of the Weston system receives audio data through a thread, controlling the back-end module of the Weston system to transmit the audio data to the client of the second operating system can provide an audio data transmission mechanism, reduce the situation where audio data cannot be transmitted, making the audio data unable to be played across systems, and improve the feasibility of audio data playback.

[0093] In some embodiments, the Weston system can use FreeRDP to implement the back-end module of the Weston system. For example, the back-end module of the Weston system can establish a socket connection with the pulseaudio sink submodule and create a thread about the pulseaudio sink submodule so that the back-end module of the Weston system can transmit audio data to the client of the second operating system.

[0094] According to some embodiments, before the backend module of the Weston system is controlled to transmit the audio data to the client of the second operating system, the method further includes:

[0095] Backend module that controls the Weston system to read audio data from a unix socket.

[0096] According to some embodiments, the backend module of the Weston system may be controlled to read audio data from a unix socket, and the backend module of the Weston system may be controlled to transmit the audio data to a client of the second operating system.

[0097] In step S204, the backend module of the Weston system is controlled to transmit the audio data to the client of the second operating system so as to play the audio data through the client of the second operating system.

[0098] The specific process is as described above and will not be repeated here.

[0099] In some embodiments, the backend module of the Weston system can be controlled to transmit audio data to the client of the second operating system so that the audio data can be played by the client of the second operating system. For example, when the client of the second operating system receives the audio data, the audio data can be transmitted to the audioflinger so that the audioflinger can play the audio.

[0100] According to some embodiments, wherein the client of the second operating system is a remote display protocol-client, controlling the backend module of the Weston system to transmit audio data to the client of the second operating system includes:

[0101] The backend module that controls the Weston system transmits audio data to the remote display protocol-server;

[0102] Controlling the remote display protocol-server to transmit audio data to the remote display protocol-client through the remote display protocol connection can provide an audio data transmission mechanism, reduce the situation where audio data cannot be transmitted, making the audio data unable to be played across systems, so that the second operating system can play the audio data, thereby improving the feasibility of audio data playback.

[0103] In one embodiment of the present disclosure, the remote display protocol-client can be run on a second operating system, for example, the second operating system can be an Android operating system.

[0104] According to some embodiments, the remote display protocol-server and the remote display protocol-client are controlled to perform at least one of the following operations through the header.command prompt channel:

[0105] negotiate audio codec;

[0106] Negotiate audio delay information;

[0107] Negotiate audio playback packets.

[0108] According to some embodiments, audio data in freeRDP can be transmitted through RDPsnd channel, for example. When the RDP connection is established, RDPsnd will automatically start and automatically stop when the connection is closed. RDP-server and RDP-client can negotiate audio codec, audio latency, audio transmission and playback packet, etc. through header.cmd. When the RDP-client receives the audio data, opensl es is used for audio playback, so that the audio data in the first operating system can be played in the Android system.

[0109] In some or related embodiments, the Pulse audio service submodule is controlled to receive audio data through the remote display protocol sink submodule; the Pulse audio service submodule is controlled to transmit the unix socket to the back-end module of the Weston system through the remote display protocol sink submodule, wherein the unix socket includes audio data. Therefore, a method for transmitting audio data can be provided, reducing the situation where audio data cannot be played, realizing cross-system playback of audio data, and improving the executability of sound wave data playback.

[0110] Figure 7is a block diagram of an audio playback device according to an exemplary embodiment. Figure 7 The device 700 includes a data sending unit 701, a data transmission unit 702 and an audio playing unit 703.

[0111] The data sending unit 701 is used to send the audio data to the pulse audio module when receiving a play instruction for the audio data in the application controlled by the first operating system;

[0112] The data transmission unit 702 is used to control the pulse audio module to transmit the audio data to the back-end module of the Weston system;

[0113] The audio playing unit 703 is used to control the back-end module of the Weston system to transmit the audio data to the client of the second operating system so as to play the audio data through the client of the second operating system.

[0114] According to some embodiments, the pulse audio module includes a pulse audio service submodule and a remote display protocol sink submodule, and the data transmission unit 702 is used to control the pulse audio module to transmit audio data to the back-end module of the Weston system, specifically for:

[0115] Through the remote display protocol sink submodule, control the Pulse audio service submodule to receive audio data;

[0116] Through the remote display protocol sink submodule, the Pulse audio service submodule is controlled to transmit the unix socket to the backend module of the Weston system, wherein the unix socket includes audio data.

[0117] According to some embodiments, the data transmission unit 702 is further configured to:

[0118] When the pulse audio module is started, the unix related modules are loaded according to the preset parameters, wherein the unix related modules include the remote display protocol sink submodule and the unix socket.

[0119] According to some embodiments, the data transmission unit 702 is used to control the back-end module of the Weston system to transmit the audio data to the client of the second operating system, specifically to:

[0120] When the Weston system is started, a thread corresponding to the remote display protocol sink submodule is created to receive audio data;

[0121] When the backend module of the Weston system receives the audio data through the thread, the backend module of the Weston system is controlled to transmit the audio data to the client of the second operating system.

[0122] According to some embodiments, the data transmission unit 702 is used to control the backend module of the Weston system to transmit the audio data to the client of the second operating system before the backend module transmits the audio data to the client of the second operating system.

[0123] Backend module that controls the Weston system to read audio data from a unix socket.

[0124] According to some embodiments, when the client of the second operating system is a remote display protocol-client, the audio playback unit 703 is used to control the back-end module of the Weston system to transmit audio data to the client of the second operating system, specifically for:

[0125] The backend module that controls the Weston system transmits audio data to the remote display protocol-server;

[0126] The remote display protocol-server is controlled to transmit the audio data to the remote display protocol-client through the remote display protocol connection.

[0127] According to some embodiments, the audio playback unit 703 is further configured to:

[0128] Control the remote display protocol-server and remote display protocol-client to perform at least one of the following operations through the header.command prompt channel:

[0129] negotiate audio codec;

[0130] Negotiate audio delay information;

[0131] Negotiate audio playback packets.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] In summary, the device provided by the embodiment of the present disclosure is used to send audio data to the pulse audio module through a data sending unit when a play instruction for audio data is received in an application controlled by a first operating system; a data transmission unit is used to control the pulse audio module to transmit the audio data to the back-end module of the Weston system; an audio playback unit is used to control the back-end module of the Weston system to transmit the audio data to the client of the second operating system, so as to play the audio data through the client of the second operating system. Therefore, the audio data of the first operating system can be transmitted to the second operating system so as to be played through the hardware sound card of the second operating system, thereby reducing the situation where the first operating system has no hardware sound card and cannot play audio data, resulting in poor executability of audio data playback, improving the executability of audio data playback, and expanding the application scenarios of desktop-level applications.

[0134] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit implementations of the present disclosure described and / or claimed herein.

[0135] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0136] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0137] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as an audio playback method or an image processing method. For example, in some embodiments, the audio playback method or the image processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the model training determination method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the audio playing method or the image processing method in any other appropriate manner (for example, by means of firmware).

[0138] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0143] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0144] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0145] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An audio playback method, characterized in that: include: When a play instruction for audio data is received in an application controlled by the first operating system, the audio data is sent to a pulse audio module; Control the pulse audio module to transmit the audio data to a back-end module of the Weston system; The back-end module of the Weston system is controlled to transmit the audio data to the client of the second operating system so as to play the audio data through the client of the second operating system.

2. The method according to claim 1, characterized in that in, The pulse audio module includes a pulse audio service submodule and a remote display protocol sink submodule, and the control of the pulse audio module to transmit the audio data to the back-end module of the Weston system includes: Controlling the Pulse audio service submodule to receive the audio data through the remote display protocol sink submodule; Through the remote display protocol sink submodule, the Pulse audio service submodule is controlled to transmit the unix socket to the back-end module of the Weston system, wherein the unix socket includes the audio data.

3. The method according to claim 2, characterized in that The method further comprises: When the pulse audio module is started, the unix related modules are loaded according to the preset parameters, wherein the unix related modules include the remote display protocol sink submodule and the unix socket.

4. The method according to claim 2 or 3, characterized in that: The back-end module controlling the Weston system transmits the audio data to the client of the second operating system, including: When the Weston system is started, a thread corresponding to the remote display protocol sink submodule is created to receive the audio data; When the backend module of the Weston system receives the audio data through the thread, the backend module of the Weston system is controlled to transmit the audio data to the client of the second operating system.

5. The method according to claim 2, characterized in that: Before the back-end module controlling the Weston system transmits the audio data to the client of the second operating system, it also includes: The backend module controlling the Weston system reads the audio data from the unix socket.

6. The method according to claim 1, characterized in that in, The client of the second operating system is a remote display protocol-client, and the back-end module controlling the Weston system transmits the audio data to the client of the second operating system, including: Control the back-end module of the Weston system to transmit the audio data to the remote display protocol-server; The remote display protocol-server is controlled to transmit the audio data to the remote display protocol-client through a remote display protocol connection.

7. The method according to claim 6, characterized in that The method further comprises: Control the remote display protocol-server and the remote display protocol-client to perform at least one of the following operations through the header.command prompt channel: negotiate audio codec; Negotiate audio delay information; Negotiate audio playback packets.

8. An audio playback device, characterized in that: include: A data sending unit, configured to send the audio data to the pulse audio module when receiving a play instruction for the audio data in the application controlled by the first operating system; A data transmission unit, used for controlling the pulse audio module to transmit the audio data to a back-end module of a Weston system; The audio playback unit is used to control the back-end module of the Weston system to transmit the audio data to the client of the second operating system so as to play the audio data through the client of the second operating system.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions for causing a computer to execute the method according to any one of claims 1 to 7.

Images