Audio processing method and apparatus based on multi-machine interaction, and storage medium

Abstract

Provided in the present invention are an audio processing method and apparatus based on multi-machine interaction, and a storage medium. The method comprises: in response to an audio transmission instruction triggered by local software, acquiring first audio data corresponding to the audio transmission instruction; synchronizing the first audio data with target software by means of a first virtual interface of a virtual loudspeaker, so that the local software outputs the audio data; receiving second audio data sent by the target software; and sending the second audio data to a virtual microphone by means of a second virtual interface of the virtual microphone, and sending the second audio data in the virtual microphone to the local software, so that the local software inputs the audio data.

Description

Audio processing method, device and storage medium based on multi-machine interaction Technical Field

[0001] The present invention relates to the field of video technology, and in particular to an audio processing method, device and storage medium based on multi-machine interaction. Background Art

[0002] Currently, paired virtual sound cards are used to implement speaker broadcasting and microphone recording. The virtual speaker and virtual microphone in the paired virtual sound card must be used in pairs. That is, one virtual sound card simulates an application (such as a virtual speaker or a virtual microphone). If the speaker and microphone of the paired virtual sound card are to be used, two paired virtual sound cards are required. The specific workflow is shown in Figure 1. Application A forwards audio data through the virtual speaker of the first virtual sound card to application B through the virtual microphone of the first virtual sound card as microphone recording input. Application B forwards the sound to the virtual microphone through the virtual speaker of the second virtual sound card, so that application A can collect the sound of the virtual microphone of the second virtual sound card in microphone recording mode.

[0003] However, for the multi-machine interaction scenario of multi-person online meetings, there are often multiple users participating in the online meeting in different places, which means that multiple applications need to use virtual sound cards to implement speaker broadcasting and microphone recording. If paired virtual sound cards are used to implement speaker broadcasting and microphone recording, multiple paired virtual sound cards are required. Moreover, since one paired virtual sound card corresponds to one application, and since the names of the virtual sound cards are different, when the user needs to adjust the sound of a virtual sound card, it is easy to make adjustments incorrectly. Technical issues

[0004] The present invention provides an audio processing method, device and storage medium based on multi-machine interaction. In a multi-machine interaction scenario, there is no need to configure multiple virtual sound cards, thereby avoiding adjustment errors caused by different naming of virtual sound cards.

[0005] In a first aspect, the present invention provides an audio processing method based on multi-machine interaction, comprising:

[0006] In response to an audio transmission instruction triggered by local software, obtaining first audio data corresponding to the audio transmission instruction;

[0007] Synchronizing the first audio data to the target software through a first virtual interface of a virtual speaker so that the local software outputs the audio data, wherein the virtual speaker is a pre-set virtual output device of the local software; wherein the virtual speaker is used to control the output audio data;

[0008] receiving second audio data sent by the target software, where the second audio data is an integration result of user audio data collected by the target software;

[0009] The second audio data is sent to the virtual microphone through the second virtual interface of the virtual microphone, and the second audio data in the virtual microphone is sent to the local software so that the local software can input audio data, and the virtual microphone is the pre-set one.

[0010] In a second aspect, the present invention further provides an audio processing device based on multi-machine interaction, comprising:

[0011] an acquisition module, configured to, in response to an audio transmission instruction triggered by local software, acquire first audio data corresponding to the audio transmission instruction;

[0012] a synchronization module, configured to synchronize the first audio data to the target software via a first virtual interface of a virtual speaker, so that the local software outputs the audio data, wherein the virtual speaker is a pre-set virtual output device of the local software; wherein the virtual speaker is configured to control the output audio data;

[0013] a receiving module, configured to receive second audio data sent by the target software, where the second audio data is an integration result of user audio data collected by the target software;

[0014] A sending module is used to send the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and send the second audio data in the virtual microphone to the local software so that the local software can input audio data. The virtual microphone is a pre-set virtual input device of the local software; wherein the virtual microphone is used to control the input audio data.

[0015] In a third aspect, the present invention further proposes a computer storage medium, wherein a computer program is stored in the computer storage medium. When the computer program is executed, any one of the above-mentioned audio processing methods based on multi-machine interaction is implemented.

[0016] The present invention provides an audio processing method, device and storage medium based on multi-machine interaction. In response to an audio transmission instruction triggered by local software, the method obtains first audio data corresponding to the audio transmission instruction, and synchronizes the first audio data to the target software through the first virtual interface of the virtual speaker so that the local software outputs the audio data. The virtual speaker is a virtual output device preset for the local software. The virtual speaker is used to control the output audio data, receive second audio data sent by the target software, and the second audio data is an integrated result of the user audio data collected by the target software. The virtual microphone sends the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and sends the second audio data in the virtual microphone to the local software so that the local software inputs audio data. The virtual microphone is a virtual input device preset for the local software. The virtual microphone is used to control the input audio data. In the audio processing solution provided by the present invention, the local software can output the first audio data to the target software through the first virtual interface of the virtual speaker, and the target software can input the second audio data to the local software through the second virtual interface of the virtual microphone, that is, the input and output of audio data to the target software are realized through a virtual sound card. It can be seen that in the scenario of multi-machine interaction, there is no need to configure multiple virtual sound cards, thereby avoiding adjustment errors due to different naming of virtual sound cards. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. The drawings described below are only drawings corresponding to some embodiments of the present invention. For ordinary technicians in this field, without paying any creative work, they can also obtain drawings of other embodiments based on these drawings.

[0018] FIG1 is a schematic structural diagram of an existing virtual sound card.

[0019] FIG2 is a flow chart of an audio processing method based on multi-machine interaction provided by an embodiment of the present invention.

[0020] FIG3 is a schematic diagram of data exchange of a unified virtual sound card provided by an embodiment of the present invention.

[0021] FIG4 is another data exchange diagram of the unified virtual sound card provided by an embodiment of the present invention.

[0022] FIG5 is a schematic diagram of the specific structure of the virtual speaker in the unified virtual sound card provided by an embodiment of the present invention.

[0023] FIG6 is another schematic diagram of data exchange of the unified virtual sound card provided by an embodiment of the present invention.

[0024] FIG7 is a display interface of the audio processing method based on multi-machine interaction provided by an embodiment of the present invention.

[0025] FIG8 is a display interface of a virtual speaker provided by an embodiment of the present invention.

[0026] FIG9 is a display interface of a virtual microphone provided by an embodiment of the present invention.

[0027] FIG10 is a schematic diagram of a scenario of an audio processing method based on multi-machine interaction provided by an embodiment of the present invention.

[0028] FIG11 is a schematic structural diagram of an audio processing device based on multi-machine interaction provided by an embodiment of the present invention.

[0029] FIG12 is another structural diagram of an audio processing device based on multi-machine interaction provided by an embodiment of the present invention. Best Mode for Carrying Out the Invention

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0032] Referring to FIG2 , the present invention provides an audio processing method based on multi-machine interaction, comprising:

[0033] 101. In response to an audio transmission instruction triggered by local software, obtain first audio data corresponding to the audio transmission instruction.

[0034] The local software can be multi-person video or online conferencing software. The audio transmission instruction can be triggered when the local software establishes a communication connection with the target software, or when the microphone of the terminal corresponding to the local software captures the user's voice. That is, the local software triggers the audio transmission instruction when it detects that the microphone captures the user's voice. The audio transmission instruction carries the audio data to be transmitted (i.e., first audio data). The first audio data can include the user's voice data and other ambient sound data.

[0035] 102. Synchronize the first audio data to the target software through the first virtual interface of the virtual speaker, so that the local software can output the audio data.

[0036] In the present invention, the virtual speaker and the virtual microphone are both components of the virtual sound card. The virtual sound card is a driver layer of the operating system, based on the Windows driver model, that simulates and implements the speaker and microphone functions of the sound card. Since the virtual sound card corresponds to a virtual speaker and a virtual microphone, its function is consistent with that of a real sound card and its speaker and microphone functions. It can also record and broadcast, support sound format settings, support volume amplification and reduction, and mixing, etc. It can be understood that local software can input and output data through its corresponding virtual sound card.

[0037] Optionally, in some embodiments, the audio processing method based on multi-machine interaction of the present invention may further include:

[0038] (11) Pre-setting a virtual sound card including a virtual speaker and a virtual microphone;

[0039] (12) Bind the virtual sound card to the local software so that the local software can input and output data through the virtual sound card.

[0040] For example, specifically, the local software can be software a, software b and software c, and virtual sound card A, virtual sound card B and virtual sound card C are pre-set, wherein each virtual sound card only includes a virtual speaker and a virtual microphone. After setting, software a is bound to virtual sound card A, software b is bound to virtual sound card B, and software c is bound to virtual sound card C. Moreover, after binding, the logo of the virtual sound card can also be displayed on the sound card display interface. For example, when virtual sound card A is displayed, it is displayed as virtual sound card (a), indicating that the virtual sound card (a) is bound to software a.

[0041] In the actual use of the software, a virtual sound card can be used to adjust the input and / or output of the software, such as increasing the output volume, or decreasing the input volume, etc., that is, a virtual sound card can be used to input and output audio data for the target software. It can be seen that in the scenario of multi-machine interaction, there is no need to configure multiple virtual sound cards, thereby avoiding adjustment errors due to different naming of virtual sound cards.

[0042] Since the virtual sound card is located in the kernel state of the operating system, that is, the virtual speaker is located in the kernel state of the operating system, and the target software is located in the user state of the operating system, in order to realize communication between the virtual speaker and the target software, in some embodiments of the present invention, a data interface (that is, a first virtual interface) for communication between the virtual speaker and the target software is pre-set for data exchange, that is, the virtual speaker is a virtual output device of the pre-set local software, and the virtual speaker is used to control the output audio data, such as adjusting the volume, etc.

[0043] When the local software transmits the first audio data to the virtual speaker, the virtual speaker does not play the audio directly, but buffers the first audio data in a buffer. Instead, the preset speaker forwarding service extracts the corresponding first audio data from the buffer through the first virtual interface and synchronizes the extracted first audio data to the terminal corresponding to the target software. That is, optionally, in some embodiments, the step of "synchronizing the first audio data to the target software through the first virtual interface of the virtual speaker" may specifically include:

[0044] (21) buffering the first audio data into a first preset buffer through the virtual speaker;

[0045] (22) Based on the speaker forwarding service, the first audio data buffered in the first preset buffer is synchronized to the terminal corresponding to the target software through the first virtual interface of the virtual speaker.

[0046] Among them, the speaker forwarding service is bound to the first preset buffer, and the speaker forwarding service is used to extract the data buffered in the first buffer and synchronize the extracted first audio data to the terminal corresponding to the target software, so that the terminal plays the audio corresponding to the first audio data through components such as speakers.

[0047] Furthermore, the virtual speaker forwards data through the first virtual interface, and the core service (SnapServer) synchronizes the first audio data to the corresponding target software, so that the terminal corresponding to the target software plays the audio corresponding to the first audio data. That is, optionally, in some embodiments, the step of "synchronizing the first audio data buffered in the first preset buffer to the terminal corresponding to the target software through the first virtual interface of the virtual speaker based on the speaker forwarding service" may specifically include:

[0048] (31) in response to an extraction instruction of the speaker forwarding service for the first preset buffer, forwarding the first audio data buffered in the first preset buffer to the core service through the first virtual interface of the virtual speaker;

[0049] (32) Based on the core service and the network time protocol, the first audio data is synchronized to the target terminal corresponding to the target software.

[0050] Optionally, in some embodiments, the present invention adopts a Snapcast audio system, which is divided into two parts: a server and a client. The server is used to capture audio data and send the captured audio data to a terminal under the audio system. The client includes a terminal corresponding to the local software and a terminal corresponding to the target software. Specifically, after the local software and the target software establish a communication connection, in response to the extraction instruction pin of the speaker forwarding service for the first preset buffer, the buffered first audio data is extracted from the first preset buffer through the first virtual interface. Then, in response to the forwarding operation of the speaker forwarding service, the first audio data is forwarded to the core service, and the core service SnapServer determines the corresponding terminal SnapClient, and based on the network time protocol, the first audio data is synchronized to the determined terminal SnapClient (that is, the target terminal corresponding to the target software).

[0051] It should be noted that the Network Time Protocol (NTP) is a protocol used to synchronize computer time. It can synchronize a computer with its server or clock source (such as a quartz clock, GPS, etc.). That is, it is understandable that the transmission delay corresponding to the core service SnapServer can be determined based on NTP, and based on the transmission delay, the first audio data can be synchronized to the target terminal. That is, optionally, in some embodiments, the step of "synchronizing the first audio data to the target terminal corresponding to the target software based on the core service and the Network Time Protocol" can specifically include:

[0052] (41) Determine the software in the same core service network as the local software as the target software;

[0053] (42) Determine the transmission delay according to the network time protocol;

[0054] (43) Based on the transmission delay, the forwarded first audio data is synchronized to the target terminal corresponding to the target software through the core service.

[0055] After the target software establishes a communication connection with the local software, it can send an audio acquisition instruction to the local software. After receiving the audio acquisition instruction, the local software generates an audio transmission instruction. When the target software sends the audio acquisition instruction to the local software, the local software sends an NTP request message to the NTP server, which contains the timestamp t1 when the NTP request message leaves the software. When the NTP request message arrives at the NTP server, the time at which the NTP server arrives is t2. After processing the message, the NTP server sends an NTP response message at time t3. This NTP response message carries timestamps t1, t2, and t3. Upon receiving the response message, the local software records the timestamp t4 returned by the message. Then, based on these timestamps, it calculates a transmission delay offset = [(t2 - t1) + (t3 - t4)] / 2. The local software can then adjust its current clock based on this transmission delay offset, thereby achieving clock synchronization with the NTP server. It should be noted that in this embodiment, the NTP server is the server corresponding to the core service.

[0056] 103. Receive second audio data sent by the target software.

[0057] Among them, the second audio data is the integration result of the user audio data collected by the target software. Specifically, in the scenario of a multi-person meeting, take the local software corresponding to the host and the target software corresponding to the participant as an example. In the scenario of a multi-person meeting, there are often multiple participants, that is, one target software can correspond to multiple terminals, that is, there are multiple channels of sound data transmitted to the target software; specifically, the physical microphone corresponding to the target software will collect the sound signal emitted by the outside world, and the sound signal emitted by the outside world contains the user's voice signal and the noise in the external environment. Before the sound signal collected by the physical microphone is transmitted, the sound signal can be noise-reduced. Then, the target terminal corresponding to the target software transmits the noise-reduced sound signal to the audio integration unit, and the audio integration unit packages the transmitted sound signal to obtain the second audio data; that is, optionally, in some embodiments, the step of "receiving the second audio data sent by the target software" can specifically include:

[0058] (51) Control the virtual microphone to collect user audio data from the physical microphone corresponding to the target software;

[0059] (52) Integrate the multiple channels of user audio data to obtain second audio data.

[0060] It should be noted that a time synchronization mechanism needs to be adopted during sound integration to achieve time synchronization consistency between multiple terminals and the terminal where the target software is located. The time synchronization mechanism can be synchronized using the NTP strategy mentioned above, which will not be repeated here.

[0061] 104. Send the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and send the second audio data in the virtual microphone to the local software so that the local software can input the audio data.

[0062] It can be understood that since the virtual sound card is located in the kernel state of the operating system, that is, the virtual microphone is located in the kernel state of the operating system, and the local software is located in the user state of the operating system, in order to realize the communication between the virtual speaker and the local software, in some embodiments of the present invention, a data interface (that is, a second virtual interface) for communication between the virtual microphone and the local software is pre-set for data exchange, that is, the virtual speaker is a virtual input device of the pre-set local software, and the virtual microphone is used to control the input audio data, such as adjusting the volume, etc.

[0063] The virtual microphone does not directly interact with the second audio data. During actual transmission, the second audio data is buffered in a second preset buffer through the second virtual interface, and the virtual microphone extracts the second audio data from the second preset buffer to send the second audio data in the virtual microphone to the local software. That is, optionally, in some embodiments, the step of "sending the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and sending the second audio data in the virtual microphone to the local software" may specifically include:

[0064] (61) Based on the microphone forwarding service, buffer the second audio data into a second preset buffer through the second virtual interface of the virtual microphone;

[0065] (62) Controlling the virtual microphone to extract the buffered second audio data from the second preset buffer, and sending the extracted second audio data to the local terminal corresponding to the local software.

[0066] Among them, the microphone forwarding service is bound to the second preset buffer, and the microphone forwarding service is used to buffer the second audio data of the sound integration unit into the second preset buffer through the second virtual interface. When the virtual microphone acts as an independent thread, it controls the virtual microphone to extract the buffered second audio data from the second preset buffer and send the extracted second audio data to the local terminal corresponding to the local software.

[0067] Optionally, in some embodiments, the step of "buffering the second audio data into a second preset buffer through the second virtual interface of the virtual microphone based on the microphone forwarding service" may specifically include: responding to a data transmission instruction of the speaker forwarding service for the second preset buffer, buffering the second audio data into the second preset buffer through the second virtual interface of the virtual microphone.

[0068] To further understand the audio processing solution of the present invention, the virtual sound card of the present invention is further described below. The present invention provides a virtual sound card through which application A exchanges audio data with application B. As shown in Figure 3, the virtual sound card includes a virtual speaker, a first buffer, a first switching interface, a virtual microphone, a second buffer, and a second switching interface. Application A broadcasts sound through the virtual speaker of the virtual sound card. The sound passes through the first buffer and the first switching interface, and the audio data is sent to application B, achieving the recording effect of "microphone recording". Similarly, application B sends audio data to the second switching interface of the virtual sound card in a "speaker broadcast" mode. The second switching interface saves the audio data to the second buffer and then forwards it to the virtual microphone. Finally, application A realizes the recording function of the virtual microphone.

[0069] Furthermore, as shown in Figure 4, third-party applications can access the virtual speaker through the sound card driver. The sound is transmitted through the multi-machine software's monitoring agent, supporting audio monitoring, logging, and control. The multi-machine software uses the multi-machine sound synchronization service via the first switching interface to extract the corresponding audio data from the first buffer and then synchronously transmit the audio data to multiple conference machines. Monitoring is also supported through other speakers on the local machine.

[0070] Optionally, the present invention provides a virtual speaker with audio data buffering, as shown in FIG5 . The application obtains the virtual speaker by acquiring the system driver, and then broadcasts to the virtual speaker using the Windows sound card interface. After the virtual speaker receives the sound, the virtual sound card buffers the sound into the audio data buffer. The buffer size can be set, and it is generally recommended not to exceed a buffer size of 100ms. The audio data sending service forwards the audio buffer data to the multi-level audio synchronous sending service. Based on the NTP synchronization mechanism, the audio data is synchronously sent to multiple audio terminals [the speakers of the conference machine]. The sound of the application is ultimately broadcast synchronously on each conference machine.

[0071] Furthermore, the virtual speaker is controlled by the multi-machine software. Audio from the conference software is broadcast through the virtual speaker. The virtual speaker forwards the audio data to the core service SnapServer, which synchronizes the audio data with the multi-machine software. SnapServer supports multiple client connections, enabling synchronized audio broadcast from conference machine 1 to conference machine N.

[0072] Because the virtual speaker's default audio interface can't directly send and receive audio data with the core SnapServer service, a customized speaker audio data forwarding service is required, providing a data cache queue and audio forwarding services. When the conferencing software sends audio to the virtual speaker, the virtual speaker doesn't play the audio directly; instead, it pushes the audio data into the data cache queue. The audio forwarding service, a separate thread, continuously retrieves audio data from the data cache queue and sends it to the core SnapServer service. The SnapServer ultimately forwards the data to each SnapClient for playback.

[0073] Alternatively, please refer to Figure 6, which illustrates another data exchange diagram for a unified virtual sound card provided by an embodiment of the present invention. The multi-channel conference machine utilizes the multi-machine software's sound mixing mechanism to synchronously integrate multiple audio channels and convert them into a single audio channel, which is then forwarded to the virtual microphone. This virtual microphone then provides audio capture to third-party applications such as conferencing software and audio and video recording software. This sound mixing requires a time synchronization mechanism to ensure consistent time synchronization between the multi-channel conference machine and the host computer hosting the multi-machine software.

[0074] Because the virtual microphone's default audio interface can't directly send and receive audio data with the multi-machine software's sound mixing output unit, a customized microphone audio data forwarding service is required, providing a data cache queue and audio forwarding service. When the multi-machine software sends audio to the virtual microphone, the virtual microphone doesn't directly receive the audio. The sound mixing output unit sends the audio data to the microphone audio data forwarding service, which then pushes the received audio into the data cache queue. The virtual microphone, as an independent thread, periodically retrieves audio data from the microphone data cache queue and sends it to the conference machine software, which then forwards it to each peer conference terminal.

[0075] It's understandable that the speakers and microphones of the multi-machine virtual sound card have a one-to-many connection. The multi-machine microphone's speakers are actually output to multiple conference machines for simultaneous playback. This is achieved by broadcasting sound from the virtual speakers to multiple conference machines, controlled by NTP time synchronization and automatic playback speed adjustment.

[0076] The multi-machine virtual sound card collects audio from the microphones of multiple conference devices. The conference devices collect the audio, eliminate background noise, and automatically adjust the volume. The multiple audio channels are synchronously mixed and then combined into a single audio channel that is provided to the virtual microphone.

[0077] In addition, please refer to Figures 7 to 10. In Figure 7, the correspondence between the conference software Kandao Meeting and its corresponding virtual microphone and virtual speaker is shown. Figure 8 is a display interface of the virtual speaker provided in an embodiment of the present invention. The user can adjust the parameters of the virtual speaker through this display interface without having to adjust the parameters of the virtual speakers corresponding to multiple virtual sound cards. Similarly, Figure 9 is a display interface of the virtual microphone provided in an embodiment of the present invention. The user can adjust the parameters of the virtual microphone through this display interface without having to adjust the parameters of the virtual microphones corresponding to multiple virtual sound cards. Figure 10 is a unified virtual sound card of the conference software in a conference scenario. The unified virtual sound card only includes one virtual microphone and one virtual speaker, that is, the input and output of audio data to the target software is realized through one virtual sound card.

[0078] The audio processing method based on multi-machine interaction provided in this embodiment responds to the audio transmission instruction triggered by the local software, obtains the first audio data corresponding to the audio transmission instruction, synchronizes the first audio data to the target software through the first virtual interface of the virtual speaker, so that the local software can output the audio data, receives the second audio data sent by the target software, sends the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and sends the second audio data in the virtual microphone to the local software, so that the local software can input the audio data. In the audio processing scheme provided by the present invention, the local software can output the first audio data to the target software through the first virtual interface of the virtual speaker, and the target software can input the second audio data to the local software through the second virtual interface of the virtual microphone, that is, the input and output of audio data to the target software is realized through a virtual sound card. It can be seen that in the scenario of multi-machine interaction, there is no need to configure multiple virtual sound cards, thereby avoiding adjustment errors due to different naming of virtual sound cards.

[0079] Accordingly, referring to FIG11 , an embodiment of the present invention provides an audio processing device based on multi-machine interaction (hereinafter referred to as a processing device), including:

[0080] The acquisition module 201 is configured to acquire first audio data corresponding to the audio transmission instruction in response to the audio transmission instruction triggered by the local software.

[0081] The synchronization module 202 is used to synchronize the first audio data to the target software through the first virtual interface of the virtual speaker so that the local software can output the audio data. The virtual speaker is a pre-set virtual output device of the local software; wherein the virtual speaker is used to control the output audio data.

[0082] The receiving module 203 is configured to receive second audio data sent by the target software, where the second audio data is an integration result of the user audio data collected by the target software.

[0083] The sending module 204 is used to send the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and send the second audio data in the virtual microphone to the local software so that the local software can input audio data. The virtual microphone is a virtual input device pre-set for the local software; wherein the virtual microphone is used to control the input audio data.

[0084] Optionally, in some embodiments, the synchronization module 202 may specifically include:

[0085] A buffering unit, configured to buffer the first audio data into a first preset buffer through the virtual speaker;

[0086] The synchronization unit is used to synchronize the first audio data buffered in the first preset buffer to the terminal corresponding to the target software through the first virtual interface of the virtual speaker based on the speaker forwarding service.

[0087] Optionally, in some embodiments, the synchronization unit may specifically include:

[0088] a buffer subunit, configured to forward the first audio data buffered in the first preset buffer to the core service through the first virtual interface of the virtual speaker in response to an extraction instruction of the speaker forwarding service for the first preset buffer;

[0089] The synchronization subunit is used to synchronize the first audio data to the target terminal corresponding to the target software based on the core service and the network time protocol.

[0090] Optionally, in some embodiments, the synchronization sub-unit can be specifically used to: determine that the software in the same core service network as the local software is the target software; determine the transmission delay according to the network time protocol; and based on the transmission delay, synchronize the forwarded first audio data to the target terminal corresponding to the target software through the core service.

[0091] Optionally, in some embodiments, the receiving module 203 may be further configured to: control the virtual microphone to collect user audio data of a physical microphone corresponding to the target software; and integrate multiple channels of user audio data to obtain second audio data.

[0092] Optionally, in some embodiments, the sending module 204 can be specifically used to: based on the microphone forwarding service, buffer the second audio data into a second preset buffer through the second virtual interface of the virtual microphone; control the virtual microphone to extract the buffered second audio data from the second preset buffer, and send the extracted second audio data to the local terminal corresponding to the local software.

[0093] Optionally, in some embodiments, the sending module 204 may be specifically configured to: in response to a data transmission instruction of the speaker forwarding service for the second preset buffer, buffer the second audio data into the second preset buffer through the second virtual interface of the virtual microphone.

[0094] Optionally, in some embodiments, referring to FIG. 12 , the processing of the present invention may further include a binding module 205, which may be specifically used to: pre-set a virtual sound card including a virtual speaker and a virtual microphone; and bind the virtual sound card to local software so that the local software can input and output data through the virtual sound card.

[0095] As can be seen from the above, in the audio processing device based on multi-machine interaction of this embodiment, the acquisition module 201 responds to the audio transmission instruction triggered by the local software to obtain the first audio data corresponding to the audio transmission instruction, the synchronization module 202 synchronizes the first audio data to the target software through the first virtual interface of the virtual speaker so that the local software can output the audio data, the receiving module 203 receives the second audio data sent by the target software, and the sending module 204 sends the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and sends the second audio data in the virtual microphone to the local software so that the local software can input the audio data. In the audio processing solution provided by the present invention, the local software can output the first audio data to the target software through the first virtual interface of the virtual speaker, and the target software can input the second audio data to the local software through the second virtual interface of the virtual microphone, that is, the input and output of audio data to the target software is realized through a virtual sound card. It can be seen that in the scenario of multi-machine interaction, there is no need to configure multiple virtual sound cards, thereby avoiding the adjustment error caused by different naming of virtual sound cards.

[0096] An embodiment of the present invention further provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is loaded and executed by the controller, the method steps described in any of the above method embodiments are implemented.

[0097] An embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores a computer program. When the computer program is executed, the method steps described in any one of the above method embodiments are implemented.

[0098] In the above embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0099] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0100] In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software functional units. If the integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium.

[0101] Based on this understanding, the technical solution of the present application, or the contributing part, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a mobile terminal, personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0102] In summary, although the present invention has been disclosed as above in terms of preferred embodiments, the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the concept of the technical solution of the present invention, should be covered by the scope of protection of the present invention.

[0103] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An audio processing method based on multi-machine interaction, comprising: In response to an audio transmission instruction triggered by local software, obtaining first audio data corresponding to the audio transmission instruction; Buffering the first audio data into a first preset buffer through the virtual speaker; Based on the speaker forwarding service, synchronizing the first audio data buffered in the first preset buffer to the terminal corresponding to the target software through the first virtual interface of the virtual speaker; The virtual speaker is a virtual output device of the local software that is preset; wherein the virtual speaker is used to control the output audio data; Receiving second audio data sent by the target software, where the second audio data is an integration result of user audio data collected by the target software; Based on the microphone forwarding service, buffering the second audio data into a second preset buffer through a second virtual interface of the virtual microphone; Control the virtual microphone to extract the buffered second audio data from the second preset buffer, and send the extracted second audio data to the local terminal corresponding to the local software; the virtual microphone is a virtual input device pre-set for the local software; wherein the virtual microphone is used to control the input audio data.

2. An audio processing method based on multi-machine interaction, comprising: In response to an audio transmission instruction triggered by local software, obtaining first audio data corresponding to the audio transmission instruction; The first audio data is synchronized to the target software through a first virtual interface of the virtual speaker so that the local software outputs the audio data, and the virtual speaker is a virtual output device of the local software that is preset; wherein the virtual speaker is used to control the output audio data; Receiving second audio data sent by the target software, where the second audio data is an integration result of user audio data collected by the target software; The second audio data is sent to the virtual microphone through the second virtual interface of the virtual microphone, and the second audio data in the virtual microphone is sent to the local software so that the local software can input audio data. The virtual microphone is a pre-set virtual input device of the local software; wherein the virtual microphone is used to control the input audio data.

3. The method according to claim 2, wherein synchronizing the first audio data to the target software through the first virtual interface of the virtual speaker comprises: Buffering the first audio data into a first preset buffer through the virtual speaker; Based on the speaker forwarding service, the first audio data buffered in the first preset buffer is synchronized to the terminal corresponding to the target software through the first virtual interface of the virtual speaker.

4. The method according to claim 3, wherein the step of synchronizing the first audio data buffered in the first preset buffer to the terminal corresponding to the target software through the first virtual interface of the virtual speaker based on the speaker forwarding service comprises: In response to an extraction instruction of the speaker forwarding service for the first preset buffer, forwarding the first audio data buffered in the first preset buffer to the core service through the first virtual interface of the virtual speaker; Based on the core service and the network time protocol, the first audio data is synchronized to a target terminal corresponding to the target software.

5. The method according to claim 4, wherein the step of synchronizing the first audio data to a target terminal corresponding to a target software based on the core service and the network time protocol comprises: Determine the software in the same core service network as the local software as the target software; Determine the transmission delay according to the network time protocol; Based on the transmission delay, the forwarded first audio data is synchronized to the target terminal corresponding to the target software through the core service.

6. The method according to claim 2, wherein sending the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and sending the second audio data in the virtual microphone to the local terminal corresponding to the local software, comprises: Based on the microphone forwarding service, buffering the second audio data into a second preset buffer through a second virtual interface of the virtual microphone; The virtual microphone is controlled to extract the buffered second audio data from the second preset buffer, and the extracted second audio data is sent to the local terminal corresponding to the local software.

7. The method according to claim 6, wherein the buffering of the second audio data into a second preset buffer through a second virtual interface of the virtual microphone based on the microphone forwarding service comprises: In response to a data transmission instruction of the speaker forwarding service for the second preset buffer, the second audio data is buffered in the second preset buffer through the second virtual interface of the virtual microphone.

8. The method according to claim 2, wherein before receiving the second audio data sent by the target software, the method comprises: Control the virtual microphone to collect user audio data of the physical microphone corresponding to the target software; The multiple channels of user audio data are integrated to obtain second audio data.

9. The method according to claim 2, characterized in that: Also includes: A virtual sound card with a virtual speaker and a virtual microphone is pre-configured; The virtual sound card is bound to the local software so that the local software can input and output data through the virtual sound card.

10. An audio processing device based on multi-machine interaction, comprising: An acquisition module, configured to acquire first audio data corresponding to the audio transmission instruction in response to the audio transmission instruction triggered by the local software; A synchronization module, used for synchronizing the first audio data to the target software through a first virtual interface of the virtual speaker, so that the local software outputs the audio data, wherein the virtual speaker is a pre-set virtual output device of the local software; wherein the virtual speaker is used to control the output audio data; A receiving module, used for receiving second audio data sent by the target software, where the second audio data is an integration result of user audio data collected by the target software; A sending module is used to send the second audio data to the virtual microphone through the second virtual interface of the virtual microphone, and send the second audio data in the virtual microphone to the local software so that the local software can input audio data. The virtual microphone is a pre-set virtual input device of the local software; wherein the virtual microphone is used to control the input audio data.

11. A storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the steps of the audio processing method based on multi-machine interaction as claimed in claim 1 are implemented.

Images