Audio sharing method, apparatus, terminal, storage medium, and program product

Abstract

The application discloses an audio sharing method and device, a terminal, a storage medium and a program product, and relates to the technical field of communication. The method comprises the following steps: acquiring first audio played by a terminal under the condition that an audio sharing function is started, the first audio being audio played by an application other than an audio sharing application in the terminal, the audio sharing application being used for sharing audio played by the terminal to other terminals, and the acquisition process of the first audio not passing through a microphone collection link; acquiring second audio obtained after echo cancellation, the echo cancellation being used for eliminating the audio played by the terminal contained in microphone collection audio; and performing audio mixing on the first audio and the second audio to obtain third audio, the third audio being used for sharing to other terminals. The method can improve the audio quality of shared audio.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an audio sharing method, apparatus, terminal, storage medium, and program product. Background Technology

[0002] Audio sharing refers to transmitting system sounds (such as music or video sounds played by other applications) and nearby audio from a local device to a remote device.

[0003] In related technologies, the local device plays system sound through a speaker. The system sound is picked up by a microphone through an echo path. At the same time, the microphone can pick up near-end speech to obtain the total microphone acquisition signal. This microphone acquisition signal is then processed by echo cancellation, noise reduction, and audio encoding before being transmitted to a remote device.

[0004] Obviously, after the system sound travels through the echo path, is captured by the microphone, and undergoes echo cancellation, it will cause severe distortion of the system sound, thus affecting the quality of the audio signal transmitted to the remote end. Summary of the Invention

[0005] This application provides an audio sharing method, apparatus, terminal, storage medium, and program product. The technical solution is as follows:

[0006] According to one aspect of this application, an audio sharing method is provided, the method being executed by a terminal, the method comprising:

[0007] With the audio sharing function enabled, the first audio played by the terminal is acquired. The first audio is audio played by an application other than the audio sharing application on the terminal. The audio sharing application is used to share the audio played by the terminal with other terminals. The acquisition process of the first audio does not go through the microphone acquisition link.

[0008] The second audio obtained after echo cancellation is acquired, wherein the echo cancellation is used to remove the audio played by the terminal contained in the audio collected by the microphone;

[0009] The first audio and the second audio are mixed to obtain a third audio, which is used to share with the other terminals.

[0010] According to another aspect of this application, an audio sharing device is provided, the device comprising:

[0011] The first acquisition module is used to acquire the first audio played by the terminal when the audio sharing function is enabled. The first audio is the audio played by an application other than the audio sharing application on the terminal. The audio sharing application is used to share the audio played by the terminal with other terminals. The acquisition process of the first audio does not go through the microphone acquisition link.

[0012] The second acquisition module is used to acquire the second audio after echo cancellation, wherein the echo cancellation is used to remove the audio played by the terminal contained in the audio collected by the microphone;

[0013] The mixing module is used to mix the first audio and the second audio to obtain a third audio, which is used to share with the other terminals.

[0014] According to another aspect of this application, a terminal is provided, the terminal including a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the audio sharing method as described above.

[0015] According to another aspect of this application, a computer-readable storage medium is provided, wherein at least one program is stored therein, the at least one program being loaded and executed by a processor to implement the audio sharing method as described above.

[0016] According to another aspect of this application, a computer program product is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a terminal reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal to perform the audio sharing method provided in the above-described optional implementation.

[0017] The beneficial effects of the technical solutions provided in this application include at least the following:

[0018] In this embodiment, audio played by applications other than the audio sharing application on the terminal (first audio) and microphone signals obtained after eliminating the audio played on the terminal (second audio) are acquired separately. The first and second audios are then mixed to obtain a third audio for audio sharing. This avoids acquiring the first audio through a microphone acquisition link, eliminating the need for speakers, microphones, air, and other media, as well as avoiding audio processing such as echo cancellation and noise reduction. This significantly improves the distortion problem of the first audio, thereby enhancing its audio quality and ultimately improving the audio quality of the shared audio. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of the structure of an echo cancellation system in the related technology is shown;

[0021] Figure 2 This is a schematic diagram of a computer system illustrated in an exemplary embodiment of this application;

[0022] Figure 3 A flowchart illustrating an exemplary embodiment of the audio sharing method provided in this application is shown;

[0023] Figure 4 A flowchart of an audio sharing method provided by another exemplary embodiment of this application is shown;

[0024] Figure 5 This illustration shows a schematic diagram of the audio acquisition process of the process loopback acquisition interface, as shown in an exemplary embodiment of this application.

[0025] Figure 6 This illustration shows a schematic diagram of the audio acquisition process of a global loopback acquisition interface, as shown in an exemplary embodiment of this application.

[0026] Figure 7 A flowchart of an audio sharing method provided by another exemplary embodiment of this application is shown;

[0027] Figure 8 A schematic diagram illustrating the principle of an audio sharing method according to an exemplary embodiment of this application is shown;

[0028] Figure 9 A flowchart of an audio sharing method provided by another exemplary embodiment of this application is shown;

[0029] Figure 10 A schematic diagram illustrating the principle of an audio sharing method shown in another exemplary embodiment of this application is provided;

[0030] Figure 11 A flowchart of an audio sharing method provided by another exemplary embodiment of this application is shown;

[0031] Figure 12 A schematic diagram illustrating the principle of an audio sharing method shown in another exemplary embodiment of this application is provided;

[0032] Figure 13This is a structural block diagram of an audio sharing device provided in an exemplary embodiment of this application;

[0033] Figure 14 A structural block diagram of a terminal provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0035] Taking a two-person call scenario as an example, when the remote user speaks, their remote microphone picks up the voice signal and sends it to the near-end application 120 via the remote application 130. After receiving and decoding the packet, the near-end application 120 obtains the remote signal x, which is then sent to the speaker for playback via the operating system interface. The microphone signal d includes the sound signal played by the near-end speaker and the sound signal of the near-end user speaking. If signal d is directly encoded and sent to the remote application 130 without echo cancellation processing, the remote user will hear their own voice. Figure 1 A schematic diagram of the echo cancellation system in the related technology is shown. After receiving and decoding packets, the near-end application 120 obtains the far-end signal x, which is sent to the speaker for playback via the operating system interface 110. The echo cancellation module is generally located on the microphone link of the near-end application 120. It uses the microphone signal d as the desired signal and the far-end signal x as the reference signal to eliminate the components in signal d that are related to signal x and retain the components that are unrelated to signal x.

[0036] Depend on Figure 1 As shown in the echo cancellation process, since the system sound (the audio that needs to be shared to the terminal of the remote application 130 for playback) will pass through devices such as speakers and microphones and transmission media such as air, and will undergo audio processing such as echo cancellation and noise reduction, it may cause serious distortion of the system sound.

[0037] To alleviate Figure 1 To address the issue of audio distortion during audio sharing, this application provides an audio sharing method based on multiple audio acquisition interfaces. Figure 2 This is a schematic diagram of a computer system illustrating an exemplary embodiment of this application. Figure 2 As shown, the computer system includes a first terminal 210 and at least one second terminal 220.

[0038] The first terminal 210 is a device running an audio sharing application. The audio sharing application 111 can be an instant messaging application, a call application, a live streaming application, etc. In this embodiment, the audio sharing application is configured with three audio sharing modes. When the first terminal 210 is configured with both a process loopback acquisition interface and a global loopback acquisition interface, the first audio sharing mode 112 can be switched to; when the first terminal 210 is configured with only a process loopback acquisition interface, the second audio sharing mode 113 can be switched to; and when the first terminal 210 is configured with only a global loopback acquisition interface, the third audio sharing mode 114 can be switched to.

[0039] Optionally, during the audio sharing process, the first terminal 210 acquires the system sound (first audio) to be shared through its internal audio acquisition interface, so that the system sound does not need to pass through a medium such as a speaker, microphone, or air, or undergo audio processing such as echo cancellation and noise reduction, thus avoiding the distortion problem of the first audio; the first terminal 210 also acquires the second audio after echo cancellation (the second audio does not include the audio played by the terminal), so that by mixing the first audio and the second audio, the shared audio to be shared with other terminals (second terminal 220) can be obtained.

[0040] To illustrate, in a live streaming scenario, if the audio sharing application is an educational live streaming application, the first audio that the host (teacher) might share with the second terminal 220 via the educational live streaming application could be: educational audio and video played on the first terminal 210, and the second audio could be the teacher's online explanation. In a voice call scenario, if the audio sharing application is an instant messaging application, the first audio that the user of the first terminal 210 might share with the second terminal 220 via the instant messaging application could be: music played by a music application on the first terminal 210, and the second audio could be the user's voice, etc.

[0041] Optionally, the first terminal 210 may include, but is not limited to, smartphones, tablets, desktop computers, smart voice interaction devices, smart home appliances, and in-vehicle terminals.

[0042] The second terminal 220 is a device running an audio sharing application. The audio sharing application can be an instant messaging application, a call application, a live streaming application, etc. In this embodiment, the second terminal 220 can receive audio shared by the first terminal 210 and can also send shared audio to the first terminal 210. Optionally, the audio sharing application in the second terminal 220 can also be configured with three audio sharing modes, allowing the selection of the corresponding audio sharing mode based on the type of the internal audio acquisition interface of the second terminal 220.

[0043] Please refer to Figure 3This document illustrates a flowchart of an audio sharing method provided in an exemplary embodiment of this application. The embodiments of this application use the application of this method to a terminal as an example, and the method includes:

[0044] Step 301: With the audio sharing function enabled, acquire the first audio played on the terminal. The first audio is the audio played by an application other than the audio sharing application on the terminal. The acquisition process of the first audio does not go through the microphone acquisition link.

[0045] The audio sharing application is used to share audio played on a terminal with other terminals. Alternatively, the audio sharing application can also share audio played on the terminal together with nearby audio with other terminals. Nearby audio can be nearby speech, such as voice uttered by a nearby user, or audio played by other nearby devices (such as music played by a smart speaker near the terminal). Optionally, the audio sharing application can be a call application, instant messaging application, live streaming application, social application, etc. This embodiment does not limit the specific application of the audio sharing application.

[0046] When an audio sharing application has its audio sharing function enabled, the audio that the application needs to share with other devices includes at least two types: audio played by other applications on the terminal besides the audio sharing application itself, and near-end audio captured by the microphone. If the audio sharing method in related technologies is used, the near-end audio and audio played by other applications are captured by the microphone. The audio played by other applications passes through speakers, microphones, and other devices, as well as the air and other transmission media. Furthermore, the signal captured by the microphone undergoes audio processing such as echo cancellation and noise reduction, which can lead to severe distortion in the audio played by other applications, thus affecting the quality of the shared audio. To alleviate the distortion problem of audio played by other applications, this application provides an audio sharing method that, when the audio sharing function is enabled, directly and separately acquires the first audio played by the terminal within the terminal. This acquisition process of the first audio does not pass through the microphone acquisition link; that is, the first audio is not mixed with the near-end audio before being acquired by the microphone acquisition link. This avoids the influence of speakers, microphones, and other media such as the air on the first audio, as well as the distortion problems caused by audio processing such as echo cancellation and noise reduction, thus improving the audio quality of the first audio.

[0047] Optionally, the first audio file can be obtained from the terminal's internal audio acquisition interface. This audio acquisition interface can be a process loopback acquisition interface or a global loopback acquisition interface.

[0048] Step 302: Obtain the second audio after echo cancellation. Echo cancellation is used to remove the audio played by the terminal from the audio captured by the microphone.

[0049] To separately acquire the first audio (audio played by other applications) and the near-end audio (audio captured by the microphone but not played by the terminal), unlike related technologies (which require retaining near-end audio and audio played by other applications during echo cancellation), the terminal only needs to retain the near-end audio when performing echo cancellation on the microphone-captured audio. Correspondingly, in one possible implementation, after acquiring the microphone-captured audio, when performing echo cancellation on it, the terminal needs to eliminate all audio played by the terminal included in the microphone-captured audio, such as audio played by an audio sharing application on the terminal, and audio played by other applications besides the audio sharing application, retaining only the near-end audio captured by the microphone to obtain the second audio after echo cancellation.

[0050] Step 303: Mix the first audio and the second audio to obtain a third audio, which is then shared with other terminals.

[0051] Since the audio shared with other terminals needs to include not only audio played by other applications but also near-end audio, after the terminal obtains the first audio (audio played by applications other than the audio sharing application) and the second audio (near-end audio), it needs to perform audio mixing processing on the first and second audio to obtain the mixed third audio. Then, after audio encoding of the third audio, it is shared with other terminals through the audio sharing application.

[0052] In summary, in this embodiment, by separately acquiring the audio played by applications other than the audio sharing application on the terminal (first audio) and the microphone signal obtained after eliminating the audio played on the terminal (second audio), and then mixing the first and second audio, a third audio for audio sharing is obtained. This avoids acquiring the first audio through the microphone acquisition link, eliminating the need for the first audio to pass through speakers, microphones, air, or other media, and eliminating the need for audio processing such as echo cancellation and noise reduction. This significantly improves the distortion problem of the first audio, thereby improving its audio quality and ultimately enhancing the audio quality of the audio sharing.

[0053] In scenarios where the microphone acquisition link is not used to acquire the first audio, the audio played by the terminal needs to be directly acquired by the internal audio acquisition interface configured on the terminal. Different audio acquisition interfaces will also affect the acquisition method of the first and second audio. Correspondingly, in one possible implementation, when the audio sharing function is enabled, it is necessary to combine the interface type of the audio acquisition interface configured on the terminal to determine the specific audio acquisition method of the first and second audio.

[0054] like Figure 4The diagram illustrates a flowchart of an audio sharing method provided in another exemplary embodiment of this application. This embodiment uses the application of this method to a terminal as an example for illustration. The method includes:

[0055] Step 401: With the audio sharing function enabled, determine the interface type of the audio acquisition interface configured on the terminal. The audio acquisition interface is used to acquire the audio played by the terminal. The acquisition methods of the first and second audio are different under different interface types.

[0056] In order to obtain the first audio (audio played by applications other than the audio sharing application on the terminal) without going through the microphone acquisition link, the first audio is obtained through the internal audio acquisition interface of the terminal. However, since the internal audio acquisition interfaces of different terminals are different, in order to adapt to different terminal environments, different methods for obtaining the first and second audio are set for different interface types. This allows the audio application to determine the interface type of the audio acquisition interface configured on the terminal during actual audio sharing, and select the corresponding audio acquisition method to obtain the first and second audio.

[0057] Optionally, the audio acquisition interfaces involved in audio sharing scenarios mainly include process loopback acquisition interfaces and global loopback acquisition interfaces. Both types of audio acquisition interfaces can directly acquire audio played on the terminal without going through a microphone acquisition link. The main difference is the audio content acquired. Among them, the process loopback acquisition interface is mainly used to acquire audio played by applications other than the audio sharing application on the terminal, while the global loopback acquisition interface is used to acquire all audio played on the terminal (including audio played by the audio sharing application and audio played by applications other than the audio sharing application).

[0058] like Figure 5 The diagram illustrates an exemplary embodiment of the audio acquisition process of a process loopback acquisition interface according to this application. In the process loopback acquisition process 500, the audio sharing application 510 is playing remote data 511 (remote data 511 is audio data obtained by the audio sharing application 510 after decoding the received audio stream), while other applications 520 (other applications in the terminal besides the audio sharing application) may also be playing other audio data 521. The process loopback acquisition interface can acquire audio streams from the operating system, and it can be specified that the remote data 511 played by the audio sharing application 510 will not be acquired. Therefore, the loopback data 512 acquired by the process loopback acquisition interface only contains other audio data 521 played by other applications 520. Optionally, the audio played by the speaker is a mixture of remote data 511 and other audio data 521.

[0059] like Figure 6The diagram illustrates an exemplary embodiment of the audio acquisition process of a global loopback acquisition interface according to this application. In the global loopback acquisition process 600, while the audio sharing application 610 is playing remote data 611 (remote data 611 is audio data obtained after the audio sharing application 610 decodes the received audio stream), other applications 620 may also be playing other audio data 621. The global loopback acquisition interface can acquire the remote data played by the audio sharing application 610 itself and other audio data 621 played by other applications 620; that is, the loopback data 612 contains remote data 611 and other audio data 621.

[0060] Optionally, based on the case where the audio acquisition interface includes a process loopback acquisition interface and a global loopback acquisition interface, the terminal configuration interface type includes the following three scenarios: one is to configure only the process loopback acquisition interface, one is to configure only the global loopback acquisition interface, and one is to configure both the process loopback acquisition interface and the global loopback acquisition interface.

[0061] The method for determining the audio acquisition interface configured on the terminal varies depending on the operating system and its version. Therefore, the interface type of the terminal's audio acquisition interface can be determined based on the operating system type and version. In an exemplary example, step 401 may include steps 401A and 401B.

[0062] Step 401A: With the audio sharing function enabled, obtain the operating system type of the terminal's configured operating system.

[0063] Step 401B: Determine the interface type of the audio acquisition interface configured on the terminal based on the operating system type.

[0064] Because the audio acquisition interfaces configured on terminals differ across operating system types—for example, Windows 10 may have both a global loopback acquisition interface and a process loopback acquisition interface, while iOS may only have a process loopback acquisition interface—one possible implementation involves obtaining the operating system type of the terminal when the audio sharing application has its audio sharing function enabled. Based on the relationship between the operating system type and the interface type, the interface type of the audio acquisition interface configured on the terminal can then be determined.

[0065] Furthermore, the audio acquisition interfaces configured for different operating system versions within the same type of operating system may also differ. Correspondingly, within the same type of operating system, the interface type of the audio acquisition interface configured on the terminal can be further determined based on the operating system itself. In an exemplary example, step 401B may further include steps 401B1 and 401B2.

[0066] Step 401B1: If the operating system is a Type I operating system, determine the interface type of the audio acquisition interface configured on the terminal based on the operating system version.

[0067] The first type of operating system can be Windows. For Windows, the process loopback acquisition interface is only supported on newer versions, while older versions only have a global loopback acquisition interface. Correspondingly, in one possible implementation, when the audio sharing application determines that the operating system is of the first type, it can further determine the operating system version in order to determine the interface type of the audio acquisition interface configured on the terminal based on the operating system version.

[0068] Optionally, if the operating system is a Category 1 operating system and its version level is higher than the target level, it is determined that the terminal is configured with both a process loopback acquisition interface and a global loopback acquisition interface. For example, the target level could be 10, on a newer version of Windows 10, or a version higher than Windows 10, and the terminal is configured with both a global loopback acquisition interface and a process loopback acquisition interface.

[0069] Optionally, in the Windows operating system, the IAudioClient interface can be set to process loopback acquisition mode via a system call, or the IAudioClient interface can be set to global loopback acquisition mode via a system call.

[0070] Optionally, if the operating system is a Category 1 operating system and its version level is below the target level, it is determined that the terminal has a global loopback acquisition interface configured, but no process loopback acquisition interface is configured. For illustration, on older versions of Windows 10 or even older Windows 7 operating systems, the terminal only has a global loopback acquisition interface configured.

[0071] Step 401B2: If the operating system is a Type II operating system, determine that the terminal is configured with a process loopback acquisition interface but not a global loopback acquisition interface.

[0072] The second type of operating system can be iOS. When the audio sharing application determines that the terminal is an iOS operating system, it can be determined that the terminal is only configured with a process loopback acquisition interface and not a global loopback acquisition interface.

[0073] Optionally, in the iOS operating system, process loopback acquisition can be achieved through Replaykit.

[0074] Optionally, for Windows operating systems, to improve the accuracy of determining the interface type of the audio acquisition interface configured on the terminal, in other possible implementations, when the audio sharing application has the audio sharing function enabled, it can also attempt to start the process loopback acquisition interface to determine whether the terminal is also configured with a process loopback acquisition interface. Corresponding to the case of a Type I operating system, the audio sharing application sends a startup command to the operating system to start the process loopback acquisition interface; if the audio sharing application receives feedback indicating successful startup, it determines that the terminal is configured with both a global loopback acquisition interface and a process loopback acquisition interface; if the audio sharing application receives feedback indicating startup failure, it determines that the terminal is configured with a global loopback acquisition interface but not a process loopback acquisition interface.

[0075] Step 402: Obtain the first audio played by the terminal based on the first audio acquisition method corresponding to the interface type.

[0076] Step 403: Based on the second audio acquisition method corresponding to the interface type, obtain the second audio after echo cancellation.

[0077] Optionally, once the audio sharing application determines the interface type of the terminal, it can select a first audio acquisition method and a second audio acquisition method that match the interface type, so as to acquire the first audio played by the terminal through the first audio acquisition method and acquire the second audio after echo cancellation through the second audio acquisition method.

[0078] Based on the interface types included in the terminal, there are three corresponding methods for acquiring the first and second audio: the first and second audio acquisition methods when the terminal is only configured with a process loopback acquisition interface; the first and second audio acquisition methods when the terminal is only configured with a global loopback acquisition interface; and the first and second audio acquisition methods when the terminal is configured with both a global loopback acquisition interface and a process loopback acquisition interface. The specific first and second audio acquisition methods are described in detail in the embodiments below.

[0079] Step 404: Mix the first audio and the second audio to obtain a third audio, which is then shared with other terminals.

[0080] The implementation method of step 404 can refer to step 303, and will not be repeated here in this embodiment.

[0081] In this embodiment, by obtaining the operating system type and version of the terminal, the interface type of the audio acquisition interface configured on the terminal is determined. Then, by setting different audio acquisition methods for different interface types, the first audio can be obtained without going through the microphone acquisition link under the terminal configuration, thereby expanding the application scenarios of the audio sharing method.

[0082] The following examples provide a detailed explanation of the audio acquisition methods corresponding to different interface types. First, for terminals equipped with both process loopback acquisition interfaces and global loopback acquisition interfaces, the process of acquiring the first and second audio audio will be explained. Please refer to... Figure 7 This document illustrates a flowchart of an audio sharing method provided in another exemplary embodiment of this application. The embodiments of this application use the method applied to a terminal as an example for description. The method includes:

[0083] Step 701: With the audio sharing function enabled, determine the interface type of the audio acquisition interface configured on the terminal.

[0084] In this embodiment, when the audio sharing function is enabled, it is determined that the terminal is configured with a global loopback acquisition interface and a process loopback acquisition interface. The corresponding audio sharing application can switch to the audio sharing mode (first audio acquisition mode and second audio acquisition mode) corresponding to the global loopback acquisition interface and the process loopback acquisition interface.

[0085] The implementation method of step 701 can be referred to the above embodiment, and will not be repeated here.

[0086] Step 702: If the interface type includes a global loopback acquisition interface and a process loopback interface, acquire the first audio through the process loopback acquisition interface.

[0087] Since the process loopback acquisition interface is used to acquire audio from applications other than a specified process and its child processes in the terminal, and the specified process and its child processes are the processes related to the audio sharing application in the audio sharing scenario, and the first audio is the audio played by other applications besides the audio sharing application, based on the function of the process loopback acquisition interface, when the interface type in the terminal includes the process loopback acquisition interface and the global loopback acquisition interface, the terminal can directly acquire the first audio through the process loopback acquisition interface, and the corresponding audio sharing application can obtain the first audio from the process loopback acquisition interface.

[0088] Step 703: Acquire the fourth audio through the global loopback acquisition interface. The global loopback acquisition interface is used to acquire all audio played by the terminal.

[0089] Since the second audio is obtained by performing echo cancellation on the microphone-captured audio to remove all audio played by the terminal from the microphone-captured audio, it is necessary to use the entire audio played by the terminal as the reference signal and the microphone-captured audio as the desired signal, and then use the reference signal to perform echo cancellation on the desired signal. The global loopback acquisition interface is precisely designed to capture all audio played by the terminal. Therefore, in one possible implementation, if the terminal is configured with both a global loopback acquisition interface and a process loopback acquisition interface, the global loopback acquisition interface can be called first to capture the fourth audio (all audio played by the terminal). This fourth audio is captured before the speaker playback and has not passed through the microphone acquisition link, thus ensuring the audio quality of the fourth audio and avoiding distortion that could affect the echo cancellation effect.

[0090] Step 704: Acquire a fifth audio signal via a microphone. The fifth audio signal includes at least all the audio played by the terminal.

[0091] Optionally, the terminal also needs to collect the fifth audio through a microphone. If the terminal is playing audio, the audio played by the terminal is collected by the microphone through the echo path, and the fifth audio includes at least all the audio played by the terminal. Optionally, if the user is also speaking at the same time, the fifth audio may also include near-field speech.

[0092] Step 705: Perform echo cancellation on the fifth audio based on the fourth audio to obtain the second audio.

[0093] Optionally, after the audio sharing application obtains the fourth audio (all audio played by the terminal) and the fifth audio (audio captured by the microphone), in order to eliminate the audio played by the terminal contained in the audio captured by the microphone, the fourth audio can be used as a reference signal to perform echo cancellation on the fifth audio, thereby obtaining a second audio that does not contain the audio played by the terminal.

[0094] Step 706: Mix the first audio and the second audio to obtain a third audio, which is then shared with other terminals.

[0095] The implementation method of step 706 can be referred to the above embodiment, and will not be repeated here.

[0096] like Figure 8The diagram illustrates the principle of an audio sharing method according to an exemplary embodiment of this application. The operating system 820 is configured with a process loopback acquisition interface 821 and a global loopback acquisition interface 822. The audio sharing application 810 can use both interfaces simultaneously to achieve audio sharing. When the audio sharing application 810 receives an audio packet from a remote audio sharing application, it obtains a sixth audio 811 (the audio played by the audio sharing application 810) through the packet decoding module. The sixth audio 811 is sent to the kernel via the operating system API (Application Program Interface), mixed with audio played by all other applications 830, to obtain a fourth audio 813, which awaits playback from the speaker. The audio sharing application 810 calls the global loopback acquisition interface 822 to obtain the fourth audio 813 (the fourth audio 813 contains the sixth audio 811 and audio played by other applications 830). The fourth audio 813 serves as a reference signal for the echo cancellation module. The speaker plays the fourth audio 813, which, after passing through the echo path, is mixed with near-end speech and captured by the microphone to obtain a fifth audio 814. The fifth audio signal 814 will serve as the desired signal for the echo cancellation module. The echo cancellation module will remove the audio components in the fifth audio signal 814 related to the fourth audio signal 813, resulting in a residual signal (i.e., the second audio signal 815). Since the fourth audio signal 813 includes the sixth audio signal 811 and audio played by other applications 830, the second audio signal 815 only contains the estimation of near-end speech. The audio sharing application 810 will also call the process loopback acquisition interface 821 to obtain the first audio signal 812 being played by other applications 830. The first audio signal 812 and the second audio signal 815 will be sent to the mixing module to obtain the third audio signal 816. The third audio signal 816 includes the estimation of near-end speech and audio played by other applications 830, thus meeting the requirements of the audio sharing function. Finally, the third audio signal 816 is sent to the encoding and packet sending module for encoding and then sent to the remote audio sharing application.

[0097] In this embodiment, when the terminal is configured with both a process loopback acquisition interface and a global loopback acquisition interface, the first audio can be directly acquired using the process loopback acquisition interface, and the echo cancellation reference signal can be acquired using the global loopback acquisition interface, based on the audio content acquired by the microphone and the global loopback acquisition interface, respectively. This eliminates the portion of the terminal-played audio included in the microphone-acquired audio, thereby obtaining the second audio. During the acquisition of the first audio, the first audio does not need to be transmitted through media such as speakers, microphones, or air, and does not undergo echo cancellation processing, thus preserving the sound quality of the first audio to the maximum extent. During the mixing process, only the second audio may have residual echo, corresponding to less echo leakage. With only a single echo cancellation module deployed, the amount of echo computation is also relatively small.

[0098] In another possible application scenario, if the terminal is only configured with a global loopback acquisition interface, since the global loopback acquisition interface can only acquire all the audio played by the terminal and cannot directly obtain the audio played by other applications, it is necessary to separate the first audio from the audio acquired by the global loopback acquisition interface.

[0099] Please refer to Figure 9 This document illustrates a flowchart of an audio sharing method provided in another exemplary embodiment of this application. The embodiments of this application use the method applied to a terminal as an example for description. The method includes:

[0100] Step 901: With the audio sharing function enabled, determine the interface type of the audio acquisition interface configured on the terminal.

[0101] In this embodiment, when the audio sharing function is enabled, it is determined that the terminal is only configured with a global loopback acquisition interface and not a process loopback acquisition interface. The corresponding audio sharing application can switch to the audio sharing mode (first audio acquisition mode and second audio acquisition mode) corresponding to the global loopback acquisition interface.

[0102] The implementation method of step 901 can be referred to the above embodiment, and will not be repeated here.

[0103] Step 902: If the interface type includes a global loopback acquisition interface, the fourth audio is acquired through the global loopback acquisition interface. The global loopback acquisition interface is used to acquire all audio played by the terminal.

[0104] Step 903: Obtain the first audio played by the terminal from the fourth audio.

[0105] When the terminal is not configured with a process loopback acquisition interface, the audio sharing application cannot directly obtain the first audio. However, the audio collected by the global loopback acquisition interface contains both the first audio and the audio played by the audio sharing application. Therefore, the audio sharing application can obtain the first audio from the audio collected by the global loopback acquisition interface. Correspondingly, if it is determined that the terminal is only configured with a global loopback acquisition interface, the audio sharing application can call the global loopback acquisition interface to collect the fourth audio (all audio played by the terminal), and then separate the first audio played by the terminal from the fourth audio.

[0106] Optionally, echo cancellation can be used to obtain the first audio from the fourth audio. In an exemplary example, step 903 may further include steps 903A and 903B.

[0107] Step 903A: Obtain the sixth audio stream played by the audio sharing application. The sixth audio stream is obtained by decoding the audio stream received by the audio sharing application.

[0108] Step 903B: Echo cancellation is performed on the fourth audio based on the sixth audio to obtain the first audio played by the terminal.

[0109] Since the global loopback acquisition interface acquires all audio played by the terminal (the fourth audio), this total audio includes not only the audio played by the audio sharing application but also audio played by other applications besides the sharing application. If it is necessary to obtain the audio played by other applications (the first audio), the audio played by the audio sharing application needs to be removed from the fourth audio. Therefore, in one possible implementation, the terminal can acquire the sixth audio played by the audio sharing application, use the sixth audio as a reference signal to perform echo cancellation on the fourth audio, thereby removing the audio played by the audio sharing application from the fourth audio and obtaining the audio played by other applications besides the sharing application (the first audio).

[0110] Step 904: Acquire a fifth audio signal via a microphone. The fifth audio signal includes at least all the audio played by the terminal.

[0111] Since the second audio is obtained by performing echo cancellation on the microphone-captured audio to remove all audio played by the terminal from the microphone-captured audio, it is necessary to use the entire audio played by the terminal as the reference signal and the microphone-captured audio as the desired signal, and then use the reference signal to cancel the echo of the desired signal. The global loopback acquisition interface is precisely designed to capture all audio played by the terminal. Therefore, in one possible implementation, if the terminal is equipped with a global loopback acquisition interface, the global loopback acquisition interface can be called first to capture the fourth audio (all audio played by the terminal), and then the fifth audio captured by the microphone can be obtained. Since the terminal is playing audio, the audio played by the terminal is captured by the microphone through the echo path, so the fifth audio includes at least all audio played by the terminal; optionally, if the user is also speaking at the same time, the fifth audio may also include near-field speech.

[0112] Step 905: Perform echo cancellation on the fifth audio based on the fourth audio to obtain the second audio.

[0113] Optionally, after the audio sharing application obtains the fourth audio (all audio played by the terminal) and the fifth audio (audio captured by the microphone), in order to eliminate the audio played by the terminal contained in the audio captured by the microphone, the fourth audio can be used as a reference signal to perform echo cancellation on the fifth audio, thereby obtaining a second audio that does not contain the audio played by the terminal.

[0114] Step 906: Mix the first audio and the second audio to obtain a third audio, which is then shared with other terminals.

[0115] The implementation method of step 906 can be referred to the above embodiment, and will not be repeated here.

[0116] like Figure 10 The diagram illustrates the principle of an audio sharing method according to another exemplary embodiment of this application. The operating system 1020 is only configured with a global loopback acquisition interface 1021, and the audio sharing application 1010 can only use the global loopback acquisition interface 1021 to implement audio sharing. Upon receiving an audio packet from a remote audio sharing application, the sixth audio 1011 is obtained after passing through the packet decoding module. The sixth audio 1011 is sent to the kernel via the operating system API and mixed with the audio played by all other applications 1030 to obtain the fourth audio 1013, awaiting speaker playback; it is also buffered as a reference signal for the echo cancellation 2 module. The audio sharing application 1010 calls the global loopback acquisition interface 1021 to obtain the fourth audio 1013. The fourth audio 1013 contains the sixth audio 1011 and the audio played by other applications 1030. The fourth audio 1013 serves as both the desired signal for the echo cancellation 2 module and a reference signal for the echo cancellation 1 module. The speaker plays a fourth audio 1013, which, after passing through an echo path, is mixed with near-end speech and captured by a microphone to obtain a fifth audio 1014. The fifth audio 1014 serves as the desired signal for the echo cancellation module 1. The echo cancellation module 1 removes components in the fifth audio 1014 related to the fourth audio 1013, resulting in a second audio 1015. Since the fourth audio 1013 contains the sixth audio 1011 and audio played by other applications 1030, the second audio 1015 only contains an estimate of the near-end speech. The echo cancellation module 2 removes components in the fourth audio 1013 related to the sixth audio 1011, resulting in a first audio 1012. Since the sixth audio 1011 is audio played by the audio sharing application 1010, the first audio 1012 only contains an estimate of the audio played by other applications 1030. The first audio 1012 and the second audio 1015 are then fed into a mixing module to obtain a third audio 1016. The third audio 1016 includes estimations of near-end speech and audio played by other applications, thus meeting the requirements of the audio sharing function. Finally, the third audio 1016 is sent to the encoding and packet sending module for encoding, and then sent to the remote audio sharing application.

[0117] In this embodiment, when the terminal is only configured with a global loopback acquisition interface, the echo cancellation reference signal can be obtained based on the audio content acquired by the global loopback acquisition interface. This eliminates the portion of the terminal-played audio included in the microphone-acquired audio, thus obtaining the second audio. Additionally, audio played by other applications (the first audio) can be obtained from the signal acquired by the global loopback acquisition interface. During the acquisition of the first audio, it does not need to be transmitted through media such as speakers, microphones, or air, thus avoiding the impact of media propagation on the sound quality of the first audio and improving the quality of audio sharing.

[0118] In another possible application scenario, if the terminal is only configured with a process loopback acquisition interface, since the process loopback acquisition interface can only acquire audio played by other applications in the terminal besides the audio sharing application, it cannot directly obtain all the audio played by the terminal. Therefore, in order to obtain the second audio after echo cancellation, it is necessary to call the echo cancellation interface inside the operating system.

[0119] Please refer to Figure 11 This document illustrates a flowchart of an audio sharing method provided in another exemplary embodiment of this application. The embodiments of this application use the method applied to a terminal as an example for description. The method includes:

[0120] Step 1101: With the audio sharing function enabled, determine the interface type of the audio acquisition interface configured on the terminal.

[0121] In this embodiment, when the audio sharing function is enabled, it is determined that the terminal is only configured with a process loopback acquisition interface and not a global loopback acquisition interface. The corresponding audio sharing application can switch to the audio sharing mode (first audio acquisition mode and second audio acquisition mode) corresponding to the process loopback acquisition interface.

[0122] The implementation method of step 1101 can be referred to the above embodiment, and will not be repeated here.

[0123] Step 1102: If the interface type includes a process loopback acquisition interface, acquire the first audio through the process loopback acquisition interface.

[0124] Since the process loopback acquisition interface is used to acquire audio from applications other than a specified process and its child processes in the terminal, and the specified process and its child processes are the processes related to the audio sharing application in the audio sharing scenario, and the first audio is the audio played by other applications besides the audio sharing application, based on the function of the process loopback acquisition interface, if the interface type in the terminal includes the process loopback acquisition interface, the terminal can directly acquire the first audio through the process loopback acquisition interface, and the corresponding audio sharing application can obtain the first audio from the process loopback acquisition interface.

[0125] Step 1103: Obtain the second audio output from the echo cancellation interface. The echo cancellation interface is used to cancel the echo of the fifth audio acquired by the microphone to obtain the second audio.

[0126] Without a global loopback acquisition interface configured on the terminal, the audio sharing application cannot obtain the fourth audio signal captured by the global loopback acquisition interface to perform echo cancellation on the microphone-captured audio to obtain the second audio signal; that is, echo cancellation cannot be implemented within the audio sharing application itself. Therefore, in order to obtain the second audio signal that does not include the audio played by the terminal, in one possible implementation, the audio sharing application can directly call the echo cancellation interface within the operating system to obtain the second audio signal. This echo cancellation interface can implement echo cancellation within the operating system to obtain the second audio signal after echo cancellation of the fifth audio signal captured by the microphone.

[0127] Step 1104: Mix the first audio and the second audio to obtain a third audio, which is then shared with other terminals.

[0128] The implementation method of step 1104 can be referred to the above embodiment, and will not be repeated here.

[0129] like Figure 12The diagram illustrates the principle of an audio sharing method according to another exemplary embodiment of this application. The operating system 1220 is configured with a process loopback acquisition interface 1221 and an echo cancellation interface 1222. The audio sharing application 1210 can only use the process loopback acquisition interface 1221 to achieve audio sharing. When the audio sharing application 1210 receives an audio packet from a remote audio sharing application, it obtains a sixth audio 1211 after passing through the packet decoding module. The sixth audio 1211 is sent to the kernel via the operating system API and mixed with the audio played by all other applications 1230 to obtain a fourth audio 1213. The fourth audio 1213 is then played by the speaker, passes through the echo path, mixes with near-end speech, and is captured by the microphone to obtain a fifth audio 1214. The fifth audio 1214 will serve as the desired signal for the operating system echo cancellation interface 1222. The operating system 1220 internally implements global loopback acquisition, which can acquire the fourth audio 1213 as a reference signal. The operating system echo cancellation interface 1222 removes components related to the fourth audio 1213 from the fifth audio 1214, resulting in the second audio 1215, which is returned to the audio sharing application 1210. The second audio 1215 retains only the estimation of near-end speech. The audio sharing application 1210 calls the process loopback acquisition interface 1221 to obtain the first audio 1212 played by other applications 1230. The first audio 1212 and the second audio 1215 are sent to the mixing module to obtain the third audio 1216. The third audio 1216 includes the estimation of near-end speech and the audio played by other applications 1230, thus meeting the requirements of the audio sharing function. Finally, the third audio 1216 is sent to the encoding and packet sending module for encoding and then sent to the remote audio sharing application.

[0130] In this embodiment, when the terminal is equipped with a process loopback acquisition interface, the first audio can be directly acquired using this interface. During the acquisition of the first audio, it does not need to be transmitted through media such as speakers, microphones, or air, nor does it undergo echo cancellation processing, thus preserving the sound quality of the first audio to the maximum extent. During the mixing process, only the second audio may have residual echo, resulting in minimal echo leakage. Furthermore, since the operating system's echo cancellation interface algorithm is often implemented in hardware, the computational load for echo cancellation is very small.

[0131] The following are device embodiments of this application. For details not described in detail in the device embodiments, please refer to the above method embodiments.

[0132] Figure 13 This is a structural block diagram of an audio sharing device provided in an exemplary embodiment of this application. The device includes:

[0133] The first acquisition module 1301 is used to acquire the first audio played by the terminal when the audio sharing function is enabled. The first audio is the audio played by an application other than the audio sharing application in the terminal. The audio sharing application is used to share the audio played by the terminal with other terminals. The acquisition process of the first audio does not go through the microphone acquisition link.

[0134] The second acquisition module 1302 is used to acquire the second audio after echo cancellation, wherein the echo cancellation is used to eliminate the audio played by the terminal contained in the audio collected by the microphone;

[0135] The mixing module 1303 is used to mix the first audio and the second audio to obtain a third audio, which is used to share with the other terminals.

[0136] Optionally, the first acquisition module 1301 is further configured to:

[0137] When the audio sharing function is enabled, the interface type of the audio acquisition interface configured on the terminal is determined. The audio acquisition interface is used to acquire the audio played by the terminal. The acquisition methods of the first audio and the second audio are different under different interface types.

[0138] Based on the first audio acquisition method corresponding to the interface type, the first audio played by the terminal is acquired;

[0139] The second acquisition module 1302 is further configured to:

[0140] Based on the second audio acquisition method corresponding to the interface type, the second audio obtained after echo cancellation is acquired.

[0141] Optionally, the terminal is configured with a global loopback acquisition interface and a process loopback acquisition interface;

[0142] The first acquisition module 1301 is further configured to:

[0143] When the interface type includes the global loopback acquisition interface and the process loopback interface, the first audio is acquired through the process loopback acquisition interface;

[0144] The second acquisition module 1302 is further configured to:

[0145] When the interface type includes the global loopback acquisition interface and the process loopback interface, the fourth audio is acquired through the global loopback acquisition interface, which is used to acquire all audio played by the terminal;

[0146] A fifth audio signal is captured via a microphone, wherein the fifth audio signal includes at least all the audio played by the terminal;

[0147] Echo cancellation is performed on the fifth audio based on the fourth audio to obtain the second audio.

[0148] Optionally, the terminal is configured with a global loopback acquisition interface but not with a process loopback acquisition interface;

[0149] The first acquisition module 1301 is further configured to:

[0150] When the interface type includes the global loopback acquisition interface, the fourth audio is acquired through the global loopback acquisition interface, which is used to acquire all audio played by the terminal;

[0151] Obtain the first audio played by the terminal from the fourth audio;

[0152] The second acquisition module 1302 is further configured to:

[0153] When the interface type includes the global loopback acquisition interface, a fifth audio signal is acquired via a microphone, and the fifth audio signal includes at least all the audio played by the terminal.

[0154] Echo cancellation is performed on the fifth audio based on the fourth audio to obtain the second audio.

[0155] Optionally, the first acquisition module 1301 is further configured to:

[0156] The sixth audio file played by the audio sharing application is obtained by decoding the audio stream received by the audio sharing application.

[0157] Echo cancellation is performed on the fourth audio based on the sixth audio to obtain the first audio played by the terminal.

[0158] Optionally, the terminal is configured with a process loopback acquisition interface, but not with a global loopback acquisition interface;

[0159] The first acquisition module 1301 is further configured to:

[0160] When the interface type includes the process loopback acquisition interface, the first audio is acquired through the process loopback acquisition interface;

[0161] The second acquisition module 1302 is further configured to:

[0162] When the interface type includes the process loopback acquisition interface, the second audio output by the echo cancellation interface is acquired. The echo cancellation interface is used to perform echo cancellation on the fifth audio acquired by the microphone to obtain the second audio.

[0163] Optionally, the first acquisition module 1301 is further configured to:

[0164] With the audio sharing function enabled, obtain the operating system type of the terminal's configured operating system;

[0165] Based on the operating system type, determine the interface type of the audio acquisition interface configured on the terminal.

[0166] Optionally, the first acquisition module 1301 is further configured to:

[0167] When the operating system is a first-class operating system, the interface type of the audio acquisition interface configured on the terminal is determined based on the operating system version;

[0168] When the operating system is a type II operating system, it is determined that the terminal is configured with a process loopback acquisition interface, but not with a global loopback acquisition interface.

[0169] Optionally, the first acquisition module 1301 is further configured to:

[0170] If the operating system is the first type of operating system and the version level of the operating system is above the target level, it is determined that the terminal is configured with the process loopback acquisition interface and the global loopback acquisition interface.

[0171] If the operating system is the first type of operating system and the version level of the operating system is below the target level, it is determined that the terminal is configured with the global loopback acquisition interface but not with the process loopback acquisition interface.

[0172] Optionally, the device further includes:

[0173] The sending module is used to send a startup command to the operating system through the audio sharing application when the operating system is the first type of operating system. The startup command is used to start the process loopback acquisition interface.

[0174] The determination module is used to determine, after receiving feedback information indicating successful startup, that the terminal is configured with the global loopback acquisition interface and the process loopback acquisition interface;

[0175] The determining module is further configured to, upon receiving feedback information indicating a startup failure, determine that the terminal is configured with the global loopback acquisition interface but not with the process loopback acquisition interface.

[0176] In summary, by separately acquiring the audio played by applications other than the audio sharing application on the terminal (first audio) and the microphone signal obtained after removing the audio played on the terminal (second audio), and then mixing the first and second audio, a third audio for audio sharing is obtained. This avoids acquiring the first audio through the microphone acquisition link, eliminating the need for speakers, microphones, air, and other media, as well as avoiding audio processing such as echo cancellation and noise reduction. This significantly improves the distortion problem of the first audio, thereby enhancing its audio quality and ultimately improving the overall audio quality of the audio sharing.

[0177] Figure 14 A structural block diagram of a terminal 1400 provided in an exemplary embodiment of this application is shown. The terminal 1400 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 1400 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0178] Typically, terminal 1400 includes a processor 1401 and a memory 1402.

[0179] Processor 1401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0180] The memory 1402 may include one or more computer-readable storage media, which may be non-transitory. The memory 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1402 are used to store at least one instruction, which is executed by the processor 1401 to implement the audio sharing method provided in the method embodiments of this application.

[0181] In some embodiments, the terminal 1400 may also optionally include a peripheral device interface 1403 and at least one peripheral device. The processor 1401, memory 1402, and peripheral device interface 1403 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: radio frequency circuitry, a display screen, a camera assembly, audio circuitry 1404, and a power supply.

[0182] The audio circuit 1404 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1401 for processing, or input to the radio frequency circuit for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1400. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1401 or the radio frequency circuit into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1404 may also include a headphone jack.

[0183] Those skilled in the art will understand that Figure 14 The structure shown does not constitute a limitation on terminal 1400 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0184] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the audio sharing method provided in any of the above exemplary embodiments.

[0185] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A terminal's processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal to perform the audio sharing method provided in the optional implementation described above.

[0186] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0187] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An audio sharing method, characterized in that, The method is executed by a terminal, and the method includes: With the audio sharing function enabled, the interface type of the audio acquisition interface configured inside the terminal is determined. The audio acquisition interface is used to acquire the audio played by the terminal. The acquisition methods of the first audio and the second audio are different under different interface types. The interface types include: only configured with a process loopback acquisition interface, only configured with a global loopback acquisition interface, configured with both a process loopback acquisition interface and a global loopback acquisition interface. The process loopback acquisition interface is used to acquire the audio played by other applications in the terminal besides the audio sharing application. The global loopback acquisition interface is used to acquire all the audio played in the terminal. Based on the first audio acquisition method corresponding to the interface type, the first audio played by the terminal is acquired. The first audio is the audio played by other applications in the terminal besides the audio sharing application. The audio sharing application is used to share the audio played by the terminal with other terminals. The acquisition process of the first audio does not go through the microphone acquisition link. Based on the second audio acquisition method corresponding to the interface type, the second audio obtained after echo cancellation is acquired. The echo cancellation is used to eliminate the audio played by the terminal contained in the audio collected by the microphone. When the terminal is configured with the global loopback acquisition interface, the second audio is obtained by using all the audio played by the terminal collected by the global loopback acquisition interface as the reference signal for echo cancellation. The first audio and the second audio are mixed to obtain a third audio, which is used to share with the other terminals.

2. The method according to claim 1, characterized in that, The terminal is equipped with a global loopback acquisition interface and a process loopback acquisition interface; The method of obtaining the first audio played by the terminal based on the first audio acquisition method corresponding to the interface type includes: When the interface type includes the global loopback acquisition interface and the process loopback interface, the first audio is acquired through the process loopback acquisition interface; The second audio acquisition method based on the interface type, which acquires the second audio after echo cancellation, includes: When the interface type includes the global loopback acquisition interface and the process loopback interface, the fourth audio is acquired through the global loopback acquisition interface, which is used to acquire all audio played by the terminal; A fifth audio signal is captured via a microphone, wherein the fifth audio signal includes at least all the audio played by the terminal; Echo cancellation is performed on the fifth audio based on the fourth audio to obtain the second audio.

3. The method according to claim 1, characterized in that, The terminal is configured with a global loopback acquisition interface, but not with a process loopback acquisition interface. The method of obtaining the first audio played by the terminal based on the first audio acquisition method corresponding to the interface type includes: When the interface type includes the global loopback acquisition interface, the fourth audio is acquired through the global loopback acquisition interface, which is used to acquire all audio played by the terminal; Obtain the first audio played by the terminal from the fourth audio; The second audio acquisition method based on the interface type, which acquires the second audio after echo cancellation, includes: When the interface type includes the global loopback acquisition interface, a fifth audio signal is acquired via a microphone, and the fifth audio signal includes at least all the audio played by the terminal. Echo cancellation is performed on the fifth audio based on the fourth audio to obtain the second audio.

4. The method according to claim 3, characterized in that, The step of obtaining the first audio played by the terminal from the fourth audio includes: The sixth audio file played by the audio sharing application is obtained by decoding the audio stream received by the audio sharing application. Echo cancellation is performed on the fourth audio based on the sixth audio to obtain the first audio played by the terminal.

5. The method according to claim 1, characterized in that, The terminal is configured with a process loopback acquisition interface, but not with a global loopback acquisition interface. The method of obtaining the first audio played by the terminal based on the first audio acquisition method corresponding to the interface type includes: When the interface type includes the process loopback acquisition interface, the first audio is acquired through the process loopback acquisition interface; The second audio acquisition method based on the interface type, which acquires the second audio after echo cancellation, includes: When the interface type includes the process loopback acquisition interface, the second audio output by the echo cancellation interface is acquired. The echo cancellation interface is used to perform echo cancellation on the fifth audio acquired by the microphone to obtain the second audio.

6. The method according to any one of claims 1 to 5, characterized in that, When the audio sharing function is enabled, determining the interface type of the audio acquisition interface configured inside the terminal includes: With the audio sharing function enabled, obtain the operating system type of the terminal's configured operating system; Based on the operating system type, determine the interface type of the audio acquisition interface configured on the terminal.

7. The method according to claim 6, characterized in that, The step of determining the interface type of the audio acquisition interface configured on the terminal based on the operating system type includes: When the operating system is a first-class operating system, the interface type of the audio acquisition interface configured on the terminal is determined based on the operating system version; When the operating system is a type II operating system, it is determined that the terminal is configured with a process loopback acquisition interface, but not with a global loopback acquisition interface.

8. The method according to claim 7, characterized in that, When the operating system is a first-type operating system, determining the interface type of the audio acquisition interface configured on the terminal based on the operating system version includes: If the operating system is the first type of operating system and the version level of the operating system is above the target level, it is determined that the terminal is configured with the process loopback acquisition interface and the global loopback acquisition interface. If the operating system is the first type of operating system and the version level of the operating system is below the target level, it is determined that the terminal is configured with the global loopback acquisition interface but not with the process loopback acquisition interface.

9. The method according to claim 7, characterized in that, The method further includes: When the operating system is the first type of operating system, a startup command is sent to the operating system through the audio sharing application. The startup command is used to start the process loopback acquisition interface. After receiving feedback indicating successful startup, it is determined that the terminal is configured with the global loopback acquisition interface and the process loopback acquisition interface. After receiving feedback indicating startup failure, it is determined that the terminal is configured with the global loopback acquisition interface but not with the process loopback acquisition interface.

10. An audio sharing device, characterized in that, The device includes: The first acquisition module is used to determine the interface type of the audio acquisition interface configured inside the terminal when the audio sharing function is enabled. The audio acquisition interface is used to acquire the audio played by the terminal. The acquisition methods of the first audio and the second audio are different under different interface types. The interface types include: only configured with a process loopback acquisition interface, only configured with a global loopback acquisition interface, and configured with both a process loopback acquisition interface and a global loopback acquisition interface. The process loopback acquisition interface is used to acquire the audio played by other applications in the terminal besides the audio sharing application. The global loopback acquisition interface is used to acquire all the audio played in the terminal. Based on the first audio acquisition method corresponding to the interface type, the first audio played by the terminal is acquired. The first audio is the audio played by other applications in the terminal besides the audio sharing application. The audio sharing application is used to share the audio played by the terminal with other terminals. The acquisition process of the first audio does not go through the microphone acquisition link. The second acquisition module is used to acquire the second audio after echo cancellation based on the second audio acquisition method corresponding to the interface type. The echo cancellation is used to eliminate the audio played by the terminal in the audio collected by the microphone. When the terminal is configured with the global loopback acquisition interface, the second audio is obtained by using all the audio played by the terminal collected by the global loopback acquisition interface as the reference signal for echo cancellation. The mixing module is used to mix the first audio and the second audio to obtain a third audio, which is used to share with the other terminals.

11. A terminal, characterized in that, The terminal includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the audio sharing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one program, which is loaded and executed by a processor to implement the audio sharing method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product stores a computer program that is loaded and executed by a processor to implement the audio sharing method as described in any one of claims 1 to 9.

Images