Conference device, audio sharing method, electronic device, and communication system

Abstract

The application discloses a conference device, an audio sharing method, an electronic device and a communication system. The conference device interacts with remote devices corresponding to each audio application through at least two audio applications. The electronic device comprises a communication circuit and an execution circuit. The communication circuit is used for communication connection with the remote devices. The execution circuit is coupled to the communication circuit. The execution circuit acquires downlink audio data transmitted by the remote devices through the corresponding audio applications in response to the start of the audio application through the communication circuit. The execution circuit sends the downlink audio data transmitted by other audio applications to the corresponding remote devices through each audio application. In this way, the application can realize audio sharing of remote devices of different audio applications.

Description

Technical Field

[0001] This application relates to the field of audio data processing technology, and in particular to conferencing equipment, audio sharing methods, electronic devices, and communication systems. Background Technology

[0002] With the development of internet and communication technologies, society has entered an era of intelligent interconnection, and interacting, entertaining, and working online has become increasingly common. Among these, the use of various audio applications or apps on electronic devices for interaction, entertainment, and work is particularly prevalent. For example, users can use audio applications for online multi-party communication, such as meetings or games.

[0003] Currently, multi-party remote interactions often rely on the same audio application. This means that each electronic device typically only supports audio interaction between remote devices that share the same audio application. If multiple remote devices use different audio applications, cross-application audio data exchange is impossible. For example, if a conference is held with multiple remote devices that have different audio applications installed (i.e., different remote devices have different audio applications installed), it can only be achieved by simultaneously holding the conference on multiple electronic devices at the local end, each with its own audio application (electronic devices and remote devices are linked one-to-one through the audio application), thus impacting the user experience. Summary of the Invention

[0004] Embodiments of this application provide conferencing equipment, audio sharing methods, electronic devices, and communication systems that enable audio interaction between remote devices corresponding to each audio application when multiple audio applications are running simultaneously in the conferencing equipment.

[0005] In a first aspect, embodiments of this application provide a conferencing device that interacts with a remote device of each audio application through at least two audio applications. The conferencing device includes: a communication circuit for establishing a communication connection with the remote device; and an execution circuit coupled to the communication circuit. In response to the startup of an audio application, the execution circuit acquires downlink audio data transmitted from the remote device via the corresponding audio application through the communication circuit. The execution circuit also transmits downlink audio data transmitted from other audio applications to the corresponding remote device through each audio application.

[0006] Secondly, embodiments of this application provide an audio sharing method for a conferencing device. The conferencing device interacts with a remote device corresponding to each audio application through at least two audio applications. The audio sharing method includes: acquiring downlink audio data transmitted from the remote device via the corresponding audio application; and sending downlink audio data transmitted from other audio applications to the corresponding remote device via each audio application.

[0007] Thirdly, embodiments of this application provide an electronic device, which includes: an execution circuit, a memory, a communication circuit, a microphone, and a speaker. The communication circuit, the memory, the microphone, and the speaker are coupled to the execution circuit. The memory is used to store a computer program, and the execution circuit is used to execute the computer program to implement the audio sharing method of the electronic device provided in this application as described above.

[0008] Fourthly, this application provides a communication system, which includes an electronic device as described above and at least two remote devices, wherein the electronic device and the remote devices are communicatively connected.

[0009] The beneficial effects of this application are as follows: Unlike the prior art, the conferencing equipment interacts with the remote device of each audio application through at least two audio applications. It can obtain the downlink audio data transmitted by each remote device through the corresponding audio application, and each audio application sends the downlink audio data transmitted by other audio applications to the corresponding remote device. In this way, the remote devices of different audio applications can receive each other's audio data, thereby realizing cross-application audio interaction between different audio applications. For a certain audio application, the remote devices corresponding to other audio applications can receive the audio information of that audio application, and can play it based on the audio data, so that users of the remote devices corresponding to different audio applications can hear the content corresponding to the audio data uploaded by the remote devices of other audio applications. By installing different audio applications on electronic devices simultaneously, and sending the downlink audio data transmitted from the remote device corresponding to each audio application to other remote devices corresponding to audio applications, audio data interaction between the remote devices corresponding to each audio application can be achieved without configuring a local conferencing device for each remote device of each audio application. This saves costs, improves the efficiency of audio sharing and interaction, and enhances the user experience. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of a system block diagram of an embodiment of the communication system of this application;

[0011] Figure 2 This is a schematic diagram of the structure of an embodiment of the conference equipment in this application;

[0012] Figure 3 This is a flowchart illustrating an embodiment of the audio sharing method for electronic devices according to this application;

[0013] Figure 4 This is a timing diagram of an embodiment of the audio sharing method for electronic devices according to this application;

[0014] Figure 5This is a schematic diagram of the preset sharing service process in an embodiment of the audio sharing method for electronic devices of this application;

[0015] Figure 6 This is a schematic diagram of the circuit structure of an embodiment of the electronic device of this application; Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0017] With the development of internet and communication technologies, society has entered an era of intelligent interconnection, and interacting, entertaining, and working online has become increasingly common. Among these, the use of various audio applications or mobile applications on electronic devices for interaction, entertainment, and work is particularly prevalent. For example, users can conduct online meetings through conferencing apps or play online games through gaming apps.

[0018] Through long-term research, the inventors discovered that during the operation of audio applications or other applications on electronic devices, audio functions can be used to interact with remote devices. If audio interaction with multiple remote devices is required, and these devices use different audio applications, it becomes difficult for them to understand each other's speech during the interaction. This is because the near-end electronic device, due to echo cancellation at the system's HAL layer, cannot pick up the echo generated by the audio sent by one remote device and forwards it to other remote devices using different audio applications for audio sharing. This results in remote devices using different audio applications not being able to hear each other's speech, thus affecting the user experience. Taking a meeting scenario as an example, in a multi-party meeting, if A and B use audio app a, C uses audio app b, and D uses audio app c, when A is the meeting initiator, the electronic device corresponding to A can be called the near-end device, and the other corresponding electronic devices can be called remote devices. During a meeting, the audio data acquired by each audio app from the near-end device is raw microphone data that has undergone echo cancellation before being sent to the remote device. Therefore, if someone from company B speaks, device A's electronic device will cancel the echo of B's ​​speech, causing the microphone to be unable to pick it up. Consequently, other remote devices, namely C and D, will not be able to hear B's voice through their respective meeting apps. To solve the above technical problem, this application proposes the following embodiments.

[0019] like Figure 1 As shown in the embodiments of this application, the communication system 200 described may include an electronic device 100 and at least two remote devices 170. The electronic device 100 is a near-end device, and the near-end device is communicatively connected to each of the remote devices 170. This allows audio sharing among the remote devices 170 to be achieved on the near-end device. Multiple applications can be run on the near-end device, enabling simultaneous conferencing with remote devices 170 that have different audio applications installed.

[0020] like Figure 2 As shown, electronic device 100 can be used as conferencing equipment. Electronic device 100 can be, for example, a mobile phone, PC, laptop, tablet, server, smart wearable device, etc. Remote device 170 can be, for example, a mobile phone, PC, laptop, tablet, server, smart wearable device, etc. At least two audio applications can be installed on the software operating system of the conferencing equipment. The audio applications must have at least audio transmission capabilities, but are not limited to audio transmission; they can also have video, text, and image transmission capabilities, thus enabling further video interaction (e.g., video conferencing, text conferencing), etc. Audio applications can be conferencing apps, office apps, social apps, game apps, or instant messaging apps, etc.

[0021] The audio sharing method embodiment of the conference equipment in this application can be applied to the above-mentioned electronic device 100 with at least two audio applications installed, or to the case where an external device with at least two audio applications installed is connected to the conference equipment, or to the case where at least two audio applications can connect to the near-end conference equipment through the cloud, and the conference equipment can interact with the corresponding remote device through the audio applications.

[0022] Specifically, the audio sharing method embodiment of the conferencing equipment in this application can be described with electronic device 100 (near-end device) as the execution subject. For example, electronic device 100 can support Android, Windows, and iOS systems that can open multiple apps simultaneously. Specifically, this embodiment takes Android system as an example, based on the multi-account function of Android 12, it supports opening multiple apps simultaneously, and AudioFlinger supports the simultaneous playback and recording functions of multiple apps, thereby realizing the scenario of audio interaction when multiple audio applications are open.

[0023] like Figure 3 As shown, this embodiment may include the following steps: S100: Obtain downlink audio data transmitted from the remote device via the corresponding audio application. S200: Send downlink audio data transmitted from other audio applications to the corresponding remote device via each audio application.

[0024] When multiple audio applications are running simultaneously, the system can acquire downlink audio data transmitted from the remote device via the corresponding audio application. Each audio application then sends the downlink audio data transmitted from other audio applications to the corresponding remote device. This allows audio applications on other remote devices, relative to a particular audio application, to receive the audio information from that application and play it based on the audio data. This enables users on remote devices to hear the corresponding content, thereby achieving audio sharing when multiple audio applications are running simultaneously, which helps improve the user experience.

[0025] like Figure 3 and Figure 4 As shown, the following is a detailed description of this embodiment.

[0026] S100: Obtain downlink audio data transmitted from the remote device via the corresponding audio application.

[0027] Audio applications can include applications installed in the system used for audio sharing. Specifically, audio applications can call the speakers and microphones of the conferencing equipment to record and play audio. The remote device uses its audio application to pick up audio data via the microphone and sends the audio data to the near-end device (i.e., the electronic device 100 mentioned above, which can be called the conferencing equipment in a conferencing scenario). This audio data can be referred to as downlink audio data by the near-end device.

[0028] Downlink audio data may include audio parameters and downlink audio signal data. Specifically, audio parameters may include audio sampling rate and data format, and downlink audio signal data may also include pulse code modulation data.

[0029] In the process of obtaining downlink audio data transmitted from a remote device via the corresponding audio application, this can be achieved by calling playback interface class functions. Specifically, when the audio application calls the playback interface class functions, it calls the first function of the local interface layer. Since the first function can accept the audio sampling rate and data format, the audio sampling rate and data format can be obtained by calling the playback interface class functions.

[0030] For example, when the audio application 'a' on the near-end device calls the new Audio Track function to play downlink audio data passed from the remote device via the audio application 'a', it will call the static jint android_media_AudioTrack_setup function in the local interface layer. The static jint android_media_AudioTrack_setup function will pass in the audio sampling rate and data format, thereby obtaining the audio sampling rate and data format corresponding to the audio passed from the audio application 'a'.

[0031] When an audio application calls a playback interface function, it invokes a second function in the native interface layer. Since this second function can accept pulse-code modulation (PCM) data, it can obtain the PCM data—which is the audio signal—by calling the playback interface function. For example, when audio application 'a' calls the `AudioTrack.write` function, it calls the `static jint android_media_AudioTrack_write_native_bytes` function in the native interface layer. This `static jint android_media_AudioTrack_write_native_bytes` function receives PCM data as input, thus allowing the application to obtain the PCM data passed in by audio application 'a'.

[0032] In one implementation, the following steps in S100 can be used to describe how to obtain the downlink audio data transmitted from the remote device via the corresponding audio application:

[0033] S110: Write the downlink audio data passed in by each audio application to the buffer area corresponding to each audio application and the buffer area corresponding to other audio applications.

[0034] The buffer can include a buffer for storing audio data; specifically, each audio application has its own corresponding buffer.

[0035] In the process of acquiring downlink audio data transmitted from remote devices via corresponding audio applications, the downlink audio data transmitted by each audio application can be written to the cache area corresponding to each audio application and the cache area corresponding to other audio applications, thereby facilitating the storage and retrieval of downlink audio data.

[0036] In one implementation, the following steps can be referenced to determine how to write the downlink audio data passed from each audio application to the buffer of each audio application and the buffers of other audio applications:

[0037] S111: Write the downlink audio data passed in by each audio application into the first buffer corresponding to each audio application and the second buffer corresponding to other audio applications.

[0038] The buffer can include a first buffer and a second buffer. Specifically, each audio application can be configured with its own corresponding first buffer. The first buffer can be used to store its corresponding audio data. For example, the first buffer can be pcm_buf. Each audio application can also be configured with its own corresponding second buffer, and the second buffer corresponding to each audio application can be used by other audio applications to write audio data from their corresponding first buffer. By writing the audio data stored in the first buffer of each audio application to the second buffer of other audio applications, audio data sharing between different audio applications can be achieved. For example, the second buffer can be mix_buf.

[0039] In one implementation, the steps in S111 for writing the downlink audio data passed by each audio application to the first buffer corresponding to each audio application and the second buffer corresponding to other audio applications can be referred to:

[0040] S1111: Request a corresponding first cache for each audio application and record the memory index number corresponding to the first cache.

[0041] During the process of requesting a corresponding first buffer for each audio application, a pre-defined sharing service process can be requested for each audio application. Specifically, the pre-defined sharing service process can be a newly configured feature in the operating system for enabling audio sharing between different audio applications. Specifically, the pre-defined sharing service process can be used to manage the audio data transmitted to the local interface when each audio application plays audio data. One pre-defined sharing service process can be configured in a system.

[0042] During the process of writing the audio data corresponding to each audio application into its corresponding first buffer, a request for the corresponding first buffer for each audio application can be made from the preset sharing service process. Specifically, this request can be made from the preset sharing service process for each audio application through a third function. For example, the `creat_playback_buffer` function can be used to request the corresponding first buffer for each audio application from the preset sharing service process.

[0043] After requesting a corresponding first buffer for each audio application from the preset sharing service process, the memory index number corresponding to each first buffer can be recorded. Specifically, the memory index number can be represented by a process identifier, making it easier to determine the corresponding first buffer based on the memory index number.

[0044] S1112: Write the downlink audio data passed in by each audio application into the first buffer corresponding to each audio.

[0045] After obtaining the audio sampling rate, data format, and pulse code modulation data transmitted from different audio applications, the audio sampling rate, data format, and pulse code modulation data corresponding to each audio application can be written into its corresponding first buffer, thereby enabling audio sharing of the audio sampling rate, data format, and pulse code modulation data corresponding to each audio application.

[0046] After requesting a corresponding first buffer for each audio application from the preset sharing service process, the audio data corresponding to each audio application can be written into the corresponding first buffer, thereby storing the audio data corresponding to each audio application in the first buffer corresponding to that audio application. For example, if the first buffer corresponding to audio application a is a1 and the first buffer corresponding to audio application b is b1, the audio sampling rate, data format, and pulse code modulation data corresponding to audio application a can be written into a1, and the audio sampling rate, data format, and pulse code modulation data corresponding to audio application b can be written into b1.

[0047] S1113: Request a corresponding second cache for each audio application and record the memory index number corresponding to the second cache.

[0048] During the process of requesting a corresponding second buffer for each audio application, a request can be made to the preset sharing service process for each audio application. Specifically, the preset sharing service process is a new configuration added to the operating system to enable audio sharing between different audio applications.

[0049] When an audio application calls the recording interface class function, it calls the fourth function of the local interface layer, and the fourth function passes in the audio sampling rate and data format. Specifically, the audio sampling rate and data format passed in by the fourth function can correspond to the audio sampling rate and data format of the audio passed in through the recording interface. In one implementation, the audio passed in through the recording interface may include recording data acquired through a microphone, which can be an external recording device that communicates with the electronic device 100 or a microphone built into the electronic device 100.

[0050] Since the audio data from the playback interface and the recording interface needs to be mixed during the audio sharing process, the fifth function can be executed during the audio application's call to the recording interface class function to request a corresponding second buffer for each audio application from the preset sharing service process. This allows the incoming audio sampling rate and data format to be stored in the second buffer, and then the audio sampling rate and data format passed through the recording interface can be mixed with the audio sampling rate, data format, and pulse code modulation data read from the first buffer.

[0051] For example, when audio application 'a' calls the `new AudioRecord` function to record audio, it calls the `static jint android_media_AudioRecord_setup` function in the local interface layer. This function receives the audio sampling rate and data format as input, thus obtaining the audio sampling rate and data format corresponding to the audio received via the recording interface. At this point, the `register_record` function can be executed to request a second buffer 'a2' from the preset sharing service process for audio application 'a', and the obtained audio sampling rate and data format are registered in the second buffer 'a2'.

[0052] After requesting a corresponding second buffer for each audio application from the preset sharing service process, the memory index number corresponding to each second buffer can be recorded. Specifically, the memory index number can be represented by a process identifier, making it easier to determine the corresponding second buffer based on the memory index number.

[0053] S1114: Write the downlink audio data cached in the first buffer corresponding to each audio application to the second buffer corresponding to other audio applications.

[0054] For example, if the first buffer for audio application 'a' is a1, the first buffer for audio application 'b' is b1, and the first buffer for audio application 'c' is c1, and the second buffer for audio application 'a' is a2, the second buffer for audio application 'b' is b2, and the second buffer for audio application 'c' is c2, then the audio sampling rate, data format, and PCM audio data stored in a1 can be written to b2 and c2; the audio sampling rate, data format, and Pulse Code Modulation (PCM) data stored in b1 can be written to a2 and c2; and the audio sampling rate, data format, and PCM data stored in c1 can be written to a2 and b2.

[0055] S1115: Mix the downlink audio data passed in by other audio applications to obtain the uplink audio data of each audio application, and write the uplink audio data into the corresponding second buffer.

[0056] After writing the downlink audio data cached in the first buffer corresponding to each audio application to the second buffer corresponding to other audio applications, the downlink audio data passed in by other audio applications can be mixed to obtain the uplink audio data of each audio application, and the uplink audio data can be written to the corresponding second buffer.

[0057] In one implementation, S1115 may include the following steps:

[0058] S11151: Read the downlink audio data cached in the first buffer corresponding to each audio application.

[0059] Since the first buffer of each audio application can be used to store the downlink audio data passed in by each audio application, in order to obtain the uplink audio data through mixing, the downlink audio data in the first buffer of each audio application can be read, and the read downlink audio data can be mixed.

[0060] In one implementation, the following steps can be referenced to determine how to read the downlink audio data cached in the first buffer corresponding to each audio application:

[0061] S11152: Monitor the number of audio applications playing audio tracks and the number of corresponding first buffers, and read the downlink audio data of the first buffer corresponding to each monitored audio application.

[0062] During the process of reading downlink audio data cached in the first buffer corresponding to each audio application, a preset mixing thread can monitor the number of audio applications playing audio tracks and the number of their corresponding first buffers. Specifically, the preset mixing thread can be created within the preset sharing service process and is used to read the first buffer corresponding to each audio application and write the read audio data to the second buffer. After reading the audio data stored in the first buffer corresponding to an audio application, the preset mixing thread can write the audio data corresponding to that audio application to the second buffer corresponding to other audio applications, thereby enabling other audio applications to receive the audio data from that audio application and achieving audio data sharing.

[0063] The preset mixing thread reads the audio data of the first buffer corresponding to each audio application. Specifically, the preset mixing thread can monitor the number of audio applications that call the playback interface class function in the system to play, as well as the number of the corresponding first buffers, so that the preset mixing thread can read the audio data stored in the first buffer corresponding to each audio application.

[0064] S11153: Mix the downlink audio data received from other audio applications to obtain uplink audio data.

[0065] Other audio applications may include each audio application installed in the electronic device 100, and other audio applications besides itself.

[0066] Uplink audio data is obtained by reading the downlink audio data cached in the first buffer corresponding to each audio application and mixing the downlink audio data passed in by other audio applications. Specifically, after monitoring the number of audio applications playing audio tracks and the number of their corresponding first buffers through a preset mixing thread, and reading the audio data from the first buffer corresponding to each monitored audio application through the preset mixing thread, the uplink audio data is obtained by mixing the audio data from other audio applications (excluding each audio application itself) through the preset mixing thread.

[0067] In one implementation, the following steps can be used to mix the downlink audio data received from other audio applications to obtain uplink audio data:

[0068] S11154: Determine whether the audio parameters passed in by other audio applications match the audio parameters passed in by each audio application.

[0069] Before mixing the audio data from other audio applications (excluding the application itself), a preset mixing thread can be used to determine whether the audio parameters from the other audio applications match those of the respective audio application. Specifically, this can be done by checking whether the audio sampling rate and data format from the other audio applications match those of the respective audio application.

[0070] For example, if the preset mixing thread can read audio data stored in the first buffer of other audio applications besides audio application a itself, that is, if the preset mixing thread reads the audio data stored in the first buffer b1 of audio application b and the first buffer c1 of audio application c, and if the audio sampling rate in the first buffer b1 is 11025Hz, the audio sampling rate in the first buffer c1 is 22050Hz, while the audio sampling rate stored in the second buffer of audio application a is 24000Hz, then it can be determined that the audio parameters of other audio applications besides themselves do not match the audio parameters of each corresponding audio application. If the data format in the first buffer b1 is 16000 pcm, the data format in the first buffer c1 is 24000 pcm, while the data format in the second buffer of audio application a is 48000 pcm, then it can be determined that the audio parameters of other audio applications besides themselves do not match the audio parameters of each corresponding audio application.

[0071] S11155: If there is a mismatch, the downlink audio signal data passed by the mismatched audio application is resampled so that the audio parameters corresponding to the resampled downlink audio signal data match the audio parameters passed by each audio application.

[0072] If it is determined that the audio parameters of other audio applications besides the one they read do not match the audio parameters of each audio application, the audio signal data of the mismatched audio applications can be resampled so that the audio parameters of the resampled audio signal data match the audio parameters of each audio application, thus facilitating the mixing of audio data with consistent audio parameters.

[0073] For example, if the audio sampling rate in the first buffer b1 is 11025Hz, the audio sampling rate in the first buffer c1 is 22050Hz, and the audio sampling rate stored in the second buffer corresponding to audio application a is 24000Hz, then it can be determined that the audio parameters read from other audio applications (excluding the application itself) do not match the audio parameters corresponding to each audio application. By resampling the audio signal data corresponding to audio applications b and c, so that the audio sampling rates in the first buffer b1 and the first buffer c1 are both 24000Hz, the audio parameters read from other audio applications (excluding the application itself) will match the audio parameters corresponding to each audio application.

[0074] S11156: Mix the resampled downlink audio signal data from other audio applications with the audio signal data that does not require resampling to obtain uplink audio data.

[0075] After resampling is performed to match the audio parameters of other audio applications (excluding each audio application itself) with the audio parameters of each audio application, the resampled audio signal data of other audio applications and the audio signal data that do not need to be resampled can be mixed, and the mixed audio signal data can be written into the second buffer of each audio application.

[0076] S200: Each audio application sends the downlink audio data passed from other audio applications to the corresponding remote device.

[0077] After the mixed downlink audio data is written to the second buffer corresponding to each audio application, each audio application can send downlink audio data from other audio applications to the corresponding remote device. Specifically, each audio application can send downlink audio data from other audio applications in its corresponding buffer to the corresponding remote device.

[0078] Since the second buffer can store a mixture of audio data from other audio applications and audio data transmitted through its own recording interface, each audio application can send the corresponding downlink audio data cached in the second buffer to the corresponding remote device. This allows the remote device corresponding to each audio application to obtain the audio data from the remote devices of other audio applications and play it based on the audio data, so that the corresponding remote user can hear the audio data and achieve audio sharing.

[0079] In one implementation, this embodiment may further include the following steps:

[0080] S310: Acquires ambient audio data via microphone.

[0081] S320: Each audio application sends ambient audio data and downlink audio data from other audio applications to the corresponding remote device.

[0082] After acquiring ambient audio data through microphone 110, the ambient audio data and downlink audio data transmitted by other audio applications can be sent to the corresponding remote device, thereby enabling the remote device to receive and play the ambient audio data and downlink audio data transmitted by other audio applications, thus achieving audio sharing.

[0083] In one implementation, before each audio application sends ambient audio data and downlink audio data from other audio applications to the corresponding remote device, the following steps may be included:

[0084] S321: Mix the ambient audio data and downlink audio data from other audio applications to form uplink audio data for each audio application.

[0085] Uplink audio data can be the audio data used for playback by each audio application. By mixing ambient audio data and downlink audio data from other audio applications to form uplink audio data for each audio application, other remote devices can simultaneously play the ambient audio data and downlink audio data from other audio applications after receiving the uplink audio data, thus achieving audio sharing between remote and near ends.

[0086] In one implementation, the following steps in S321 can be used to describe how to mix ambient audio data with downlink audio data from other audio applications to form uplink audio data for each audio application:

[0087] S3211: Read downlink audio data from the first buffer corresponding to other audio applications, and mix the downlink audio data and ambient audio data passed in by other audio applications to obtain uplink audio data.

[0088] S3212: Writes uplink audio data to the second buffer of each audio application.

[0089] S3213: Each audio application sends the uplink audio data cached in the corresponding second buffer to the corresponding remote device.

[0090] Since the audio data in the second buffer can include audio data from other audio applications besides the audio application itself, during the process of mixing ambient audio data and downlink audio data from other audio applications to form uplink audio data for each audio application, downlink audio data can be read from the first buffer corresponding to other audio applications. The read downlink audio data from other audio applications and the ambient audio data are then mixed to obtain the uplink audio data. By writing the uplink audio data into the second buffer of each audio application, each audio application sends the corresponding uplink audio data buffered in its second buffer to the corresponding remote device.

[0091] When multiple audio applications are running simultaneously, the system can acquire downlink audio data transmitted from the remote device via the corresponding audio application. Each audio application then sends the downlink audio data transmitted from other audio applications to the corresponding remote device. This allows audio applications on other remote devices, relative to a particular audio application, to receive the audio information from that application and play it based on the audio data. This enables users on remote devices to hear the corresponding content, thereby achieving audio sharing when multiple audio applications are running simultaneously, which helps improve the user experience.

[0092] Based on the audio sharing method of the conference equipment described above, the following describes the audio sharing method from the hardware perspective of the electronic device 100 (which can be referred to as the conference equipment in the conference scenario).

[0093] like Figure 2 As shown, the electronic device 100 may include a communication circuit 120 and an execution circuit 130. The communication circuit 120 can be used to establish a communication connection with a remote device. The execution circuit 130 may be coupled to the communication circuit 120.

[0094] The execution circuit 130 can respond to the launch of an audio application by acquiring downlink audio data transmitted from a remote device via the corresponding audio application through the communication circuit 120. The execution circuit 130 then transmits the downlink audio data transmitted from other audio applications to the corresponding remote device through each audio application, enabling the user on the remote device to hear the corresponding content. This achieves audio sharing in the case of multiple audio applications running simultaneously, improving the user experience. The execution circuit 130 can be, for example, a processor, which can be a general-purpose processor. The electronic device 100 can support multiple audio applications running simultaneously (referred to as multiple instances), and each audio application transmits downlink audio data from the corresponding remote device when it is launched.

[0095] In one implementation, when the execution circuit 130 obtains downlink audio data transmitted from the remote device via the corresponding audio application through the communication circuit 120, it can cache the downlink audio data in the buffer corresponding to the audio application, write the downlink audio data transmitted by each audio application into the buffer corresponding to other audio applications, and then send the downlink audio data transmitted by other audio applications in the buffer corresponding to each audio application to the corresponding remote device.

[0096] Specifically, the buffer may include a first buffer and a second buffer. The execution circuit 130 can be used to write the downlink audio data received from each audio application into the first buffer corresponding to each audio application. It can also write the downlink audio data received from each audio application into the second buffer corresponding to other audio applications. The execution circuit 130 can then send the downlink audio data stored in the second buffer corresponding to each audio application to the corresponding remote device.

[0097] In one implementation, the execution circuit 130 can also be used to write downlink audio data cached in the first buffer corresponding to each audio application into the second buffer corresponding to other audio applications.

[0098] In one implementation, the execution circuit 130 can request a corresponding first buffer and a second buffer for each audio application, and record the memory index of the first buffer and the memory index number of the second buffer, and then write the downlink audio data passed in by each audio application into the first buffer corresponding to each audio application and the second buffer corresponding to other audio applications.

[0099] In one implementation, the execution circuit 130 can also perform mixing processing on the downlink audio data passed in by other audio applications to obtain uplink audio data for each audio application, and write the uplink audio data into the second buffer corresponding to each audio application.

[0100] In one implementation, the execution circuit 130 can monitor the number of audio applications playing audio tracks and the corresponding number of first buffers, and read the downlink audio data buffered in the first buffer corresponding to each monitored audio application. The read downlink audio data from other audio applications (excluding the audio application itself) is then mixed to obtain uplink audio data.

[0101] In one implementation, the downlink audio data may include audio parameters and downlink audio signal data. The execution circuit 130 can determine whether the audio parameters read from other audio applications match the audio parameters corresponding to each audio application. If they do not match, the execution circuit 130 can resample the downlink audio signal data corresponding to the mismatched audio applications so that the audio parameters corresponding to the resampled downlink audio signal data match the audio parameters corresponding to each audio application. The execution circuit 130 can then mix the resampled audio signal data from other audio applications with the audio signal data that does not require resampling to obtain the uplink audio data.

[0102] In one implementation, the electronic device 100 may further include a microphone 110, which may be coupled to the execution circuit 130. Specifically, the microphone 110 may acquire ambient audio data, and the execution circuit 130 may transmit the ambient audio data and downlink audio data from other audio applications to the corresponding remote device via each audio application.

[0103] In one implementation, the execution circuit 130 can mix ambient audio data and downlink audio data from other audio applications to obtain uplink audio data for each audio application, and then each audio application can send its uplink audio data to the corresponding remote device.

[0104] In one implementation, the execution circuit 130 can write the downlink audio data passed from each audio application into the first buffer corresponding to each audio application, read downlink audio data from the first buffers corresponding to other audio applications, and mix the read downlink audio data passed from other audio applications with ambient audio data to obtain uplink audio data. Then, the execution circuit 130 can write the uplink audio data of each audio application into the corresponding second buffer, so that each audio application can send the uplink audio data buffered in the corresponding second buffer to the corresponding remote device, thereby realizing audio sharing.

[0105] For other steps in the first embodiment of the electronic device of this application, please refer to the description in the embodiment of the audio sharing method of the electronic device of this application, which will not be repeated here.

[0106] like Figure 6 As shown in the embodiments of the electronic device described in this application, the electronic device 100 can be a device for executing the audio sharing method of the electronic device in the embodiments of this application. The electronic device 100 may include a communication circuit 120, an execution circuit 130, a memory 140, a microphone 110, and a speaker 150. Specifically, the communication circuit 120, the memory 140, the microphone 110, and the speaker 150 are coupled to the execution circuit 130.

[0107] The memory 140 is used to store computer programs and may be RAM (Read-Only Memory), ROM (Random Access Memory), or other types of storage devices. Specifically, the memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory is used to store at least one line of program code.

[0108] The execution circuit 130 is used to control the operation of the electronic device 100. The execution circuit 130 can be a processor, which can also be called a CPU (Central Processing Unit). The processor may be an integrated circuit chip with signal processing capabilities. The processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor, etc.

[0109] The execution circuit 130 is used to execute the computer program stored in the memory 140 to implement the audio sharing method of the electronic device described in the embodiments of the present application.

[0110] The electronic device 100 may also include a communication circuit 120, which is a communication connection device or circuit used by the electronic device 100 to communicate with external devices, so that the execution circuit 130 can interact with external devices via the communication circuit 120.

[0111] For a detailed description of the functions and execution processes of each functional module or component in the embodiments of the electronic device of this application, please refer to the description in the above embodiments of the audio sharing method of the electronic device of this application, which will not be repeated here.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed electronic device 100 and audio sharing method can be implemented in other ways. For example, the embodiments of the electronic device 100 described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0114] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A conference device, characterized in that, The conferencing device interacts with a remote device corresponding to each of at least two audio applications, and the conferencing device includes: A communication circuit for communicating with the remote device; An execution circuit, coupled to the communication circuit, responds to the launch of the audio application by acquiring downlink audio data transmitted from the remote device via the corresponding audio application through the communication circuit, and the execution circuit transmits the downlink audio data transmitted from other audio applications to the corresponding remote device through each audio application. When the execution circuit obtains the downlink audio data passed in by the audio application, it caches the downlink audio data into the cache area corresponding to the audio application. Write the downlink audio data passed by each of the audio applications into the buffer corresponding to the other audio applications; send the downlink audio data passed by the other audio applications in the buffer corresponding to each audio application to the corresponding remote device.

2. The conference equipment according to claim 1, characterized in that, The cache area includes a first cache area and a second cache area; The execution circuit is used to write the downlink audio data passed in by each audio application into the first buffer corresponding to each audio application; write the downlink audio data passed in by each audio application into the second buffer corresponding to other audio applications; and send the downlink audio data stored in the second buffer corresponding to each audio application to the corresponding remote device.

3. The conference equipment according to claim 2, characterized in that, The execution circuit is used to write the downlink audio data cached in the first buffer corresponding to each audio application into the second buffer corresponding to other audio applications.

4. The conference equipment according to claim 2, characterized in that, The execution circuit requests the first cache and the second cache corresponding to each audio application, records the memory index of the first cache and the memory index number of the second cache, and then writes the downlink audio data passed in by each audio application into the first cache corresponding to each audio application and the second cache corresponding to the other audio applications.

5. The conference equipment according to claim 2, characterized in that, The execution circuit performs mixing processing on the downlink audio data input from other audio applications to obtain uplink audio data for each audio application, and writes the uplink audio data into the second buffer corresponding to each audio application.

6. The conference equipment according to claim 5, characterized in that, The execution circuit monitors the number of audio applications playing using the audio track and the corresponding number of the first buffer. Read the downlink audio data cached in the first buffer corresponding to each of the monitored audio applications; perform mixing processing on the downlink audio data passed in by other audio applications besides the audio application itself to obtain the uplink audio data.

7. The conference equipment according to claim 6, characterized in that, The downlink audio data includes audio parameters and downlink audio signal data; The execution circuit determines whether the audio parameters read from other audio applications match the audio parameters corresponding to each audio application. If there is a mismatch, the execution circuit resamples the downlink audio signal data corresponding to the mismatched audio application so that the audio parameters corresponding to the resampled downlink audio signal data match the audio parameters corresponding to each audio application. The execution circuit mixes the resampled audio signal data corresponding to other audio applications with the audio signal data that does not require resampling to obtain the uplink audio data.

8. The conference equipment according to claim 1, characterized in that, The conference equipment also includes a microphone, which is coupled to the execution circuit. The microphone acquires ambient audio data; the execution circuit sends the ambient audio data and the downlink audio data transmitted by the other audio applications to the corresponding remote device via each audio application.

9. The conference equipment according to claim 8, characterized in that, The execution circuit mixes the ambient audio data with the downlink audio data transmitted from other audio applications to obtain uplink audio data for each audio application; each audio application then sends its uplink audio data to the corresponding remote device.

10. The conference equipment according to claim 9, characterized in that, The execution circuit writes the downlink audio data passed by each audio application into the first buffer corresponding to each audio application; reads the downlink audio data from the first buffer corresponding to other audio applications, mixes the read downlink audio data passed by other audio applications with the ambient audio data to obtain the uplink audio data, and writes the uplink audio data of each audio application into the corresponding second buffer; each audio application sends the uplink audio data cached in the corresponding second buffer to the corresponding remote device.

11. An audio sharing method for a conferencing device, characterized in that, The conferencing equipment interacts with the remote device corresponding to each of the at least two audio applications, characterized in that the audio sharing method includes: Obtain downlink audio data transmitted from the remote device via the corresponding audio application; Each of the aforementioned audio applications sends the downlink audio data received from other audio applications to the corresponding remote device; The step of obtaining the downlink audio data transmitted from the remote device via the corresponding audio application includes: Write the downlink audio data passed in by each of the audio applications into the cache area corresponding to each of the audio applications and the cache areas corresponding to other audio applications; The step of sending the downlink audio data transmitted from other audio applications to the corresponding remote device via each of the audio applications includes: Each audio application sends the downlink audio data passed from other audio applications in the corresponding buffer to the corresponding remote device.

12. The method according to claim 11, characterized in that, The cache area includes a first cache area and a second cache area; The step of caching the downlink audio data passed from each of the audio applications to the cache area corresponding to the audio application and the cache areas corresponding to other audio applications includes: Write the downlink audio data passed in by each audio application into the first buffer corresponding to each audio application and the second buffer corresponding to the other audio applications; The step of sending the downlink audio data from other audio applications in the corresponding buffer to the corresponding remote device via each audio application includes: Each audio application sends the corresponding downlink audio data cached in the second buffer to the corresponding remote device.

13. The method according to claim 12, characterized in that, The step of writing the downlink audio data passed by each audio application into the first buffer corresponding to each audio and the second buffer corresponding to other audio applications includes: Write the downlink audio data passed in by each audio application into the first buffer corresponding to each audio; The downlink audio data cached in the first cache area corresponding to each of the audio applications is written to the second cache area corresponding to the other audio applications.

14. The method according to claim 13, characterized in that, Before writing the downlink audio data passed by each audio application into the first buffer corresponding to each audio application, the process includes: For each audio application, apply for a corresponding first cache area and record the memory index number corresponding to the first cache area; Before writing the downlink audio data cached in the first buffer corresponding to each of the audio applications into the second buffer corresponding to other audio applications, the process includes: For each audio application, apply for a corresponding second cache area and record the memory index number corresponding to the second cache area.

15. The method according to claim 12, characterized in that, The step of writing the downlink audio data passed by each audio application into the first buffer corresponding to each audio application and the second buffer corresponding to the other audio applications includes: The downlink audio data from other audio applications is mixed to obtain uplink audio data for each audio application, and the uplink audio data is written to the corresponding second buffer.

16. The method according to claim 15, characterized in that, The step of mixing the downlink audio data received from other audio applications to obtain the uplink audio data for each audio application includes: Read the downlink audio data cached in the first cache area corresponding to each of the audio applications; The uplink audio data is obtained by mixing the downlink audio data received from other audio applications.

17. The method according to claim 16, characterized in that, The step of reading the downlink audio data cached in the first cache area corresponding to each audio application includes: The system monitors the number of audio applications playing using audio tracks and the corresponding number of first buffers, and reads the downlink audio data of the first buffer corresponding to each monitored audio application.

18. The method according to claim 16, characterized in that, The downlink audio data includes audio parameters and downlink audio signal data; The step of mixing the downlink audio data corresponding to the other audio applications read to obtain the uplink audio data includes: Determine whether the audio parameters read from other audio applications (excluding each audio application itself) match the audio parameters passed by each audio application. If there is a mismatch, the downlink audio signal data passed by the mismatched audio application is resampled so that the audio parameters corresponding to the resampled downlink audio signal data match the audio parameters passed by each audio application. The uplink audio data is obtained by mixing the resampled downlink audio signal data corresponding to other audio applications and the audio signal data that does not require resampling.

19. The method according to claim 11, characterized in that, The method further includes: Acquire ambient audio data using a microphone; The step of sending the downlink audio data transmitted from other audio applications to the corresponding remote device via each of the audio applications includes: Each of the aforementioned audio applications sends the ambient audio data and the downlink audio data transmitted by the other audio applications to the corresponding remote device.

20. The method according to claim 19, characterized in that, The step of sending the ambient audio data and the downlink audio data transmitted by the other audio applications to the corresponding remote device via each audio application includes: The ambient audio data and the downlink audio data transmitted from other audio applications are mixed to form the uplink audio data for each audio application; Each of the audio applications sends its uplink audio data to the corresponding remote device.

21. The method according to claim 20, characterized in that, Its features are, The step of obtaining the downlink audio data transmitted from the remote device via the corresponding audio application includes: Write the downlink audio data passed in by each audio application into the first buffer corresponding to each audio application; The step of mixing the environmental audio data with the downlink audio data transmitted from other audio applications to form the uplink audio data for each audio application includes: The downlink audio data is read from the first buffer corresponding to the other audio applications, and the downlink audio data and the ambient audio data are mixed to obtain the uplink audio data. The upstream audio data is written to the second cache area of ​​each audio application; The step of each audio application sending its uplink audio data to the corresponding remote device includes: Each audio application sends the uplink audio data cached in the corresponding second buffer to the corresponding remote device.

22. An electronic device, characterized in that, The device includes an execution circuit, a memory, a communication circuit, a microphone, and a speaker, wherein the communication circuit, the memory, the microphone, and the speaker are coupled to the execution circuit, the memory is used to store a computer program, and the execution circuit is used to execute the computer program to implement the method as described in any one of claims 11-21.

23. A communication system, characterized in that, It includes the electronic device of claim 22 and at least two remote devices, wherein the electronic device is communicatively connected to the remote devices.

Images