Audio transmission method, system and apparatus

Abstract

The embodiment of the application discloses an audio transmission method, system and device, the method comprises: obtaining at least two session audios from different sender clients; determining a corresponding target filtering strategy according to a current session scene; filtering target session audio from the at least two session audios according to the target filtering strategy; and sending the target session audio to a receiver client. By sending the filtered target session audio to the receiver client, the transmission of the filtered target session audio is realized. Since the number of the filtered session audio is less than the total number of the received session audio before filtering, the data amount of the session audio sent to the receiver client is reduced, the pressure on the load and bandwidth during the transmission of the voice is reduced, the transmission quality of the session audio is improved, the quality of the voice call is improved, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, in particular to an audio transmission method, system and device. BACKGROUND

[0002] With the rapid development of computer technology, the computing power of devices has been improved. For example, in the process of voice call, audio information of multiple users can be mixed, and the mixed audio information is sent to a receiving user, so that the receiving user can receive audio information of multiple users at the same time.

[0003] However, with the increase in the number of users in the process of voice call, the transmission of audio information brings great challenges to load and bandwidth, affecting the quality of voice call. SUMMARY

[0004] Embodiments of the present application provide an audio transmission method, system and device, which can improve the quality of voice call and improve user experience.

[0005] In a first aspect, embodiments of the present application provide an audio transmission method, comprising:

[0006] obtaining at least two session audios from different sender clients;

[0007] determining a corresponding target filtering strategy according to a current session scenario;

[0008] filtering a target session audio from the at least two session audios according to the target filtering strategy;

[0009] sending the target session audio to a receiving client.

[0010] In a second aspect, embodiments of the present application further provide an audio transmission system, which comprises a sender client, a routing server and a receiving client, wherein:

[0011] the sender client is configured to transmit session audio to the routing server;

[0012] the routing server is configured to determine a corresponding target filtering strategy according to a current session scenario, filter a target session audio from at least two session audios from different sender clients according to the target filtering strategy, and send the target session audio to the receiving client;

[0013] the receiving client is configured to receive the target session audio sent by the routing server.

[0014] In a third aspect, embodiments of the present application further provide an audio transmission device, comprising:

[0015] an acquisition module configured to acquire at least two pieces of session audio from different sender clients;

[0016] a determination module configured to determine a target filtering strategy according to a current session scenario;

[0017] a routing module configured to filter target session audio from the at least two pieces of session audio according to the target filtering strategy;

[0018] a sending module configured to send the target session audio to a receiver client.

[0019] In some embodiments of the present application, the sending module includes:

[0020] a first determination unit configured to determine a target audio component from at least two audio components of the target session audio if the target session audio includes the at least two audio components of different users;

[0021] a processing unit configured to perform transition processing on the target audio component to obtain processed target session audio;

[0022] a first sending unit configured to send the processed target session audio to the receiver client.

[0023] In some embodiments of the present application, the processing unit includes:

[0024] a first processing subunit configured to extract the target audio component from the at least two audio components and construct session audio according to the target audio component to obtain processed target session audio.

[0025] In some embodiments of the present application, the processing unit includes:

[0026] a second processing subunit configured to perform enhancement processing on the target audio component in the target session audio to obtain processed target session audio.

[0027] In some embodiments of the present application, the processing unit includes:

[0028] a third processing subunit configured to determine a non-target audio component from the target session audio according to the target audio component and perform suppression processing on the non-target audio component to obtain processed target session audio.

[0029] In some embodiments of the present application, the first determination unit includes:

[0030] a determination subunit configured to determine a target audio component from the at least two audio components according to energy information of the audio components.

[0031] In some embodiments of the present application, the determining subunit is configured to:

[0032] determine the user identity tag information corresponding to the audio component;

[0033] obtain audio data transmission records;

[0034] determine the audio transmission frequency corresponding to the user identity tag information according to the audio data transmission records;

[0035] determine the target audio component from the at least two audio components according to the energy information and the audio transmission frequency.

[0036] In some embodiments of the present application, the determining subunit is configured to:

[0037] determine the audio transmission level of the audio component according to the user identity tag information;

[0038] determine the target audio component from the at least two audio components according to the energy information, the audio transmission frequency, and the audio transmission level.

[0039] In some embodiments of the present application, the determining module comprises:

[0040] The second determining unit is configured to, when the current session scenario is a conference scenario, take the audio energy of the filtered conversation audio satisfying a preset condition as the target filtering strategy corresponding to the conference scenario.

[0041] In some embodiments of the present application, the obtaining module comprises:

[0042] The obtaining unit is configured to obtain at least two paths of conversation audio from at least two uplink engine servers.

[0043] For each path of conversation audio, the conversation audio is obtained by the uplink engine server performing correction processing on original conversation audio from at least two uplink transparent servers, and the original conversation audio is obtained by the at least two uplink transparent servers from a sender client through an uplink transmission link.

[0044] The correction processing comprises a converging operation and a deduplication operation on the original conversation audio transmitted by the at least two uplink transmission links.

[0045] In some embodiments of the present application, the sending module comprises:

[0046] The second sending unit is configured to send the target conversation audio to a listener client through a downlink engine server.

[0047] The target conversation audio is sent to at least two downlink transparent servers by the downlink engine server respectively, and is sent to the call recipient client by the downlink transparent server through a downlink transmission link.

[0048] In a fourth aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor implements the steps of the audio transmission method described above when executing the computer program.

[0049] In a fifth aspect, a computer readable storage medium is provided, which stores a computer program. The computer program is executable on a processor to implement the steps of the audio transmission method described above.

[0050] In the embodiments of the present application, the target conversation audio is selected from the multiple conversation audios sent by different sender clients according to the target filtering strategy determined according to the current conversation scenario, and the selected target conversation audio is sent to the call recipient client, so as to realize the filtering and transmission of the conversation audio. The filtering strategy of the conversation audio is determined according to the conversation scenario, and the conversation audio is filtered based on the filtering strategy, so as to realize the filtering of the conversation audio based on the conversation scenario. The selected target conversation audio is sent to the call recipient client, so as to realize the transmission of the selected target conversation audio. Since the number of the filtered conversation audios is less than the total number of the received conversation audios before filtering, the data amount of the conversation audios sent to the call recipient client is reduced, the load and bandwidth pressure of the conversation audio in the transmission process is reduced, the transmission quality of the conversation audio is improved, the quality of the voice call is improved, and the user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 is a scene schematic diagram of the audio transmission method provided by the embodiments of the present application;

[0053] Figure 2 is a flowchart of the audio transmission method provided by the embodiments of the present application;

[0054] Figure 3 is a flowchart of the audio transmission method in a conference scenario provided by the embodiments of the present application;

[0055] Figure 4 is a framework structure diagram of an audio transmission system provided by an embodiment of the present application;

[0056] Figure 5 is a structure schematic diagram of a routing server provided by an embodiment of the present application;

[0057] Figure 6 is a flow schematic diagram of audio data transmitted from a sender client to a routing server provided by an embodiment of the present application;

[0058] Figure 7 is a flow schematic diagram of audio data transmitted from a routing server to a receiver client provided by an embodiment of the present application;

[0059] Figure 8 is a transmission control flow schematic diagram of audio data transmitted from a routing server to a receiver client provided by an embodiment of the present application;

[0060] Figure 9 is a structure schematic diagram of an audio transmission device provided by an embodiment of the present application;

[0061] Figure 10 is a structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0063] The embodiments of the present application provide an audio transmission method, system and device. Specifically, the embodiments of the present application provide an audio transmission device suitable for an electronic device, wherein the electronic device includes a terminal (such as a user terminal) or a server and the like, wherein the terminal includes but is not limited to a mobile phone, a computer, a tablet, a smart voice interaction device, a smart home appliance or a vehicle-mounted terminal and the like, and the server can be a physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms and the like basic cloud computing services, and the server can be directly or indirectly connected through wired or wireless communication.

[0064] The embodiments of the present application can execute the audio transmission method independently by the terminal device, or can execute the audio transmission method independently by the server, or can execute the audio transmission method by the terminal and the server together. Please refer to Figure 1 For example, the terminal device and the server execute the audio transmission method together, and the specific execution process of the audio transmission method is as follows:

[0065] The server 10 acquires at least two session audios from different sender clients 11, determines the corresponding target filtering strategy according to the current session scenario, filters the target session audio from the at least two session audios according to the target filtering strategy, and sends the filtered target session audio to the receiver client 12.

[0066] In the embodiments of the present application, multiple users can participate in the session at the same time, so multiple session audios can be acquired from multiple sender clients during the session. For example, in the conference process, multiple users participate at the same time, and multiple users make sound at the same time for the conference content or conference subject, and different users can correspond to different sender clients, that is, each user participates in the session based on a single sender client, or multiple users are in the same environment where the sender client is located, that is, multiple users participate in the session based on the same sender client, so the session scenario of multiple users exists the case that multiple session audios need to be transmitted.

[0067] In the embodiments of the present application, the filtering strategy of the session audio can be determined according to the session scenario, wherein the session scenario can be identified according to the content of the session audio, or the theme information of the session can be identified to obtain the scenario of the session. In the embodiments of the present application, the session scenario can include a conference scenario, a multi-user communication in a local area network, a multi-user group chat, or a multi-user video call, etc.

[0068] In the embodiments of the present application, multiple session audios sent from different sender clients are acquired, the corresponding target filtering strategy is determined according to the current session scenario, the target session audio is filtered from the multiple session audios according to the target filtering strategy, and the filtered target session audio is sent to the receiver client, so as to realize the filtering and transmission of the session audio. The filtering strategy of the session audio is determined according to the session scenario, and the filtering of the session audio is based on the filtering strategy, so as to realize the filtering of the session audio based on the session scenario. The filtered target session audio is sent to the receiver client, so as to realize the transmission of the filtered target session audio. Since the number of the filtered session audio is less than the total number of the received session audio before filtering, the data amount of the session audio sent to the receiver client is reduced, the pressure on the load and bandwidth in the transmission process of the session audio is reduced, the transmission quality of the session audio is improved, the quality of the voice call is improved, and the user experience is improved.

[0069] The following will be described in detail respectively. It should be noted that the sequence of the following embodiments is not limited as the priority order of the embodiments.

[0070] Please refer to Figure 2 , Figure 2 A flowchart of an audio transmission method provided in an embodiment of the present application is shown in FIG. 1. The specific flow of the audio transmission method can be as follows:

[0071] 101. Obtain at least two channels of conversation audio from different sender clients.

[0072] In the embodiment of the present application, a sender client refers to a device that receives a user's voice and transmits the user's voice to a listener client in the form of encoding or the like. By obtaining the conversation audio transmitted by multiple sender clients, multiple channels of conversation audio can be obtained.

[0073] In the embodiment of the present application, each sender client can correspond to one user, i.e., each user can make a voice call through a single sender client, for example, a single user makes a call with other users through a mobile phone or a personal computer; or each sender client can correspond to multiple users, i.e., multiple users make a voice call through a single sender client, for example, in a conference scenario, multiple users make a call through a video and audio system device in a conference room.

[0074] In the embodiment of the present application, each sender client can correspond to one channel of conversation audio, and each channel of conversation audio can include a single user's conversation data stream or multiple users' conversation data streams.

[0075] In the embodiment of the present application, the conversation audio transmitted by the uplink engine server can be received, and the uplink engine server is used to obtain the conversation audio of the sender client, i.e., in some embodiments of the present application, the step of "obtaining at least two channels of conversation audio from different sender clients" includes:

[0076] Obtaining at least two channels of conversation audio from at least two uplink engine servers;

[0077] For each channel of conversation audio, the conversation audio is obtained by the uplink engine server performing correction processing on the original conversation audio from at least two uplink transparent servers, and the original conversation audio is obtained by the at least two uplink transparent servers from the sender client through uplink transmission links.

[0078] The correction processing includes a converging operation and a deduplication operation on the original conversation audio transmitted by the at least two uplink transmission links.

[0079] In the embodiment of the present application, the uplink engine server has the ability of audio data processing and forwarding. Through the correction processing of the original conversation audio sent by the sender client, it is ensured that the conversation audio received from the uplink engine server is consistent with the conversation audio sent by the sender client, avoiding the influence caused by data loss, missing or data error in the audio data receiving process, and improving the transmission quality of the audio data.

[0080] In the embodiment of the present application, the uplink transparent server has the ability of audio data transparent forwarding. The uplink transparent server is used to transparently forward the audio data received from the sender client, for example, the original audio sent by the sender client is transmitted to the uplink engine server.

[0081] In the embodiment of the present application, the uplink engine server has the ability of audio data processing and forwarding. Through the correction processing of the original conversation audio sent by the sender client, it is ensured that the conversation audio received from the uplink engine server is consistent with the conversation audio sent by the sender client, avoiding the influence caused by data loss, missing or data error in the audio data receiving process, and improving the transmission quality of the audio data.

[0082] In the embodiment of the present application, the uplink engine server has the ability of audio data processing and forwarding. Through the correction processing of the original conversation audio sent by the sender client, it is ensured that the conversation audio received from the uplink engine server is consistent with the conversation audio sent by the sender client, avoiding the influence caused by data loss, missing or data error in the audio data receiving process, and improving the transmission quality of the audio data.

[0083] 102. Determine the corresponding target filtering strategy according to the current conversation scene.

[0084] In the embodiment of the present application, the conversation scene includes the scene in which the conversation is located, for example, the conversation scene can include a conference scene, a multi-user communication in a local area network, a multi-user group chat or a multi-user video call.

[0085] In the embodiment of the present application, the filtering strategy includes a strategy for filtering conversation audio. Different conversation audios are filtered from multiple conversation audios based on different filtering strategies. In the embodiment of the present application, the filtered conversation audio can include a single user data stream or multiple user data streams.

[0086] According to the conversation scene, the strategy for filtering conversation audio is determined, so that the filtering of conversation audio can meet the needs of the current scene or the needs of the user.

[0087] In this embodiment of the application, when the conversation scenario is a conference scenario, the audio capabilities of the conversation audio can meet preset conditions as a filtering strategy, and the target conversation audio can be filtered based on this filtering strategy. That is, optionally, in some embodiments of this application, the step "determine the corresponding target filtering strategy according to the current conversation scenario" includes:

[0088] When the current conversation scenario is a meeting scenario, the audio energy of the selected conversation audio that meets the preset conditions will be used as the target selection strategy corresponding to the meeting scenario.

[0089] In this process, the audio of a conversation that meets the preset conditions for audio energy in a meeting scenario is filtered to ensure that the filtered audio meets the audio energy requirements. By transmitting the audio of the conversation that meets the audio energy requirements, the quality of the conversation during the meeting is improved.

[0090] In this embodiment of the application, for a conference scenario, the number of participants can be set, and the conversation audio can be selected based on the number of participants. Optionally, in this embodiment, a preset threshold for the number of participants can be used as a filtering condition for the conversation audio. For example, conversation audio with three or fewer participants can be selected as the target conversation audio. Specifically, when each sending client corresponds to one user, the conversation audio of three or fewer sending clients can be selected as the target conversation audio. When each sending client corresponds to multiple users, the conversation audio of three or fewer users can be selected from among the multiple users as the target conversation audio.

[0091] 103. Select the target conversation audio from at least two conversation audio streams according to the target filtering strategy.

[0092] The filtering strategy for conversation audio ensures that the selected audio meets the relevant requirements. Specifically, by filtering the target conversation audio from multiple sources, the data volume corresponding to the target audio is reduced compared to the original multi-source audio. This decreases the amount of data sent to the receiving client, reduces the load and bandwidth pressure on the conversation audio during transmission, and improves voice call quality.

[0093] 104. Send the target session audio to the receiving client.

[0094] Specifically, by sending the selected target conversation audio to the receiving client, the target conversation audio can be decoded and played by the receiving client, thus realizing the transmission of the target conversation audio.

[0095] In the embodiments of the present application, the sender client and the listener client can be the same type of call device. Based on the user's speaking needs and the listening needs, the call device is divided into a sender client or a listener client.

[0096] In the embodiments of the present application, since the conversation audio of a single sender client can contain the speaking information of multiple users, each conversation audio also contains multiple audio components of the speaking of multiple users. Therefore, the audio components in the screened conversation audio can also be screened to further optimize the transmission process of the audio, that is, in some embodiments of the present application, the step of "sending the target conversation audio to the listener client" comprises:

[0097] If the target conversation audio includes at least two audio components of different users, determining a target audio component from the at least two audio components;

[0098] Performing transition processing on the target audio component to obtain a processed target conversation audio;

[0099] Sending the processed target conversation audio to the listener client.

[0100] In the embodiments of the present application, the audio component refers to the audio data formed after the independent speaking of a user, and the combination of the audio components of multiple users forms the conversation audio. In the embodiments of the present application, each audio component can correspond to a data stream of speaking, and the convergence of multiple data streams forms the conversation audio.

[0101] By determining the target audio component from the multiple audio components of the conversation audio and performing transition processing on the target audio component, the influence degree of the target audio component in the target conversation audio is highlighted, and the speaking effect of the target audio component is improved.

[0102] In the embodiments of the present application, the transition processing on the target audio component includes extraction, enhancement processing or suppression processing of the target audio component. By performing transition processing on the target audio component, the influence degree of the target audio component in the target conversation audio is improved, that is, in some embodiments of the present application, the step of "performing transition processing on the target audio component to obtain a processed target conversation audio" comprises:

[0103] extracting the target audio component from the at least two audio components and constructing the conversation audio according to the target audio component to obtain the processed target conversation audio, or

[0104] performing enhancement processing on the target audio component in the target conversation audio to obtain the processed target conversation audio, or

[0105] The non-target audio component is determined from the target audio component in the target conversation audio, and the non-target audio component is suppressed to obtain processed target conversation audio.

[0106] The target audio component is extracted from the multiple audio components in the target conversation audio, and the conversation audio is constructed based on the target audio component to obtain the processed target conversation audio, so that the target audio component is included in the processed target conversation audio. When the processed target audio component is sent to the receiving client, the receiving client can play the user's voice containing only the target audio component, thereby improving the voice effect of the corresponding user.

[0107] In the embodiments of the present application, the multiple audio components can be aggregated to construct the conversation audio.

[0108] The enhancement processing of the target audio component improves the influence of the target audio component in the target conversation audio and improves the voice effect of the target audio component. In the embodiments of the present application, the enhancement processing of the audio component includes noise reduction, energy improvement, volume, etc.

[0109] The suppression processing of the non-target audio component reduces the influence of the non-target audio component in the conversation audio, thereby improving the influence of the target audio component in the conversation audio. The non-target audio component is an audio component other than the target audio component in the conversation audio.

[0110] In the embodiments of the present application, the target audio component can be selected from the target conversation audio according to the energy information of each audio component, i.e., in some embodiments of the present application, the step of "determining the target audio component from at least two audio components" includes:

[0111] The target audio component is determined from at least two audio components according to the energy information of the audio component.

[0112] The energy information of the audio component refers to the energy information of the sound corresponding to the audio component. The higher the energy of the sound, the greater the influence of the sound in the conversation audio. Therefore, the audio component with greater influence on the conversation audio can be selected as the target audio component. For example, in the embodiments of the present application, the audio component with a larger sound volume can be selected as the target audio component.

[0113] In the embodiments of the present application, the audio component can also be selected according to the voice frequency of each user, i.e., in some embodiments of the present application, the step of "determining the target audio component from at least two audio components according to the energy information of the audio component" includes:

[0114] determine user identity tag information corresponding to the audio component;

[0115] obtain audio data transmission records;

[0116] determine an audio transmission frequency corresponding to the user identity tag information according to the audio data transmission records;

[0117] determine a target audio component from the at least two audio components according to the energy information and the audio transmission frequency.

[0118] In the embodiments of the present application, the user identity tag information is used to identify the identity of a user, and each user corresponds to a unique identity tag.

[0119] Through analysis of the audio data transmission records, the audio transmission frequency of each user can be obtained. The higher the audio transmission frequency of a user, the more the user speaks, and the higher the importance of the user in the conversation process. Transmitting the audio component of the user with high importance to the receiving client can improve the accuracy of audio transmission in the conversation process. Suppressing or hiding the audio component with low importance can filter the sound irrelevant to the conversation, such as filtering the voice of ineffective communication and private communication in the conference conversation process, and ensuring the smoothness of voice conference communication.

[0120] The target audio component is screened through the energy information of the audio component and the audio transmission frequency of the user corresponding to the audio component, which realizes screening of the audio component with high importance, ensures the quality of the conversation, and reduces the amount of conversation data to be transmitted and the pressure on load and bandwidth caused by transmission of conversation data.

[0121] In the embodiments of the present application, an audio transmission level can also be set for each user in the conversation process, and the target audio component to be transmitted is screened according to the audio transmission level, that is, in some embodiments of the present application, the step of "determining a target audio component from the at least two audio components according to the energy information and the audio transmission frequency" includes:

[0122] determine the audio transmission level of the audio component according to the user identity tag information;

[0123] determine a target audio component from the at least two audio components according to the energy information, the audio transmission frequency, and the audio transmission level.

[0124] In the embodiments of the present application, the audio transmission level refers to the transmission level of the audio component, and the higher the audio transmission level, the higher the demand for the audio component to be transmitted. In the embodiments of the present application, the audio transmission level of each audio component can be obtained according to the user identity tag, for example, according to the mapping relationship between the user identity tag information and the audio transmission level, the audio transmission level corresponding to each user identity tag information is obtained, and then the audio transmission level corresponding to each audio component is obtained.

[0125] In the embodiments of the present application, the target audio component is screened by the energy information of the audio component, the audio transmission frequency of the user corresponding to the audio component, and the audio transmission level, which realizes the screening of the audio component with high importance, ensures the quality of the conversation, and at the same time, reduces the amount of transmitted conversation data and the pressure of conversation data transmission on load and bandwidth.

[0126] In the embodiments of the present application, the target conversation audio can be sent to the receiving party client by the downlink engine server, that is, in some embodiments of the present application, the step of “sending the target conversation audio to the receiving party client” comprises:

[0127] The target conversation audio is sent to the receiving party client by the downlink engine server.

[0128] The target conversation audio is sent to the receiving party client by the downlink engine server.

[0129] In the embodiments of the present application, the downlink engine server has the capability of audio data processing and forwarding, and the filtered target conversation audio is sent to the receiving party client by the downlink engine server.

[0130] In the embodiments of the present application, the downlink transparent server has the capability of data transparent forwarding, and the target conversation audio is sent to the receiving party client by the downlink transparent server through the downlink engine server.

[0131] In the embodiments of the present application, the target conversation audio is sent to multiple downlink transparent servers respectively, and is sent to the receiving party client by the multiple downlink transparent servers based on the downlink transmission link respectively, so that the receiving party client obtains the target conversation audio through multiple transmission links. In the embodiments of the present application, after the receiving party client obtains the target conversation audio according to the multiple transmission links, the receiving party client can modify the conversation audio according to the multiple target conversation audios, obtain the target conversation audio filtered according to the filtering strategy, and through the modification of the conversation audio, the accuracy of the conversation audio acquisition is improved, the accuracy of the audio call transmission is improved, and the receiving party client can receive and play the accurate conversation audio.

[0132] In the embodiments of the present application, the way of modifying the received conversation audio can also include the way of obtaining redundant packets, wherein the redundant packets can be used to complete the modification of the conversation audio or the recovery of lost packets, etc. by adding the redundant packets of the conversation audio. In the embodiments of the present application, the data recovery or the recovery of lost packets after partial loss of audio data can also be achieved by reinitiating the request for obtaining audio data to reobtain the conversation audio.

[0133] In the embodiments of the present application, the sending frequency of the conversation audio of the sender client can also be adjusted according to the audio data receiving quality of the uplink engine server, or the sending frequency of the target conversation audio of the downlink engine server can be adjusted according to the audio data receiving quality of the listener client. By adjusting the sending frequency according to the receiving quality, the quality of audio transmission is improved, and the quality of voice call is improved.

[0134] In the embodiments of the present application, the target conversation audio is selected from the multiple conversation audios sent from different sender clients according to the target screening strategy determined according to the current conversation scene, and the selected target conversation audio is sent to the listener client, so as to achieve the screening and transmission of the conversation audio. The screening strategy of the conversation audio is determined according to the conversation scene, and the conversation audio is screened based on the screening strategy, so as to achieve the screening of the conversation audio based on the conversation scene. The selected target conversation audio is sent to the listener client, so as to achieve the transmission of the selected target conversation audio. Since the number of the selected conversation audio is less than the total number of the received conversation audio before screening, the amount of data of the conversation audio sent to the listener client is reduced, the pressure on the load and bandwidth in the transmission process of the conversation audio is reduced, the transmission quality of the conversation audio is improved, the quality of voice call is improved, and the user experience is improved.

[0135] In the conference scene, when multiple users speak at the same time in the conference process, multiple user voices need to be transmitted. For this situation, if the number of users speaking is large, the transmission of the conversation audio will cause pressure on the load and bandwidth, and thus affect the conversation quality. Therefore, in the embodiments of the present application, for the multi-user call scene, the conversation audio can be screened, and only part of the key audio is selected for transmission, so as to reduce the pressure on the load and bandwidth of the audio transmission.

[0136] For example, refer to Figure 3 , Figure 3 is a flowchart of the conference scene audio transmission method provided in the embodiments of the present application, and the specific process of the conference scene audio transmission method includes:

[0137] 111. Obtain at least two pieces of conversation audio from different sender communication devices in a conference scenario of at least two users;

[0138] In the embodiments of the present application, the sender communication device in the conference scenario includes a mobile phone, a computer, or an audio-video system device in a conference room, etc.

[0139] In the embodiments of the present application, the conversation audio includes voice information of multiple users participating in the conference, wherein the voice information of the users is transmitted through a communication device, thus each piece of conversation audio corresponds to conversation data sent by one communication device, and the multiple users deliver audio sounds through multiple communication devices during the conference, forming multiple pieces of conversation audio.

[0140] 112. According to the requirements of the conference scenario, screen a plurality of target conversation audios from the multiple pieces of conversation audio, wherein the number of the plurality of target conversation audios is less than or equal to the number of the multiple pieces of conversation audio;

[0141] 113. According to the multiple audio components corresponding to the plurality of target conversation audios, screen a plurality of target audio components from the multiple audio components according to the energy information corresponding to each audio component, the audio transmission frequency of the user voice corresponding to the audio component, or the audio transmission level of the user voice, wherein the number of the plurality of target audio components is less than or equal to the number of the multiple audio components corresponding to the plurality of target conversation audios;

[0142] 114. Perform transition processing on the screened plurality of target audio components to obtain processed target conversation audios, wherein the transition processing includes extracting the target audio components, constructing new target conversation audios (processed target conversation audios) based on the target audio components, or performing enhancement processing on the target audio components in the target conversation audios, or performing suppression processing on the non-target audio components in the target conversation audios;

[0143] 115. Send the processed target conversation audios to a receiver communication device.

[0144] In the multi-person conference scenario, the target conversation audio is selected from the conversation audios of multiple users according to energy information of the conversation audios, and the target conversation audio is processed according to the energy information, an audio transmission frequency or an audio transmission level, so as to realize the optimized processing of the transmitted conversation audios. Since the number of the selected conversation audios is less than the total number of the received conversation audios before the selection, the amount of data of the conversation audios sent to the receiving party client is reduced, the pressure on the load and bandwidth of the conversation audios in the transmission process is reduced, the transmission quality of the conversation audios is improved, the quality of the voice call is improved, and the user experience is improved. Through the transition processing of the conversation audios, the influence degree of the target audio component in the conversation audios is improved, and the call effect in the conference process is improved.

[0145] In the conference scenario with more parameters, the part of the conversation audios selected in the scheme is sent to the receiving party call device, so that the receiving party call device plays the voice corresponding to the part of the conversation audios, and the playing effect of the part of the voice is improved.

[0146] In the embodiments of the present application, the target conversation audio can also be selected from the multiple conversation audios in other ways.

[0147] Please refer to Figure 4 , Figure 4 The framework structure diagram of the audio transmission system provided by the embodiments of the present application is shown. The audio transmission system can include:

[0148] The sender client 201 is configured to transmit the conversation audios to the routing server;

[0149] The routing server 202 is configured to determine the corresponding target filtering strategy according to the current conversation scenario, and select the target conversation audios from at least two conversation audios from different sender clients 201 according to the target filtering strategy, and send the target conversation audios to the receiving party client 203;

[0150] The receiving party client 203 is configured to receive the target conversation audios sent by the routing server 202.

[0151] In the embodiments of the present application, the routing server is configured to select part of the conversation audios from the multiple conversation audios for transmission according to the selection condition, so as to reduce the pressure on the load and bandwidth of the audio transmission. For details, please refer to Figure 5 , Figure 5 The structure diagram of the routing server provided by the embodiments of the present application is shown, wherein the routing server includes:

[0152] The transmission layer 211 is configured to receive the multiple audio data packets sent by different sender clients;

[0153] a buffer 212, configured to store the multi-channel audio data packets in the buffer;

[0154] a timer 213, configured to time the audio data packets in the buffer, wherein only audio data packets in the buffer for a certain time duration are kept;

[0155] a selector 214, configured to select three target conversation audios from the buffer according to energy information meeting a requirement.

[0156] In the embodiments of the present application, the buffer is used to buffer data packets in a certain time duration in a conversation, and the buffering time should not be too long. The selector executes a routing strategy once every packet sending interval: when the number of conversation participants exceeds N (for example, N = 3), the data packets of N conversation participants are selected each time. When the number of conversation participants is less than N, the data packets of all conversation participants are selected.

[0157] In the embodiments of the present application, the function of the routing server is to select several key channels from the multi-channel data streams, so as to not only alleviate the pressure of the downlink bandwidth, but also reduce the performance pressure of the data server and the client caused by processing a large amount of media data.

[0158] In some embodiments of the present application, the system further includes an uplink engine server 204, specifically including:

[0159] a sender client 201, configured to send original conversation audios to the uplink engine server 204;

[0160] the uplink engine server 204, configured to receive the original conversation audios sent by the sender client 201, and perform correction processing on the original conversation audios to obtain conversation audios, and transmit the conversation audios to the routing server 202;

[0161] the routing server 202, configured to receive the conversation audios transmitted by the uplink engine server 204.

[0162] In some embodiments of the present application, the system further includes at least two uplink transparent servers 205, specifically including:

[0163] the sender client 201, configured to transmit the original conversation audios to each uplink transparent server 205 through each uplink transmission link respectively;

[0164] the uplink transparent server 205, configured to transmit the obtained at least two original conversation audios to the uplink engine server 204;

[0165] the uplink engine server 204, configured to perform correction processing according to the received at least two original conversation audios to obtain conversation audios, and transmit the conversation audios to the routing server 202;

[0166] Among the at least two uplink transparent servers 205 corresponding to the same uplink engine server 204, the transmission types of the uplink transmission links connected by different uplink transparent servers are different.

[0167] Correspondingly, please refer to Figure 6 , Figure 6 FIG. 1 is a flowchart of a process of transmitting audio data from a sender client to a routing server according to an embodiment of the present application. In the embodiment of the present application, the process of transmitting audio data from the sender client to the routing server includes the following steps.

[0168] 221. The sender client establishes single-link or double-link connection with the uplink transparent server (the embodiment of the present application takes double-link as an example), and adds redundant packets to the audio and video data collected and processed according to the transmission quality of the uplink transmission link of the sender client (the transmission quality is poor, and the redundant proportion is high), and sends the data to the corresponding uplink transparent server through the two links.

[0169] 222. The uplink transparent server performs data transmission, and forwards the two-way media data to the uplink engine server of the sender client.

[0170] 223. The uplink engine server performs packet loss recovery and double-link data deduplication on the two-way media data, and then sends the recovered data packets to the routing server.

[0171] 224. The routing server receives the data packets sent by the uplink engine server.

[0172] In the embodiment of the present application, after the audio data packets of the sender client arrive at the routing server, the data packets are selected together with the data packets of other users in the call by the routing server.

[0173] In the embodiment of the present application, the double-link can include 4G link and Wi-Fi link. When the user joins the call, the terminal active network card information is brought to the signaling background by the client, the signaling background allocates the optimal transparent server for each active network card, the client can establish the same number of links as the number of active network cards for uplink and downlink data transmission, and the data transmitted by the two uplink or downlink links is completely the same. In this way, even if the data of one of the links is lost due to network problems, it can be recovered from the data packets transmitted by the other link. The uplink double-link data is deduplicated in the uplink engine of the engine server, and the downlink double-link data is deduplicated in the audio and video engine of the client.

[0174] In the embodiment of the present application, the client and the signaling background cooperate to manage the double-link. When the user turns on or off the network, the client notifies the signaling background to update the double-link state (create link, activate link, deactivate link, link data timeout trigger redirection, etc.).

[0175] In the embodiments of the present application, the client uplink data is recovered from packet loss in the uplink engine of the uplink engine server. The first way is to recover by using the redundant packets added by the client. The second way is to initiate an automatic repeat request (ARQ) to the client after detecting packet loss. The third way is multi-link packet deduplication. If the user uses uplink dual links, the data of the 4G link and the Wi-Fi link will be converged at the uplink engine and deduplicated by the uplink engine. Even if the data of one of the links is lost due to network problems, it can be recovered from the data packets transmitted by the other link. The packet loss recovery of the uplink engine ensures the transmission quality of the media data uplink.

[0176] In some embodiments of the present application, the system further comprises a downlink engine server 206, specifically comprising:

[0177] The routing server 202 is configured to perform transition processing on the target session audio to obtain processed target session audio, and transmit the processed target session audio to the downlink engine server 206.

[0178] The downlink engine server 206 is configured to receive the processed target session audio sent by the routing server 202, and transmit the processed target session audio to the client 203.

[0179] In some embodiments of the present application, the system further comprises at least two downlink transparent servers 207, specifically comprising:

[0180] The downlink engine server 206 is configured to transmit the processed target session audio to the at least two downlink transparent servers 207 respectively.

[0181] The at least two downlink transparent servers 207 are configured to transmit the processed target session audio to the client 203 through each downlink transmission link respectively.

[0182] Among the at least two downlink transparent servers 207 corresponding to the same downlink engine server 206, the downlink transmission links connected by different downlink transparent servers 207 are of different transmission types.

[0183] Correspondingly, please refer to Figure 7 , Figure 7 is a flowchart of audio data sent from the routing server to the client in the embodiments of the present application. In the embodiments of the present application, the flow of audio data sent from the routing server to the client includes:

[0184] 231、Routing server selects several key streams from several data streams in the call according to the energy information of the session audio, and forwards the data packets to the downlink engine server of the client of the call (all other users) (such as the data packets of user A are not forwarded to the downlink engine server of A);

[0185] 232、The downlink engine server of the client of the call receives the data packets of other users in the call, adds redundant packets to the downlink data according to the transmission quality of the downlink transmission link of the client of the call (the transmission quality is poor, and the redundant ratio is high), and forwards the data packets and the redundant packets to the two downlink transparent servers of the client of the call;

[0186] 233、The downlink transparent server of the client of the call performs data transparent transmission, and sends the two media data to the client of the call;

[0187] 234、The client of the call receives the two media data, performs packet loss recovery and double-link data deduplication, and plays the media data.

[0188] In the embodiments of the present application, the sender client and the listener client are hardware devices for user participation in audio and video calls, and run call applications carrying audio and video engines. The terminal has media data collection capability, sending capability, receiving capability and playing capability, and the audio and video engine has specific media data processing capability. At the same time, the terminal can select to use a single link or double links to send and receive media data according to the number of active network cards.

[0189] In the embodiments of the present application, the uplink engine server or the downlink engine server is used to deploy a server similar to the client audio and video engine, so that the data server has the capability of processing media data. The uplink engine server is used to process the media data transmitted by the uplink transparent server, and mainly performs packet loss recovery and double-link data deduplication. The downlink engine server is used to process the media data to be sent to the downlink transparent server, and is mainly used to add redundant data packets (FEC, forward error correction) to the downlink media data, so as to facilitate the listener client to perform packet loss recovery.

[0190] In the embodiments of the present application, the uplink transparent server or the downlink transparent server is used to transparently transmit media data, and plays a role of a portal in the end-to-end media data communication between the data server and the user terminal. The uplink transparent server or the downlink transparent server is also called an access server, and the selection of the access server is closely related to the information such as the region where the user is located and the type of operator. Selecting the optimal access point is of great help to improve the communication quality.

[0191] In the embodiments of the present application, the server that selects several key streams from several data streams in a call according to certain conditions and forwards the key streams is a final convergence point of all media data in the call. The routing criteria are not unique and different schemes can be formulated according to different requirements, for example, the conference scenario can be filtered according to the energy of the sound.

[0192] In some embodiments of the present application, the system further includes at least a transmission control server 208, specifically including:

[0193] The uplink engine server 204 is configured to generate an uplink transmission quality result corresponding to the sent session audio and send the uplink transmission quality result to the transmission control server 208.

[0194] The receiving client 203 is configured to generate a downlink transmission quality result of the received target session audio and send the downlink transmission quality result to the transmission control server 208.

[0195] The transmission control server 208 is configured to adjust the sending strategy of the sending client 201 for sending the original session audio according to the uplink transmission quality result and adjust the sending strategy of the downlink engine server 206 for sending the target session audio according to the downlink transmission quality result.

[0196] In the embodiments of the present application, the transmission control server is a server that can adjust the uplink and downlink data transmission strategy in real time. The principle of the adjustment is to adjust the sending strategy according to the receiving quality. In the embodiments of the present application, the sending strategy includes the control strategy for the transmission rate and the data size of a single transmission in the process of audio data transmission.

[0197] In the embodiments of the present application, the transmission control server is a server that can adjust the uplink and downlink data transmission strategy in real time. The principle of the adjustment is to adjust the sending strategy according to the receiving quality. In the embodiments of the present application, the sending strategy includes the control strategy for the transmission rate and the data size of a single transmission in the process of audio data transmission. Figure 4 The uplink engine server is configured to synchronize the uplink transmission quality of the user to the transmission control server through the point-to-point communication technology between the uplink engine server and the transmission control server, the transmission control server formulates an uplink transmission control strategy according to the uplink transmission quality information of the user, and synchronizes the uplink transmission control strategy to the sending client through the point-to-point communication technology between the transmission control server and the sending client, the sending client adjusts the sending strategy of the original audio data according to the uplink transmission control strategy; and the receiving client is configured to synchronize the downlink transmission quality of the user to the transmission control server through the point-to-point communication technology between the receiving client and the transmission control server, the transmission control server formulates a downlink transmission control strategy according to the downlink transmission quality information, and synchronizes the downlink transmission control strategy to the downlink engine server through the point-to-point communication technology between the transmission control server and the downlink engine server, the downlink engine server adjusts the sending strategy of the target session audio according to the downlink transmission control strategy.

[0198] In the embodiments of the present application, the role of the downlink engine server is to process the routed data packets, facilitating the client to recover the lost packets. At this time, the downlink engine server acts as the data sending end, and there are two ways that can be used as the anti-packet loss strategy. The first way is forward error correction (FEC). In the architecture introduced in the present application, the regulation of downlink redundancy is user-based, that is, the downlink engine of each user sets the redundancy information for the user individually. As shown in Figure 8 Figure 8 is a transmission control process diagram provided by the embodiments of the present application for audio data sent from the routing server to the client of the listener. In one call, there are four users A, B, C and D. When the downlink engines of users B, C and D receive the data packets of user A sent by the routing server, the basis for setting the redundancy information mainly comes from the decision of the transmission control server. The transmission quality of each user's downlink transmission link is different, and the redundancy information is also different. Taking user B as an example, the regulation-transmission-regulation of the downlink forms a closed loop, so that the targeted redundancy packet can maximize the guarantee of normal recovery of data at the client. The second way is to respond to the automatic retransmission request of the client.

[0199] Among the above packet loss recovery strategies of the engine server, the transmission reliability is improved, and the improvement of the call quality is also significant.

[0200] Among the above packet loss recovery strategies of the engine server, the transmission reliability is improved, and the improvement of the call quality is also significant.

[0201] Among the above packet loss recovery strategies of the engine server, the transmission reliability is improved, and the improvement of the call quality is also significant.

[0202] Among the above packet loss recovery strategies of the engine server, the transmission reliability is improved, and the improvement of the call quality is also significant.

[0203] ​The sending strategy of the session audio in the transmission process is adjusted through the setting of the transmission control server, control of the transmission process is realized, and transmission quality is improved.

[0204] To better implement the audio transmission method of the present application, the present application further provides an audio transmission device based on the above audio transmission method. The meaning of the third target word is the same as in the above audio transmission method, and the specific implementation details can be referred to the description in the method embodiment.

[0205] Please refer to Figure 9 , Figure 9 The structure diagram of the audio transmission device provided by the present application is shown in the figure, wherein the audio transmission device can include:

[0206] The acquisition module 301 is configured to acquire at least two paths of session audio from different sender clients.

[0207] The determination module 302 is configured to determine a corresponding target filtering strategy according to a current session scenario.

[0208] The routing module 303 is configured to filter out target session audio from the at least two paths of session audio according to the target filtering strategy.

[0209] The sending module 304 is configured to send the target session audio to a listener client.

[0210] In some embodiments of the present application, the sending module 304 includes:

[0211] The first determination unit is configured to determine a target audio component from at least two audio components if the target session audio includes the at least two audio components of different users.

[0212] The processing unit is configured to perform transition processing on the target audio component to obtain processed target session audio.

[0213] The first sending unit is configured to send the processed target session audio to the listener client.

[0214] In some embodiments of the present application, the processing unit includes:

[0215] The first processing subunit is configured to extract the target audio component from the at least two audio components and construct session audio according to the target audio component to obtain processed target session audio.

[0216] In some embodiments of the present application, the processing unit includes:

[0217] The second processing subunit is configured to perform enhancement processing on the target audio component in the target session audio to obtain processed target session audio.

[0218] In some embodiments of this application, the processing unit includes:

[0219] The third processing subunit is used to determine the non-target audio components from the target conversation audio based on the target audio components, and to suppress the non-target audio components to obtain the processed target conversation audio.

[0220] In some embodiments of this application, the first determining unit includes:

[0221] A determination subunit is used to determine the target audio component from at least two audio components based on the energy information of the audio components.

[0222] In some embodiments of this application, the determined subunit is used for:

[0223] Determine the user identity tag information corresponding to the audio components;

[0224] Retrieve audio data transmission records;

[0225] Determine the audio transmission frequency corresponding to the user's identity tag information based on the audio data transmission records;

[0226] Based on energy information and audio transmission frequency, the target audio component is determined from at least two audio components.

[0227] In some embodiments of this application, the determined subunit is used for:

[0228] The audio transmission level of the audio components is determined based on the user's identity tag information;

[0229] Based on energy information, audio transmission frequency, and audio transmission level, the target audio component is determined from at least two audio components.

[0230] In some embodiments of this application, the determining module 302 includes:

[0231] The second determining unit is used to determine the target filtering strategy corresponding to the meeting scenario when the current session scenario is a meeting scenario, based on the audio energy of the filtered session audio meeting the preset conditions.

[0232] In some embodiments of this application, the acquisition module 301 includes:

[0233] The acquisition unit is used to acquire at least two sessions of audio from at least two uplink engine servers;

[0234] The session audio is obtained by the uplink engine server from original session audios from at least two uplink transparent servers, and the original session audios are obtained by the at least two uplink transparent servers from the sender client through uplink transmission links respectively.

[0235] The modification processing includes a convergence operation and a deduplication operation on the original session audios transmitted by the at least two uplink transmission links.

[0236] In some embodiments of the present application, the sending module 304 includes:

[0237] The second sending unit is configured to send the target session audio to the receiver client through the downlink engine server.

[0238] The target session audio is sent to the at least two downlink transparent servers through the downlink engine server, and is sent to the receiver client through the downlink transmission link by the downlink transparent server.

[0239] The embodiments of the present application obtain at least two session audios from different sender clients by the obtaining module 301, then determine the corresponding target filtering strategy according to the current session scenario by the determining module 302, then select the target session audio from the at least two session audios according to the target filtering strategy by the routing module 303, and finally send the target session audio to the receiver client by the sending module 304.

[0240] In some embodiments of the present application, the target session audio is obtained by the obtaining module 301 from the multiple session audios sent by different sender clients, the corresponding target filtering strategy is determined according to the current session scenario, the target session audio is selected from the multiple session audios according to the target filtering strategy, and the selected target session audio is sent to the receiver client, so as to realize the filtering and transmission of the session audio. The filtering strategy of the session audio is determined according to the session scenario, and the session audio is filtered based on the filtering strategy, so as to realize the filtering of the session audio based on the session scenario. The selected target session audio is sent to the receiver client, so as to realize the transmission of the selected target session audio. Since the number of the selected session audio is less than the total number of the received session audio before the filtering, the data amount of the session audio sent to the receiver client is reduced, the pressure on the load and bandwidth in the transmission process of the session audio is reduced, the transmission quality of the session audio is improved, the quality of the voice call is improved, and the user experience is improved.

[0241] In addition, the present application also provides an electronic device, as shown in Figure 10 The structure of the electronic device is shown in the structure diagram of the electronic device.

[0242] The electronic device can include a processor 401 having one or more processing cores, a memory 402 having one or more computer-readable storage media, a power supply 403, and an input unit 404, among other components. Those skilled in the art can understand that Figure 10 The electronic device structure shown in the figure is not a limitation on the electronic device, and can include more or fewer components than shown, or combine certain components, or different component arrangements. Among them:

[0243] The processor 401 is the control center of the electronic device, and connects various parts of the entire electronic device through various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402. Optionally, the processor 401 can include one or more processing cores; preferably, the processor 401 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the object interface, and the application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 401.

[0244] The memory 402 can be used to store software programs and modules, and the processor 401 executes various function applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 402 can also include a memory controller to provide access for the processor 401 to the memory 402.

[0245] The electronic device also includes a power supply 403 for powering various components, and preferably the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 can also include one or more direct current or alternating current power supplies, a recharging system, a power supply failure detection circuit, a power supply converter or inverter, a power supply state indicator, and any other components.

[0246] The electronic device can also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to object settings and function control.

[0247] Although not shown, the electronic device can further include a display unit, etc., which will not be described herein. Specifically, in the present embodiment, the processor 401 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and run the application program stored in the memory 402 by the processor 401, thereby implementing the steps in any of the audio transmission methods provided in the present application.

[0248] The embodiments of the present application achieve the filtering and transmission of the conversation audio by obtaining the multi-channel conversation audio sent from different sender clients, determining the corresponding target filtering strategy according to the current conversation scene, filtering the target conversation audio from the multi-channel conversation audio according to the target filtering strategy, and sending the filtered target conversation audio to the receiver client. Among them, the filtering strategy corresponding to the conversation audio is determined according to the conversation scene, and the filtering of the conversation audio is based on the filtering strategy, which realizes the filtering of the conversation audio based on the conversation scene. Among them, the transmission of the filtered target conversation audio is realized by sending the filtered target conversation audio to the receiver client. Among them, since the number of filtered conversation audios is less than the total number of received conversation audios before filtering, the data amount of the conversation audio sent to the receiver client is reduced, the pressure on the load and bandwidth in the transmission process of the conversation audio is reduced, the transmission quality of the conversation audio is improved, the quality of the voice call is improved, and the user experience is improved.

[0249] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0250] To this end, the present application provides a computer readable storage medium, which stores a computer program capable of being loaded by a processor to execute the steps in any of the audio transmission methods provided in the present application.

[0251] Among them, the computer readable storage medium can include read only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0252] Since the instructions stored in the computer readable storage medium can execute the steps in any of the audio transmission methods provided in the present application, the beneficial effects that can be achieved by any of the audio transmission methods provided in the present application can be achieved, which are described in detail in the above embodiments and will not be described herein.

[0253] The audio transmission method, system and device provided by the present application are described in detail above, and the principles and implementation modes of the present application are described by using specific examples. The above example is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation modes and application ranges will be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

[0254] Among them, it can be understood that in the specific embodiments of the present application, data related to user information, user identity tag information, audio information after user voice, application use data and transmission quality of user terminal equipment are involved. When the above embodiments of the present application are applied to specific products or technologies, the user's permission or consent is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. An audio transmission method, characterized by, The method comprises the following steps: acquiring at least two pieces of conversation audio from different sender clients; wherein each piece of conversation audio corresponds to conversation data sent by a sender client, and the conversation audio from at least one sender client comprises audio components of sounds uttered by multiple users, and the audio components are audio data formed after the users utter independently; determining a target filtering strategy corresponding to a current conversation scenario; filtering target conversation audio from the at least two pieces of conversation audio according to the target filtering strategy, sending the target conversation audio to a receiver client; wherein the step of sending the target conversation audio to the receiver client comprises the following steps: if the target conversation audio comprises at least two audio components of different users, determining a target audio component from the at least two audio components according to energy information of each audio component and an audio transmission frequency; the energy information refers to energy information of sound corresponding to the audio component, and the audio transmission frequency represents an utterance frequency of a user corresponding to the audio component; performing transition processing on the target audio component to obtain processed target conversation audio, and the transition processing is used to highlight the influence degree of the target audio component in the target conversation audio; sending the processed target conversation audio to the receiver client.

2. The method of claim 1, wherein, The step of performing transition processing on the target audio component to obtain processed target conversation audio comprises the following steps: extracting the target audio component from the at least two audio components, and constructing conversation audio according to the target audio component to obtain the processed target conversation audio, or performing enhancement processing on the target audio component in the target conversation audio to obtain the processed target conversation audio, or determining a non-target audio component from the target conversation audio according to the target audio component, and performing suppression processing on the non-target audio component to obtain the processed target conversation audio.

3. The method of claim 1, wherein, The step of determining a target audio component from the at least two audio components according to energy information of each audio component and an audio transmission frequency comprises the following steps: determining user identity tag information corresponding to the audio component; acquiring an audio data transmission record; determining the audio transmission frequency corresponding to the user identity tag information according to the audio data transmission record; determining the target audio component from the at least two audio components according to the energy information of the audio component and the audio transmission frequency.

4. The method of claim 1, wherein, The step of determining a target filtering strategy corresponding to a current conversation scenario comprises the following step: when the current conversation scenario is a conference scenario, taking that the audio energy of the filtered conversation audio satisfies a preset condition as the target filtering strategy corresponding to the conference scenario.

5. The method of claim 1, wherein, The step of acquiring at least two pieces of conversation audio from different sender clients comprises the following step: acquiring the at least two pieces of conversation audio from at least two uplink engine servers. The session audio is obtained by the uplink engine server from original session audios from at least two uplink transparent servers after modification processing, and the original session audios are obtained by the at least two uplink transparent servers from the sender client through uplink transmission links. The modification processing includes converging operation and de-duplication operation on the original session audios transmitted through the at least two uplink transmission links.

6. The method of claim 1, wherein, The target session audio is transmitted to the receiver client through the downlink engine server. The target session audio is transmitted to the receiver client through the downlink engine server. The audio transmission system comprises a sender client, a routing server and a receiver client, wherein 7. An audio transmission system characterized by, The sender client is configured to transmit session audio to the routing server. The routing server is configured to determine a target filtering strategy according to a current session scenario, filter a target session audio from at least two session audios from different sender clients according to the target filtering strategy, and transmit the target session audio to the receiver client. The receiver client is configured to receive the target session audio transmitted by the routing server. The routing server is configured to determine a target audio component from at least two audio components of different users in the target session audio according to energy information of each audio component and audio transmission frequency of the target audio component, perform transition processing on the target audio component to obtain processed target session audio, and transmit the processed target session audio to the receiver client. The audio transmission system further comprises an uplink engine server, wherein 8. The audio transmission system of claim 7, wherein, The sender client is configured to transmit original session audio to the uplink engine server. The uplink engine server is configured to receive the original session audio transmitted by the sender client, perform modification processing on the original session audio to obtain the session audio, and transmit the session audio to the routing server. The routing server is configured to receive the session audio transmitted by the uplink engine server. The audio transmission system further comprises at least two uplink transparent servers, wherein 9. The audio transmission system of claim 8, wherein, The sender client is configured to transmit the original session audio to each uplink transparent server through each uplink transmission link. ​ The uplink transparent server is configured to transmit the obtained at least two original conversation audios to the uplink engine server; The uplink engine server is configured to perform correction processing according to the received at least two original conversation audios to obtain the conversation audio, and transmit the conversation audio to the routing server; In the at least two uplink transparent servers corresponding to the same uplink engine server, the uplink transmission links connected by different uplink transparent servers are of different transmission types.

10. The audio transmission system of claim 9, wherein, The audio transmission system further comprises a downlink engine server, wherein The routing server is configured to transmit the processed target conversation audio to the downlink engine server; The downlink engine server is configured to receive the processed target conversation audio sent by the routing server, and transmit the processed target conversation audio to the callee client.

11. The audio transmission system of claim 10, wherein, The audio transmission system further comprises at least two downlink transparent servers, wherein The downlink engine server is configured to transmit the processed target conversation audio to the at least two downlink transparent servers respectively; The at least two downlink transparent servers are configured to transmit the processed target conversation audio to the callee client through each downlink transmission link respectively; In the at least two downlink transparent servers corresponding to the same downlink engine server, the downlink transmission links connected by different downlink transparent servers are of different transmission types.

12. The audio transmission system of claim 11, wherein, The audio transmission system further comprises a transmission control server, wherein The uplink engine server is configured to generate an uplink transmission quality result corresponding to the transmission of the conversation audio, and send the uplink transmission quality result to the transmission control server; The callee client is configured to generate a downlink transmission quality result corresponding to the reception of the target conversation audio, and send the downlink transmission quality result to the transmission control server; The transmission control server is configured to adjust the transmission strategy of the sender client for transmitting the original conversation audio according to the uplink transmission quality result, and adjust the transmission strategy of the downlink engine server for transmitting the target conversation audio according to the downlink transmission quality result.

13. An audio transmission device, characterized by The audio transmission system comprises: An acquisition module configured to acquire at least two conversation audios from different sender clients; wherein each conversation audio corresponds to conversation data transmitted by a sender client, and the conversation audio from at least one sender client contains audio components formed by independent voice of a user; A determination module configured to determine a target filtering strategy according to a current conversation scenario; A routing module configured to filter a target conversation audio from the at least two conversation audios according to the target filtering strategy; A sending module configured to send the target conversation audio to a callee client; and An acquisition module configured to acquire at least two conversation audios from different sender clients; wherein each conversation audio corresponds to conversation data transmitted by a sender client, and the conversation audio from at least one sender client contains audio components formed by independent voice of a user; A determination module configured to determine a target filtering strategy according to a current conversation scenario; A routing module configured to filter a target conversation audio from the at least two conversation audios according to the target filtering strategy; A sending module configured to send the target conversation audio to a callee client; and The sending module is configured to: if the target conversation audio includes at least two audio components of different users, determine a target audio component from the at least two audio components according to energy information of each audio component and an audio transmission frequency, wherein the energy information indicates energy information of sound corresponding to the audio component, and the audio transmission frequency indicates a sound production frequency of a user corresponding to the audio component; perform transition processing on the target audio component to obtain processed target conversation audio, wherein the transition processing is configured to highlight an influence degree of the target audio component in the target conversation audio; and send the processed target conversation audio to the called party client.

14. An electronic device, comprising: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the audio transmission method according to any one of claims 1-6 when executing the computer program.

15. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and executable on the processor, and the processor implements the steps in the audio transmission method according to any one of claims 1-6 when executing the computer program.

Images