Multi-person voice communication method and related equipment

Abstract

The embodiment of the invention discloses a multi-person voice communication method and related equipment. According to the embodiment of the invention, after the current audio selection parameter and the network packet loss parameter of the target receiving terminal in the current voice call are obtained, the current audio selection parameter is adjusted according to the network packet loss parameter to obtain the target audio selection parameter, and when the current audio data sent by the plurality of sending terminals are received, the target audio selection parameter is obtained. And obtaining audio features of the current audio data, then screening out at least one piece of target audio data corresponding to the target receiving terminal from the current audio data based on the audio features and the target audio selection parameters, and then sending the target audio data to the target receiving terminal. The current voice communication between the target receiving terminal and the sending terminal is ensured; according to the scheme, the call quality of the multi-person voice call can be improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a multi-person voice call method and related equipment. Background Art

[0002] In recent years, with the rapid development of Internet technology, voice calls have become more and more convenient, especially multi-person voice calls. Multi-person voice calls involve multiple parties collecting audio signals and performing various audio processing through different devices (terminals). Since the human ear has limited ability to recognize simultaneous sounds from different sound sources, it is necessary to screen the audio data (signals) of different participants to ensure the call quality in multi-person voice calls. Existing multi-person voice calls often screen audio data based on the voice features and preset selection parameters of each audio data, and send the screened target audio data to the receiving terminal, thereby realizing multi-person voice calls.

[0003] In the process of research and practice of the prior art, the inventor of the present application found that for the existing multi-person voice call mode, the network environment of different receiving terminals in the multi-person voice call may be different, and the network environment of the same receiving terminal in different time periods may also be different. The preset selection parameters may have the risk of not being able to match the network environment, causing the participants in the multi-person voice call to experience audio lag or failure to interrupt the call, thereby resulting in low call quality. Summary of the invention

[0004] The embodiment of the present invention provides a multi-person voice call method and related equipment. The related equipment may include a multi-person voice call device, an electronic device and a computer-readable storage medium, which can improve the call quality of multi-person voice calls.

[0005] A multi-person voice call method, comprising:

[0006] Obtain the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call;

[0007] According to the network packet loss parameter, the current audio selection parameter is adjusted to obtain a target audio selection parameter;

[0008] When receiving current audio data sent by multiple sending terminals, obtaining audio features of the current audio data;

[0009] Based on the audio feature and the target audio selection parameter, screening out at least one target audio data corresponding to the target receiving terminal from the current audio data;

[0010] The target audio data is sent to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0011] Optionally, the embodiment of the present application may also provide a multi-person voice call method, including:

[0012] Collect the audio signal in the current voice call to obtain the current audio data;

[0013] Identifying a speech energy value corresponding to the audio signal in the current audio data;

[0014] Predicting a noise energy value in the current audio data, and using the noise energy value and the speech energy value as audio features of the current audio data;

[0015] The current audio data and the audio features are sent to a server, so that the server determines the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal.

[0016] Accordingly, an embodiment of the present invention provides a multi-person voice call device, including:

[0017] A first acquisition unit, configured to acquire a current audio selection parameter and a network packet loss parameter of a target receiving terminal in a current voice call;

[0018] an adjusting unit, configured to adjust the current audio selection parameter according to the network packet loss parameter to obtain a target audio selection parameter;

[0019] A second acquisition unit, configured to acquire audio features of the current audio data when receiving current audio data sent by multiple sending terminals;

[0020] a screening unit, configured to screen out at least one target audio data corresponding to the target receiving terminal from the current audio data based on the audio feature and the target audio selection parameter;

[0021] The sending unit is used to send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0022] Optionally, the embodiment of the present application may also provide another multi-person voice call device, including:

[0023] A collection unit, used to collect the audio signal in the current voice call to obtain the current audio data;

[0024] an identification unit, configured to identify a speech energy value corresponding to the audio signal in the current audio data;

[0025] A prediction unit, configured to predict a noise energy value in the current audio data, and use the noise energy value and the speech energy value as audio features of the current audio data;

[0026] The call unit is used to send the current audio data and audio features to the server, so that the server determines the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal.

[0027] Optionally, in some embodiments, the adjustment unit may be specifically configured to compare the network packet loss parameter with a preset packet loss parameter threshold; and adjust the current audio selection parameter according to the comparison result to obtain a target audio selection parameter.

[0028] Optionally, in some embodiments, the adjustment unit can be specifically used to adjust the current audio selection parameter to obtain the target audio selection parameter when the network packet loss parameter is greater than a preset packet loss parameter threshold; when the network packet loss parameter is less than or equal to the preset packet loss parameter threshold, replace the current audio selection parameter with the preset initial audio selection parameter to obtain the target audio selection parameter.

[0029] Optionally, in some embodiments, the adjustment unit can be specifically used to generate a candidate audio selection parameter corresponding to the network packet loss parameter based on a preset parameter generation function, the candidate audio selection parameter including a candidate audio quantity threshold and a candidate noise energy threshold; replace the current audio quantity threshold with the candidate audio quantity threshold to obtain a target audio quantity threshold; replace the current noise energy threshold with the candidate noise energy threshold to obtain a target noise energy threshold, and use the target audio quantity threshold and the target noise energy threshold as target audio selection parameters.

[0030] Optionally, in some embodiments, the adjustment unit can be specifically used to adopt the audio quantity generation function to generate a candidate audio quantity threshold corresponding to the network packet loss parameter, and the audio quantity generation function is a monotonically decreasing function; adopt the noise energy generation function to generate a candidate noise energy threshold corresponding to the network packet loss parameter, and the noise energy generation function is a monotonically increasing function; use the candidate audio quantity threshold and the candidate noise energy threshold as the candidate audio selection parameters.

[0031] Optionally, in some embodiments, the adjustment unit can be specifically used to obtain a preset threshold range corresponding to the candidate audio quantity threshold; when the candidate audio quantity threshold is within the preset threshold range, the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain a target audio quantity threshold; when the candidate audio quantity threshold is outside the preset threshold range, a basic audio quantity threshold is screened out within the preset threshold range, the basic audio quantity threshold is used as the candidate audio quantity threshold, and the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold.

[0032] Optionally, in some embodiments, the screening unit can be specifically used to screen out at least one audio data whose noise energy value is less than or equal to the target noise energy threshold from the current audio data, to obtain candidate audio data corresponding to the target receiving terminal; based on the speech energy value, sort the candidate audio data corresponding to the target receiving terminal; and according to the sorting result and the target audio quantity threshold, screen out at least one audio data from the candidate audio data to obtain the target audio data.

[0033] Optionally, in some embodiments, the screening unit can be specifically used to determine a target sorting range corresponding to the candidate audio data based on the target audio quantity threshold; and according to the sorting result, screen out at least one audio data corresponding to the target sorting range from the candidate audio data to obtain the target audio data.

[0034] Optionally, in some embodiments, the first acquisition unit can be specifically used to obtain the current audio selection parameter of the target receiving terminal in the current voice call; receive the current network packet loss parameter of the current period corresponding to the current audio selection parameter returned by the target receiving terminal or the predicted network packet loss parameter of the next period of the current period; and use the current network packet loss parameter or the predicted network packet loss parameter as the network packet loss parameter of the target receiving terminal.

[0035] Optionally, in some embodiments, the first sending unit can be specifically used to, when receiving the updated network packet loss parameter returned by the target receiving terminal based on the target audio data, use the updated network packet loss parameter as the network packet loss parameter, and use the target audio selection parameter as the current audio selection parameter; return to execute the step of adjusting the current audio selection parameter according to the network packet loss parameter until the current voice call ends.

[0036] Optionally, in some embodiments, the recognition unit can be specifically used to perform frequency domain transformation on the current audio data to obtain power spectrum values ​​of frequency points in each frame of audio; fuse the power spectrum values ​​to obtain initial speech energy values ​​corresponding to each frame of audio; and smooth the initial speech energy values ​​to obtain speech energy values ​​corresponding to the audio signal.

[0037] Optionally, in some embodiments, the identification unit can be specifically used to divide the current audio data into frames to obtain time domain signal data corresponding to each frame of audio; perform frequency domain conversion on the time domain signal data to obtain frequency domain signal data corresponding to each frame of audio; and determine the power spectrum value of the frequency point in each frame of audio based on the frequency domain signal data.

[0038] Optionally, in some embodiments, the prediction unit can be specifically used to compare the power spectrum values ​​to obtain the initial noise power spectrum value of the frequency point; calculate the probability of speech existence at the frequency point based on the initial noise power spectrum value; and fuse the speech existence probability with the power spectrum value to obtain the noise energy value in the current audio data.

[0039] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the multi-person voice call method provided by an embodiment of the present invention.

[0040] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the steps in any one of the multi-person voice call methods provided in the embodiments of the present invention.

[0041] In addition, an embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps in the multi-person voice call method provided in an embodiment of the present application.

[0042] After obtaining the current audio selection parameter and network packet loss parameter of the target receiving terminal in the current voice call, the embodiment of the present invention adjusts the current audio selection parameter according to the network packet loss parameter to obtain the target audio selection parameter. When receiving the current audio data sent by multiple sending terminals, the audio feature of the current audio data is obtained. Then, based on the audio feature and the target audio selection parameter, at least one target audio data corresponding to the target receiving terminal is screened out in the current audio data, and then the target audio data is sent to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal. Since the scheme can adjust the audio selection parameter by the network packet loss parameter that characterizes the transmission quality of the receiver, the audio selection parameter can be adaptively adjusted or regulated in different network environments, thereby adapting to different network environments and reducing the occurrence of call freezes or failures in call grabbing. Therefore, the call quality of multi-person voice calls can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 is a system schematic diagram of a multi-person voice call system provided by an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of a scenario of a multi-person voice call method provided by an embodiment of the present invention;

[0046] Figure 3 It is a flowchart of a multi-person voice call method provided by an embodiment of the present invention;

[0047] Figure 4 is another schematic diagram of a multi-person voice call process provided by an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of a page for a multi-person voice call provided by an embodiment of the present invention;

[0049] Figure 6 is another schematic diagram of a multi-person voice call process provided by an embodiment of the present invention;

[0050] Figure 7 This is a schematic diagram of the overall process of a multi-person voice call provided by an embodiment of the present invention;

[0051] Figure 8 is a schematic structural diagram of a first multi-person voice communication device provided by an embodiment of the present invention;

[0052] Fig. 9 is a schematic diagram of the structure of a second multi-person voice communication device provided by an embodiment of the present invention;

[0053] Fig.10 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] The embodiment of the present invention provides a multi-person voice call method and related equipment, and the related equipment may include a multi-person voice call device, an electronic device and a computer-readable storage medium. Among them, the multi-person voice call can be integrated in an electronic device, and the electronic device can be a server or a terminal. Specifically, the embodiment of the present invention provides a multi-person voice call device applicable to a first electronic device (in order to distinguish, it can be called a first multi-person voice call device), and a multi-person voice call device applicable to a second electronic device (in order to distinguish, it can be called a second voice call device). Among them, the first electronic device can be a network-side device such as a server, and the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The second electronic device can be a terminal and other devices, and the terminal includes but is not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application is not limited here. The embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, cloud security, artificial intelligence, smart transportation, assisted driving, etc.

[0056] In an embodiment of the present invention, a method for multi-person voice call is introduced by taking the first electronic device as a server and the second electronic device as a terminal, where the terminal may include a sending terminal and a receiving terminal. The sending terminal may include a terminal for sending audio data in a multi-person voice call, and the receiving terminal may include a terminal for receiving audio data in a multi-person voice call.

[0057] For example, see Figure 1 An embodiment of the present invention provides a multi-person voice call system including a server 10, a sending terminal 20 and a target receiving terminal 30; the sending terminal 20 and the target receiving terminal 30 are connected to the receiving server 10 through a network, for example, through a wired or wireless network connection, etc., wherein the multi-person voice call device can be integrated in the receiving terminal or the sending terminal, for example, integrated in the receiving terminal and the sending terminal in the form of a client.

[0058] Among them, the server 10 can be used to obtain the current audio selection parameters and network packet loss parameters of the target receiving terminal 30 in the current voice call, and then adjust the current audio selection parameters according to the network packet loss parameters to obtain the target audio selection parameters. When receiving the current audio data sent by multiple sending terminals 20, the audio features of the current audio data are obtained, and then, based on the audio features and the target audio selection parameters, at least one target audio data corresponding to the target receiving terminal 30 is screened out from the current audio data, and then, the target audio data is sent to the target receiving terminal 30 to ensure the current voice call between the target receiving terminal 30 and the sending terminal 20, thereby improving the call quality of multi-person voice calls. Specifically, it can be as follows Figure 2 shown.

[0059] The sending terminal 20 may send the collected current audio data and the audio features of the current audio data to the server 10, so that the server 10 can filter out the target audio data from the current audio data and send it to the target receiving terminal 30, so as to ensure the current voice call between the target receiving terminal 30 and the sending terminal 20. There are many ways for the sending terminal 20 to send the collected current audio data and audio features to the server 10, and the sending terminal 20 may specifically:

[0060] Collect the voice signal in the current voice call to obtain the current audio data, identify the voice energy value corresponding to the voice signal in the current audio data, predict the noise energy value in the current audio data, and use the noise energy value and the voice energy value as the audio features of the current audio data, and send the current audio data and the audio features to the server 10, so that the server 10 determines the target audio data corresponding to the target receiving terminal 30 based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal 30, thereby improving the call quality of multi-person voice calls.

[0061] It can be understood that in the specific implementation of the present application, when the following embodiments of the present application are applied to specific products or technologies, relevant data such as the voice signal or current audio data of the object need to be licensed or consented, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0062] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0063] In this embodiment, it will be described from the perspective of a first multi-person voice call device. The first multi-person voice call device can be specifically integrated in a first electronic device, with the first electronic device as a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0064] A multi-person voice call method, comprising:

[0065] Obtain the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call, adjust the current audio selection parameters according to the network packet loss parameters, and obtain the target audio selection parameters. When receiving current audio data sent by multiple sending terminals, obtain the audio features of the current audio data, and based on the audio features and the target audio selection parameters, filter out at least one target audio data corresponding to the target receiving terminal in the current audio data, and send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0066] like Figure 3 As shown, the specific process of the multi-person voice call method is as follows:

[0067] 101. Obtain current audio selection parameters and network packet loss parameters of a target receiving terminal in a current voice call.

[0068] Among them, the current audio selection parameter can be the audio selection parameter of the target receiving terminal before the current audio data sent by the receiving and sending terminal. The current audio selection parameter can be a pre-set audio selection parameter, or it can be an audio selection parameter corresponding to when receiving historical audio data. The so-called audio selection parameter can be understood as the parameter of the server selecting the target audio data corresponding to the target receiving terminal from multiple audio data in a multi-person voice call for the target receiving terminal. The audio selection parameter may include a noise energy threshold and an audio quantity threshold. The so-called noise energy threshold can be understood as the threshold of the noise energy in the target audio data that the target receiving terminal is allowed to receive, that is, audio data exceeding the noise energy threshold will not be received by the target receiving terminal. The so-called audio quantity threshold can be understood as the maximum number of audio data that the target receiving terminal can receive.

[0069] The network packet loss parameter may be parameter information of data packet loss of the target receiving terminal during a multi-person voice call, for example, it may include a packet loss rate, a number of consecutive packet losses, or a maximum number of packet losses, etc. The network packet loss parameter may characterize or indicate the current network environment or network transmission quality of the target receiving terminal. The higher the network packet loss parameter, that is, the higher the packet loss rate or the number of consecutive packet losses, the lower the current network environment or network transmission quality.

[0070] Among them, a multi-person voice call can usually include multiple objects participating in the call. When an object makes a sound in the voice call, the terminal corresponding to the object making the sound can be the sending terminal, and the terminals corresponding to other objects can be used as the receiving terminal, or the terminals corresponding to all objects in the multi-person voice call can be used as the receiving terminals. When there are multiple objects making sounds in a multi-person voice call at the same time, the terminal corresponding to each sending object can be a sending terminal or a receiving terminal, and the terminals corresponding to other objects that have not made a sound can be receiving terminals. The target receiving terminal can be a receiving terminal selected from the receiving terminals, and the object corresponding to the receiving terminal can be a sounding object or an unsound object.

[0071] There are multiple ways to obtain the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call, which can be as follows:

[0072] For example, you can obtain the current audio selection parameters of the target receiving terminal in the current voice call, receive the current network packet loss parameters of the current period corresponding to the current audio selection parameters returned by the target receiving terminal, or the predicted network packet loss parameters of the next period after the current period, and use the current network packet loss parameters or the predicted network packet loss parameters as the network packet loss parameters of the target receiving terminal.

[0073] Among them, there can be multiple ways to obtain the current audio selection parameters of the target receiving terminal in the current voice call. For example, when the target receiving terminal has just joined the current voice call between the sending terminal, that is, when it has not received any audio data, at this time, the preset audio selection parameters can be used as the current audio selection parameters of the target receiving terminal. The preset audio selection parameters can be audio selection parameters pre-set for the target receiving terminal, or audio selection parameters pre-set for the current voice call; when the target receiving terminal has received historical audio data, at this time, the historical audio selection parameters corresponding to the historical audio data closest to the current one can be obtained, and the historical audio selection parameters can be used as the current audio selection parameters, and so on.

[0074] After obtaining the current audio selection parameter, the current network packet loss parameter of the current cycle corresponding to the current audio selection parameter or the predicted network packet loss parameter of the next cycle of the current cycle returned by the target receiving terminal can be received. The current cycle corresponding to the current audio selection parameter can be the time period of the corresponding audio data received by the target receiving terminal through the current audio selection parameter. For example, taking the current audio selection parameter as the historical audio selection parameter as an example, the corresponding current cycle can be the statistical period when the target receiving terminal receives the historical audio data corresponding to the historical audio selection parameter; taking the current audio selection parameter as the preset audio selection parameter as an example, at this time, the target receiving terminal has not received audio data in the current voice call. In this case, test audio data or preset audio data can also be sent to the target receiving terminal, and the statistical period when the target receiving terminal receives the test audio data or predicted audio data can be the current period.

[0075] Among them, the current network packet loss parameter may be a network packet loss parameter counted by the target receiving terminal in the current period. There may be multiple ways for the target receiving terminal to count the current network packet loss parameter of the current period. For example, after receiving audio data such as historical audio data, test audio data, or preset audio data, the target receiving terminal may identify the timestamp recorded in the data packet of each frame in the audio data, determine the spacing between the timestamps of adjacent data packets, and when the spacing exceeds the preset spacing threshold, it indicates that there is data packet loss in adjacent data packets. Based on the spacing of the timestamps, the number of packet losses between adjacent data packets is determined, thereby counting the number of packet losses within a time period, and based on the number of packet losses, calculating the packet loss rate, the number of continuous packet losses, or the maximum number of packet losses, thereby obtaining the current network packet loss parameter.

[0076] The interval of the timestamps can be understood as the difference between the timestamp values ​​in the timestamps. It should be noted that the timestamp values ​​in the timestamps can increase over time.

[0077] Among them, the predicted network packet loss parameter can be the network packet loss parameter of the next cycle of the current cycle predicted by the target receiving terminal based on the current network packet loss parameter. There are many ways for the target receiving terminal to predict the predicted network packet loss parameter of the next cycle of the current cycle. For example, after the target receiving terminal has calculated the current network packet loss parameter of the current cycle, it can obtain the historical network packet loss parameter before the current cycle, and predict the network packet loss parameter of the next cycle of the current cycle based on the historical network packet loss parameter and the current network packet loss parameter, thereby obtaining the predicted network packet loss parameter, or obtain the current network attribute information, and predict the network packet loss parameter of the next cycle of the current cycle based on the current network attribute information and the current network packet loss parameter, thereby obtaining the predicted network packet loss parameter, and so on.

[0078] After the target terminal has counted the current network packet loss parameter or predicted the predicted network packet loss parameter, it can receive the current network packet loss parameter or predicted network packet loss parameter returned by the target receiving terminal and use the current network packet loss parameter or predicted network packet loss parameter as the network packet loss parameter of the target receiving terminal.

[0079] Optionally, in some embodiments, after receiving the current network packet loss parameters returned by the target receiving terminal, the historical network packet loss parameters of the target receiving terminal can be obtained, and the network packet loss parameters of the next cycle of the current cycle can be predicted based on the current network packet loss parameters and the historical network packet loss parameters, and the predicted network packet loss parameters can be used as the network packet loss parameters of the target receiving terminal. Alternatively, the current network attribute information of the target receiving terminal can be obtained, and the network packet loss parameters of the next cycle of the current cycle can be predicted based on the current network packet loss parameters and the current network attribute information, and the predicted network packet loss parameters can be used as the network packet loss parameters of the target receiving terminal, and so on.

[0080] 102. According to the network packet loss parameter, adjust the current audio selection parameter to obtain the target audio selection parameter.

[0081] For example, the network packet loss parameter may be compared with a preset packet loss parameter threshold, and the current audio selection parameter may be adjusted according to the comparison result to obtain the target audio selection parameter.

[0082] Among them, according to the comparison results, there can be multiple ways to adjust the current audio selection parameters. For example, when the network packet loss parameter is greater than the preset packet loss parameter threshold, the current audio selection parameter is adjusted to obtain the target audio selection parameter. When the network packet loss parameter is less than or equal to the preset packet loss parameter threshold, the current audio selection parameter is replaced with the preset initial audio selection parameter to obtain the target audio selection parameter.

[0083] Among them, the current audio selection parameter may include a current audio quantity threshold and a current noise energy threshold. When the network packet loss parameter is greater than the preset packet loss parameter threshold, there may be multiple ways to adjust the current audio selection parameter, for example, a candidate audio selection parameter corresponding to the network packet loss parameter may be generated based on a preset parameter generation function, the candidate audio selection parameter may include a candidate audio quantity threshold and a candidate noise energy threshold, the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain a target audio quantity threshold, the current noise energy threshold is replaced with the candidate noise energy threshold to obtain a target noise energy threshold, and the target audio quantity threshold and the target noise energy threshold are used as the target audio selection parameter.

[0084] Among them, the preset parameter generation function may include an audio quantity generation function and a noise energy generation function. The independent variable of the audio quantity generation function may be a network packet loss parameter, and the dependent variable may be an audio quantity threshold, that is, the input network packet loss parameter may generate the corresponding audio quantity threshold. The audio quantity generation function is a monotonically decreasing function, that is, as the network packet loss parameter increases, the corresponding audio quantity threshold decreases accordingly. The independent variable of the noise energy generation function may be a network packet loss parameter, and the dependent variable may be a noise energy threshold, that is, the input network packet loss parameter may generate the corresponding noise energy threshold. The noise energy generation function is a monotonically increasing function, that is, as the network packet loss parameter increases, the corresponding noise energy threshold also increases accordingly. Based on the preset parameter generation function, there may be multiple ways to generate candidate audio selection parameters corresponding to the network packet loss parameter, for example, using the audio quantity generation function to generate the candidate audio quantity threshold corresponding to the network packet loss parameter, using the noise energy generation function to generate the candidate noise energy threshold corresponding to the network packet loss parameter, and using the candidate audio quantity threshold and the candidate noise energy threshold as the candidate audio selection parameters.

[0085] After generating the candidate audio selection parameters corresponding to the network packet loss parameters, the current audio quantity threshold can be replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold. There are many ways to replace the current audio quantity threshold with the candidate audio quantity threshold. For example, a preset threshold range corresponding to the candidate audio quantity threshold can be obtained. When the candidate audio quantity threshold is within the preset threshold range, the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold. When the candidate audio quantity threshold is outside the preset threshold range, the basic audio quantity threshold is screened out within the preset threshold range, the basic audio quantity threshold is used as the candidate audio quantity threshold, and the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold.

[0086] Among them, the preset threshold range can be a value range of a preset audio quantity threshold. When the candidate audio quantity threshold is within the preset threshold range, it indicates that the candidate audio quantity threshold does not exceed or fall below the extreme value of the value range. At this time, the current audio quantity threshold can be directly replaced with the generated candidate audio quantity threshold to obtain the target audio quantity threshold. It should be noted that the target audio quantity threshold at this time is essentially the generated candidate audio quantity threshold. When the candidate audio quantity threshold is outside the preset threshold range, it indicates that the candidate audio quantity threshold exceeds or falls below the extreme value of the value range, and at this time, the basic audio quantity threshold can be screened out within the preset threshold range. There can be many ways to screen out the basic audio quantity threshold. For example, when the candidate audio quantity threshold exceeds the maximum value in the value range, the maximum value in the value range can be used as the basic audio quantity threshold, and when the candidate audio quantity threshold is less than the minimum value in the value range, the minimum value in the value range can be used as the basic audio quantity threshold.

[0087] After the basic audio quantity is screened out, the basic audio quantity threshold can be used as the candidate audio quantity threshold, and then the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold.

[0088] The method of replacing the current noise energy threshold with the candidate noise energy threshold is similar to the method of replacing the current audio quantity threshold with the candidate audio quantity threshold, as described above, and will not be described in detail here.

[0089] After the target audio quantity threshold and the target noise energy threshold are obtained, the target audio quantity threshold and the target noise energy threshold can be used as target audio selection parameters.

[0090] Among them, the preset initial audio selection parameter can be the initial value of the pre-set audio selection parameter, and the preset initial audio parameter can include the initial value of the noise energy threshold and the initial value of the audio quantity threshold. The initial value here can be the initial value set when the target receiving terminal joins the current voice call, and can also include the initial value set when the current voice call starts. When the network packet loss parameter is less than or equal to the preset packet loss parameter threshold, the current audio selection parameter can be replaced with the preset initial audio selection parameter to obtain the target audio selection parameter. The current audio selection parameter can include the current audio quantity threshold and the current noise energy threshold, then the current audio quantity threshold can be replaced with the initial value of the audio quantity threshold to obtain the target audio quantity threshold, the current noise energy threshold can be replaced with the initial value of the noise energy threshold to obtain the target noise energy threshold, and the target audio quantity threshold and the target noise energy threshold are used as the target audio selection parameter.

[0091] Among them, taking the network packet loss parameter as the packet loss rate as an example, the method of adjusting the current audio selection parameter may include: if the packet loss rate (lossrate) value exceeds the preset packet loss rate threshold (THRD_LOSS), THRD_NOISE (noise energy threshold) = func1 (lossrate), where func1 (noise energy generation function) is a monotonically increasing function. As the lossrate value increases, the THRD_NOISE value also increases accordingly. Of course, THRD_NOISE is subject to an upper limit constraint, that is, it will not exceed the preset upper limit value, and when the lossrate is equal to or lower than THR D_LOSS, the noise threshold value is the initial value, so as to obtain the target noise energy threshold; if the lossrate value exceeds THRD_LOSS, SEL_NUM (audio quantity threshold) = func2 (lossrate), where func2 (audio quantity generation function) is a monotonically decreasing function. As the lossrate value increases, the SEL_NUM value decreases accordingly. Of course, the SEL_NUM value is constrained by a lower limit, that is, it will not be lower than the preset lower limit. When the lossrate is equal to or lower than THRD_LOSS, SEL_NUM is set to the initial value, so as to obtain the target audio quantity threshold. The target noise energy threshold and the target audio quantity threshold are used as target audio selection parameters.

[0092] 103. When current audio data sent by multiple sending terminals is received, audio features of the current audio data are obtained.

[0093] The audio feature may be feature information representing voice information in the current audio data. The audio feature may include a voice energy value and a noise energy value. The voice energy value may represent the strength of the voice information contained in the current audio data. The noise energy value may represent the degree of noise contained in the current audio data. The larger the voice energy value and the lower the noise energy value, the easier it is for the current audio data to be selected and sent to the target receiving terminal in a multi-person voice call. On the contrary, the smaller the voice energy value and the higher the noise energy value, the less likely it is for the current audio data to be selected and sent to the target receiving terminal in a multi-person voice call.

[0094] When receiving current audio data sent by multiple sending terminals, there are multiple ways to obtain the audio features of the current audio data, which can be specifically as follows:

[0095] For example, the audio features corresponding to the current audio data sent by the sending terminal can be received, or the features of the current audio data can be extracted to obtain the audio features corresponding to the current audio data, or the current audio data can be sent to a third-party server so that the third-party server can extract the features of the current audio data, and the audio features corresponding to the current audio data returned by the third-party server can be received, and so on.

[0096] There are many ways to extract features from the current audio data. For example, the speech energy value corresponding to the speech signal identified in the current audio data can be used to predict the noise energy value in the current audio data, and the speech energy value and the noise energy value can be used as the audio features of the current audio data. The method of identifying the speech energy value and predicting the noise energy value can be similar to the method of identifying the speech energy value and predicting the noise energy value by the sending terminal, as described below.

[0097] 104. Based on the audio feature and the target audio selection parameter, filter out at least one target audio data corresponding to the target receiving terminal from the current audio data.

[0098] For example, at least one audio data having a noise energy value less than or equal to a target noise energy threshold can be screened out from the current audio data to obtain candidate audio data corresponding to the target receiving terminal. The candidate audio data corresponding to the target receiving terminal can be sorted based on the speech energy value. According to the sorting result and the target audio quantity threshold, at least one audio data can be screened out from the candidate audio data to obtain the target audio data.

[0099] Among them, there can be multiple ways to filter out at least one audio data from the candidate audio data based on the sorting results and the target audio quantity threshold. For example, the target sorting range corresponding to the candidate audio data can be determined based on the target audio quantity threshold, and according to the sorting results, at least one audio data corresponding to the target sorting range can be filtered out from the candidate audio data to obtain the target audio data.

[0100] Among them, the target sorting range can be the sorting range corresponding to the target audio quantity threshold. For example, if the target audio quantity is 3 and the sorting method is to sort from large to small, then the target sorting range can be the sorting range corresponding to the first 3 in the sorting result. For another example, still taking the target audio quantity as 3 as an example, if the sorting range is to sort from small to large, then the target sorting range can be the sorting range corresponding to the last 3 in the sorting result. Therefore, it can be found that after the noise energy value exceeds the preset noise energy threshold, the larger the speech energy value, the easier it is to be selected as the target audio data and sent to the target receiving terminal.

[0101] Among them, the specific process of performing audio selection on the current audio data sent by the sending terminal may include: filtering the noise energy values ​​of all current audio data (participants), and the channels corresponding to the current audio data higher than the noise energy threshold (THRD_NOISE) will not participate in the routing, and the channels that meet the noise energy threshold requirements are sorted from large to small according to the voice energy value, and the SEL_NUM channels with the top sorting are selected according to the audio quantity threshold (SEL_NUM) value as the current routing output channel, and the current audio data corresponding to the selected channel is used as the target audio data corresponding to the target receiving terminal.

[0102] Among them, it should be noted that the current audio data sent by each sending terminal is taken as one sound code stream. For example, if the number of sending terminals is 5, the current audio data corresponding to 5 sound code streams can be received. Then, at least one of the 5 channels is selected and sent to the target receiving terminal. This process can be called route selection in multi-person voice calls, that is, the selection of the number of call channels. In essence, it can also be understood as selecting the current audio data of at least one sending terminal from the current audio data of multiple sending terminals as the target audio data corresponding to the target receiving terminal, and then sending the target audio data to the target receiving terminal.

[0103] 105. Send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0104] For example, the target audio data can be sent to the target receiving terminal so that the target terminal can perform voice decoding on the target audio data, process the decoded data as a mixed superposition of sounds, and play the mixed audio signal, thereby maintaining the current voice call with at least one sending terminal.

[0105] After the target audio data is sent to the target receiving terminal, the target receiving terminal can count the network packet loss parameters of the current period corresponding to the target audio data or predict the network packet loss parameters of the next period of the current period based on the target audio data to obtain the updated network packet loss parameters. The statistical or prediction methods are described above in detail and will not be repeated here.

[0106] It should be noted that, after receiving the target audio data, the target receiving terminal may periodically detect network packet loss parameters for data packets corresponding to the target audio data, and return the periodically updated network packet loss parameters to the first electronic device.

[0107] In some embodiments, when an updated network packet loss parameter returned by the target receiving terminal is received, the updated network packet loss parameter is used as the network packet loss parameter, and the target audio selection parameter is used as the current audio selection parameter, and the process returns to the step of adjusting the current audio selection parameter according to the network packet loss parameter until the current voice call ends.

[0108] From the above, it can be seen that after obtaining the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call, the embodiment of the present application adjusts the current audio selection parameters according to the network packet loss parameters to obtain the target audio selection parameters. When the current audio data sent by multiple sending terminals is received, the audio features of the current audio data are obtained, and then, based on the audio features and the target audio selection parameters, at least one target audio data corresponding to the target receiving terminal is screened out from the current audio data, and then the target audio data is sent to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal; since the scheme can adjust the audio selection parameters through the network packet loss parameters that characterize the transmission quality of the receiving party, the audio selection parameters can be adaptively adjusted or adjusted in different network environments, thereby adapting to different network environments and reducing the occurrence of call freezes or failures in call grabbing, and therefore, the call quality of multi-person voice calls can be improved.

[0109] According to the method described in the above embodiment, the following is further described in detail with examples.

[0110] This embodiment will be described from the perspective of a second multi-person voice call device, which can be specifically integrated in a second electronic device, which can be a terminal or other device; wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device or other smart device that can conduct multi-person voice calls.

[0111] A multi-person voice call method, comprising:

[0112] Collect the voice signal in the current voice call to obtain the current audio data, identify the voice energy value corresponding to the voice signal in the current audio data, predict the noise energy value in the current audio data, and use the noise energy value and the voice energy value as the audio features of the current audio data, and send the current audio data and the audio features to the server, so that the server can determine the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and send the target audio data to the target receiving terminal.

[0113] like Figure 4 As shown, a method for multi-person voice call, the specific process is as follows:

[0114] 201. Collect an audio signal in a current voice call to obtain current audio data.

[0115] The voice signal may be a signal in which the object makes a sound and generates vibration.

[0116] There are many ways to collect the audio signal in the current voice call, which can be as follows:

[0117] For example, the audio signal emitted by the target object corresponding to the sending terminal can be collected by a sound collection device, and the collected audio signal can be audio-encoded to obtain the current audio data. Alternatively, the audio signal within a preset range around the sending terminal can be collected, and the collected audio signal can be audio-encoded to obtain the current audio data, and so on.

[0118] 202. A speech energy value corresponding to an audio signal identified in current audio data.

[0119] For example, the current audio data can be transformed in the frequency domain to obtain the power spectrum value of the frequency point in each frame of audio, the power spectrum values ​​can be fused to obtain the initial speech energy value corresponding to each frame of audio, and the initial speech energy value can be smoothed to obtain the speech energy value corresponding to the speech signal.

[0120] Among them, there are many ways to perform frequency domain transformation on the current audio data. For example, the current audio data can be divided into frames to obtain time domain signal data corresponding to each frame of audio, the time domain signal data can be converted into frequency domain to obtain frequency domain signal data corresponding to each frame of audio, and the power spectrum value of the frequency point in each frame of audio can be determined based on the frequency domain signal data.

[0121] There are many ways to divide the current audio data into frames. For example, a Hamming window can be used to divide the current audio data into frames and perform windowing processing, so as to obtain the time domain signal data corresponding to each frame of audio. The window function can be shown in formula (1), which can be specifically as follows:

[0122]

[0123] Among them, win(n) is the value of the window function, n is the location of the sampling point, N is the total number of sampling points in a single window (window length), and n belongs to

[0124] Optionally, in some embodiments, other window functions may be used to perform frame windowing on the current audio data, so as to obtain time domain signal data corresponding to each frame of audio. There may be multiple types of other window functions, such as Hamming window, Hanning window, rectangular window or other types of window functions, etc.

[0125] After the current audio data is framed, the time domain signal data after framing can be converted to the frequency domain to obtain the frequency domain signal data corresponding to each frame of audio. There are many ways to convert the time domain signal data to the frequency domain. For example, the time domain signal data can be discrete Fourier transformed to obtain the frequency domain signal data corresponding to each frame of audio, as shown in formula (2). Specifically, it can be as follows:

[0126]

[0127] Among them, i corresponds to the frame number, k is the corresponding frequency index value, k=1, 2, 3..., N, X(i, k) is the frequency domain signal data of the kth frequency point in the i-th frame, N is the window length, and n is the position of the sampling point.

[0128] After the time domain signal data is converted to the frequency domain, the power spectrum value of each frequency point in the audio frame can be determined according to the converted frequency domain signal data. The power spectrum value can be determined as shown in formula (3), which can be specifically as follows:

[0129] S(i, k) = |X(i, k)| 2 (3)

[0130] Among them, S(i, k) is the power spectrum value of the kth frequency point in the i-th frame, X(i, k) is the frequency domain signal data corresponding to the frequency point, i corresponds to the frame number, k is the corresponding frequency point index value, k=1, 2, 3..., N, X(i, k) is the frequency domain signal data of the kth frequency point in the i-th frame, and N is the window length.

[0131] After the current audio data is transformed in the frequency domain, the power spectrum values ​​of the frequency points can be fused to obtain the initial speech energy corresponding to each frame of audio. There are many ways to fuse the power spectrum values ​​of the frequency points. For example, the power spectrum values ​​of all frequency points in each frame can be obtained, and the power spectrum values ​​can be accumulated to obtain the initial speech energy value of the frame of audio, as shown in formula (4). Specifically, it can be as follows:

[0132]

[0133] Among them, E(i) is the initial speech energy value of the i-th frame, S(i,k) is the power spectrum value of the k-th frequency point in the i-th frame, i corresponds to the frame number, k is the corresponding frequency point index value, k = 1, 2, 3..., N, N is the window length.

[0134] After the power spectrum values ​​are fused, the fused initial speech energy values ​​can be smoothed to obtain the speech energy values ​​corresponding to the audio signal. There are many ways to smooth the initial speech energy values. For example, the initial speech energy values ​​are sorted based on the number of audio frames. Based on the sorting results, the initial speech energy values ​​corresponding to the target frame and the previous frame of the target frame are selected from the initial speech energy values ​​to obtain the target energy value pair, and the smoothing coefficient is obtained. Based on the smoothing coefficient, the initial speech energy values ​​in the target energy value pair are fused to obtain the smoothed speech energy value of the target frame, and the step of selecting the initial speech energy values ​​corresponding to the target frame and the previous frame of the target frame from the initial speech energy values ​​is returned until the smoothed speech energy values ​​corresponding to all frames are obtained, thereby obtaining the speech energy value corresponding to the audio signal, which can be shown as formula (5). Specifically, it can be as follows:

[0135] E′(i)=c*E′(i-1)+(1-c)*E(i) (5)

[0136] Wherein, i is the target frame, c is the smoothing coefficient, E′(i) is the smoothed speech energy value of the target frame, and E′(i-1) is the smoothed speech energy value of the previous frame of the target frame.

[0137] The smoothing coefficient may be pre-set, and may be any value greater than 0 and less than 1, for example, 0.93 or other values.

[0138] 203. Predict a noise energy value in the current audio data, and use the noise energy value and the speech energy value as the current audio data.

[0139] The noise energy value can represent the intensity of the noise in the audio data.

[0140] There are many ways to predict the noise energy value in the current audio data, which can be as follows:

[0141] For example, the power spectrum values ​​can be compared to obtain the initial noise power spectrum value of the frequency point, and the speech existence probability of the frequency point can be calculated based on the initial noise power spectrum value. The speech existence probability and the power spectrum value can be fused to obtain the noise energy value in the current audio data.

[0142] There are many ways to compare the power spectrum values ​​to obtain the initial noise power spectrum values ​​of the frequency points. For example, the power spectrum values ​​of the frequency points can be smoothed in the time-frequency domain to obtain the smoothed power spectrum values. Based on the minimum tracking algorithm, the smoothed power spectrum values ​​are compared to obtain the initial noise power spectrum values ​​of each frequency point.

[0143] Among them, there can be many ways to perform time-frequency domain smoothing on the power spectrum values ​​of frequency points. For example, a frequency domain smoothing weighting factor group can be obtained, and based on the frequency domain smoothing weighting factor group, the power spectrum value of each frequency point is smoothed to obtain the initial smoothed power spectrum value of each frequency point, and the time domain smoothing factor is obtained. Based on the time domain smoothing factor, the initial smoothed power spectrum value is adjusted to obtain the smoothed power spectrum value of each frequency point.

[0144] Among them, there are many ways to smooth the power spectrum value of each frequency point based on the frequency domain smoothing weighting factor group. For example, a target frequency point is determined in the frequency points, and based on the frequency domain smoothing weighting factor group, the associated frequency points corresponding to the target frequency point are screened out in the frequency points, and the power spectrum values ​​of the associated frequency points are fused to obtain the initial smoothed power spectrum value of the target frequency point. Return to the step of determining the target frequency point in the frequency points until each frequency point in the frequency points is used as the target frequency point, and the initial smoothed power spectrum value of each frequency point is obtained, which can be shown as formula (6), and can be specifically as follows:

[0145]

[0146] in, is the initial smoothed power spectrum value of the kth frequency point in the i-th frame, i corresponds to the frame number, k is the corresponding frequency point index value, b is the frequency domain smoothing weighting factor group, for example, b[5] = [0.1, 0.2, 0.4, 0.2, 0.1] or other factor groups. j and w are the parameters constituting the associated frequency point, S(i, k+j) is the associated frequency point, and b(j+w) is the weighting factor corresponding to the associated frequency point.

[0147] After smoothing the power spectrum value of each frequency point, the time domain smoothing factor can be obtained, and the initial smoothed power spectrum value is adjusted based on the time domain smoothing factor to obtain the smoothed power spectrum value of each frequency point. There are many ways to adjust the initial smoothed power spectrum value, for example, select a target frequency point from the frequency points, adjust the initial smoothed power spectrum value of the target frequency point based on the time domain smoothing factor to obtain the smoothed power spectrum value of the target frequency point, adjust the initial smoothed power spectrum value of the next frequency point of the target frequency point based on the smoothed power spectrum value and the time domain smoothing factor to obtain the smoothed power spectrum value, and use the next frequency point of the target frequency point as the target frequency point, return to execute the step of adjusting the initial smoothed power spectrum value of the next frequency point of the target frequency point based on the smoothed power spectrum value and the time domain smoothing factor, until there is no next frequency point of the target frequency point, and obtain the smoothed power spectrum value of each frequency point, which can be shown as formula (7), specifically as follows:

[0148]

[0149] Among them, i corresponds to the frame number, k is the corresponding frequency index value, is the smoothed power spectrum value of the kth frequency point in the i-th frame, α0 is the time domain smoothing factor, The smoothed power spectrum value of the previous frequency point of this frequency point, is the initial smoothed power spectrum value of the kth frequency point in the i-th frame.

[0150] After the power spectrum values ​​of the frequency points are smoothed in the time-frequency domain, the smoothed power spectrum values ​​can be compared based on the minimum tracking algorithm to obtain the initial noise power spectrum value of each frequency point, as shown in the following code:

[0151]

[0152] After comparing the power spectrum values, the probability of speech existence at the frequency point can be calculated based on the initial noise power spectrum value obtained by comparison. The so-called speech existence probability can be the probability of speech existence in the frequency domain signal data corresponding to the frequency point. When there is no speech, the frequency domain signal data can be noise data. There are many ways to calculate the probability of speech existence. For example, the ratio between the initial noise power spectrum value and the smoothed power spectrum value is calculated to obtain the power spectrum ratio, and the power spectrum value ratio is compared with the preset ratio threshold to obtain the initial speech existence probability of each frequency point, and the speech adjustment factor is obtained. Based on the speech adjustment factor, the initial speech existence probability is adjusted to obtain the speech existence probability of each frequency point, which can be shown as formula (8), and can be specifically as follows:

[0153]

[0154] Among them, i corresponds to the frame number, k is the corresponding frequency index value, S r (i,k) is the power spectrum ratio, is the power spectrum value after smoothing, S min is the initial noise power spectrum value, p(i,k) is the initial speech existence probability, δ is the preset ratio threshold, α p is the voice adjustment factor, is the probability of speech existence.

[0155] After calculating the probability of speech existence at the frequency point, the speech existence probability can be fused with the power spectrum value to obtain the noise energy value in the current audio data. There are many ways to fuse the speech existence probability with the power spectrum value, for example, based on the speech existence probability, the power spectrum value is converted to obtain the noise power spectrum value corresponding to each frequency point, the noise power spectrum values ​​of the frequency points in each frame are fused to obtain the initial noise energy value corresponding to each frame, the initial noise energy value is smoothed, and the smoothed noise energy value is fused to obtain the noise energy value in the current audio data.

[0156] Among them, there are many ways to convert the power spectrum value based on the probability of speech existence. For example, the first frequency point can be selected from the frequency points to obtain the target frequency point. Based on the probability of speech existence at the target frequency point, the power spectrum value of the target frequency point is adjusted to obtain the noise power spectrum value of the target frequency point. The next frequency point of the target frequency point is selected from the frequency points as the current frequency point. Based on the probability of speech existence at the current frequency point, the noise power spectrum value of the target frequency point and the power spectrum value of the current frequency point are fused to obtain the noise power spectrum value of the current frequency point. The current frequency point is used as the target frequency point, and the step of selecting the next frequency point of the target frequency point from the frequency points as the current frequency point is returned to execute until the next frequency point of the target frequency point does not exist. The noise power spectrum value of each frequency point can be obtained, which can be shown as formula (9). Specifically, it can be as follows:

[0157]

[0158] Among them, i corresponds to the frame number, k is the corresponding frequency index value, is the noise power spectrum value, is the probability of speech existence, S(i,k) is the power spectrum value.

[0159] After the power spectrum value is converted, the converted noise power spectrum value can be fused to obtain the initial noise energy value corresponding to each frame, which can be shown in formula (10). Specifically, it can be as follows:

[0160]

[0161] Among them, i corresponds to the frame number, k is the corresponding frequency index value, En(i) is the initial noise energy, N is the window length, is the noise power spectrum value.

[0162] After fusing the noise power spectrum values ​​of the mid-frequency points of each frame, the initial noise energy value obtained after fusion can be smoothed, as shown in formula (11). Specifically, it can be as follows

[0163] En′(i)=c*En′(i-1)+(1-c)*En(i) (11)

[0164] Among them, i corresponds to the frame number, En′(i) is the smoothed noise energy value, c is the smoothing coefficient, the value range of c can be any value in [0,1], for example, it can be 0.93 or other values, and En(i) is the initial noise energy value.

[0165] The smoothing process is similar to the smoothing process for the initial speech energy value, as described above, and will not be described in detail here.

[0166] After the initial noise energy value is smoothed, the smoothed noise energy value of each frame after the smoothing process may be fused to obtain the noise energy value in the current audio data.

[0167] After the noise energy value in the current audio data is predicted, the speech energy value and the noise energy value can be used as audio features of the current audio data.

[0168] 204. Send the current audio data and the audio features to the server, so that the server determines the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal.

[0169] There are many ways to send the current audio data and audio features to the server, which can be as follows:

[0170] For example, the current audio data and audio features may be directly sent to the server, or the current audio data and audio features may be compressed and the compressed current audio data and the compressed audio features may be sent to the server, or the current audio data may be encrypted and the encrypted current audio data and audio features may be sent to the server, and so on.

[0171] After receiving the audio data and audio features, the server can determine the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and send the target audio data to the target receiving terminal. For details, please refer to the above description and will not be repeated here.

[0172] After the server determines the current audio data as the target audio data of the target receiving terminal and sends it to the target receiving terminal, the sending terminal and the target receiving terminal can continue to maintain a multi-person voice call. The page diagram of the multi-person voice call can be as follows Figure 5 As shown, it should be noted that the call type of the multi-person voice call in this solution can include voice call and video call. In a video call, the video data and the audio data can be separated. This solution mainly focuses on the transmission of audio data (i.e., the voice call in the video call).

[0173] From the above, it can be seen that the embodiment of the present application collects the audio signal in the current voice call and obtains the current audio data, then identifies the speech energy value corresponding to the audio signal in the current audio data, and then predicts the noise energy value in the current audio data, and uses the noise energy value and the speech energy value as the audio features of the current audio data, and sends the current audio data and the audio features to the server, so that the server can determine the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and send the target audio data to the target receiving terminal; because the scheme can identify the speech energy value and the noise energy value as audio features after collecting the current audio data, and send the current audio data and the corresponding audio features to the server, so that the server can determine the target audio data corresponding to the target receiving terminal based on the audio features, thereby reducing the occurrence of call freezes or failures to grab the call, and therefore, the call quality of multi-person voice calls can be improved.

[0174] According to the method described in the above embodiment, the following is further described in detail with examples.

[0175] In this embodiment, the first multi-person voice call device is integrated in the server, the second multi-person voice call device is integrated in the terminal, and the terminal is a sending terminal, the network packet loss parameter is the packet loss rate, the audio selection parameter is the routing parameter, and the audio feature is sent from the sending terminal to the server.

[0176] like Figure 6 As shown, the specific process of the multi-person voice call method can be as follows:

[0177] 301. The server obtains current routing parameters and packet loss rate of a target receiving terminal in a current voice call.

[0178] For example, the server can obtain the current routing parameters of the target receiving terminal in the current voice call, receive the current packet loss rate of the current period or the predicted packet loss rate of the next period after the current period corresponding to the current routing parameters returned by the target receiving terminal, and use the current packet loss rate or the predicted packet loss rate as the packet loss rate of the target receiving terminal.

[0179] 302. The server adjusts the current routing parameters according to the packet loss rate to obtain target routing parameters.

[0180] For example, the server can compare the packet loss rate with a preset packet loss parameter threshold. When the packet loss rate is greater than the preset packet loss parameter threshold, an audio quantity generation function is used to generate a candidate audio quantity threshold corresponding to the packet loss rate, and a noise energy generation function is used to generate a candidate noise energy threshold corresponding to the packet loss rate. The candidate audio quantity threshold and the candidate noise energy threshold are used as candidate routing parameters.

[0181] The server obtains a preset threshold range corresponding to the candidate audio quantity threshold. When the candidate audio quantity threshold is within the preset threshold range, the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold. When the candidate audio quantity threshold is outside the preset threshold range, the basic audio quantity threshold is screened out within the preset threshold range, and the basic audio quantity threshold is used as the candidate audio quantity threshold, and the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold. The current noise energy threshold is replaced with the candidate noise energy threshold to obtain the target noise energy threshold, and the target audio quantity threshold and the target noise energy threshold are used as target routing parameters.

[0182] When the packet loss rate is less than or equal to the preset packet loss parameter threshold, the server replaces the current routing parameters with the preset initial routing parameters to obtain the target routing parameters.

[0183] 303. The sending terminal collects the audio signal in the current voice call to obtain current audio data.

[0184] For example, the sending terminal can collect the audio signal emitted by the target user corresponding to the sending terminal through a sound collection device, and perform audio encoding on the collected audio signal to obtain the current audio data, or it can also collect the audio signal within a preset range around the sending terminal, and perform audio encoding on the collected audio signal to obtain the current audio data, and so on.

[0185] 304. Send a speech energy value corresponding to the audio signal recognized by the terminal in the current audio data.

[0186] For example, the transmitting terminal can use a Hamming window to perform frame-by-frame windowing on the current audio data, thereby obtaining the time domain signal data corresponding to each frame of audio, and performing a discrete Fourier transform on the time domain signal data to obtain the frequency domain signal data corresponding to each frame of audio, and determine the power spectrum value of the frequency point in each frame of audio based on the converted frequency domain signal data. The power spectrum values ​​of all frequency points in each frame are obtained, and the power spectrum values ​​are accumulated to obtain the initial speech energy value of the frame of audio.

[0187] The sending terminal sorts the initial speech energy values ​​based on the frame number of the audio frame, and based on the sorting result, selects the initial speech energy values ​​corresponding to the target frame and the previous frame of the target frame from the initial speech energy values ​​to obtain a target energy value pair, obtains a smoothing coefficient, and based on the smoothing coefficient, fuses the initial speech energy values ​​in the target energy value pair to obtain a smoothed speech energy value of the target frame, and returns to execute the step of selecting the initial speech energy values ​​corresponding to the target frame and the previous frame of the target frame from the initial speech energy values, until the smoothed speech energy values ​​corresponding to all frames are obtained, thereby obtaining the speech energy value corresponding to the audio signal, which can be shown as formula (5).

[0188] 305. The sending terminal predicts a noise energy value in the current audio data, and uses the noise energy value and the speech energy value as the current audio data.

[0189] For example, the transmitting terminal determines the target frequency point among the frequency points, selects the associated frequency points corresponding to the target frequency point among the frequency points based on the frequency domain smoothing weighting factor group, fuses the power spectrum values ​​of the associated frequency points, obtains the initial smoothed power spectrum value of the target frequency point, returns to execute the step of determining the target frequency point among the frequency points, until each frequency point in the frequency points is used as the target frequency point, and obtains the initial smoothed power spectrum value of each frequency point, which can be shown as formula (6).

[0190] The transmitting terminal can select a target frequency point from the frequency points, adjust the initial smoothed power spectrum value of the target frequency point based on the time domain smoothing factor to obtain the smoothed power spectrum value of the target frequency point, adjust the initial smoothed power spectrum value of the next frequency point of the target frequency point based on the smoothed power spectrum value and the time domain smoothing factor to obtain the smoothed power spectrum value, and use the next frequency point of the target frequency point as the target frequency point, return to execute the step of adjusting the initial smoothed power spectrum value of the next frequency point of the target frequency point based on the smoothed power spectrum value and the time domain smoothing factor, until there is no next frequency point of the target frequency point, and obtain the smoothed power spectrum value of each frequency point, which can be shown as formula (7). Based on the minimum tracking algorithm, the smoothed power spectrum values ​​are compared to obtain the initial noise power spectrum value of each frequency point.

[0191] The transmitting terminal can calculate the ratio between the initial noise power spectrum value and the smoothed power spectrum value to obtain the power spectrum ratio, compare the power spectrum value ratio with the preset ratio threshold to obtain the initial speech existence probability of each frequency point, obtain the speech adjustment factor, and adjust the initial speech existence probability based on the speech adjustment factor to obtain the speech existence probability of each frequency point, which can be shown as formula (8).

[0192] The transmitting terminal can select the first frequency point from the frequency points to obtain the target frequency point, adjust the power spectrum value of the target frequency point based on the speech existence probability of the target frequency point, obtain the noise power spectrum value of the target frequency point, select the next frequency point of the target frequency point from the frequency points as the current frequency point, fuse the noise power spectrum value of the target frequency point and the power spectrum value of the current frequency point based on the speech existence probability of the current frequency point, obtain the noise power spectrum value of the current frequency point, take the current frequency point as the target frequency point, return to execute the step of selecting the next frequency point of the target frequency point from the frequency points as the current frequency point, until the next frequency point of the target frequency point does not exist, and obtain the noise power spectrum value of each frequency point, which can be shown as formula (9).

[0193] The transmitting terminal fuses the noise power spectrum values ​​of the mid-frequency points in each frame to obtain the initial noise energy value corresponding to each frame, smoothes the initial noise energy value, and fuses the smoothed noise energy value to obtain the noise energy value in the current audio data. The speech energy value and the noise energy value are used as the audio features of the current audio data.

[0194] 306. The sending terminal sends the current audio data and audio features to the server.

[0195] For example, the sending terminal can directly send the current audio data and audio features to the server, or can compress the current audio data and audio features and send the compressed current audio data and compressed audio features to the server, or can encrypt the current audio data and send the encrypted current audio data and audio features to the server, and so on.

[0196] 307. The server selects at least one target audio data corresponding to the target receiving terminal from the current audio data based on the audio feature and the target routing parameter.

[0197] For example, the server can filter out at least one audio data whose noise energy value is less than or equal to the target noise energy threshold value from the current audio data, obtain candidate audio data corresponding to the target receiving terminal, and sort the candidate audio data corresponding to the target receiving terminal based on the speech energy value. Based on the target audio quantity threshold value, determine the target sorting range corresponding to the candidate audio data, and according to the sorting result, filter out at least one audio data corresponding to the target sorting range from the candidate audio data to obtain the target audio data.

[0198] 308. The server sends the target audio data to the target receiving terminal.

[0199] For example, the server may directly send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0200] 309. The target receiving terminal plays an audio signal based on the target audio data to maintain a multi-person voice call with the sending terminal.

[0201] For example, the target terminal may perform voice decoding on the target audio data, process the decoded data as a mixed and superimposed sound, and play the mixed audio signal, thereby maintaining the current voice call with at least one sending terminal.

[0202] After sending the target audio data to the target receiving terminal, the target receiving terminal can calculate the packet loss rate of the current period corresponding to the target audio data or predict the packet loss rate of the next period of the current period based on the target audio data, obtain the updated packet loss rate, and send the updated packet loss rate to the server.

[0203] In the multi-person voice call scenario of this solution, a server, a sending terminal and a target receiving terminal may be included. The number of sending terminals may be one or more. Taking the server as a routing server as an example, the overall process of a multi-person voice call may be as follows: Figure 7 As shown, the sending terminal collects and processes the audio signal and extracts features, and performs voice encoding on the collected audio signal to obtain current audio data and corresponding audio features, and sends the current audio data and audio features to the routing server through the network. The routing server receives the packet loss rate of the current period returned by the target receiving terminal, adjusts the routing parameters based on the packet loss rate, obtains the target routing parameters, and performs audio routing on the received current audio data through the target routing parameters, that is, one or more channels of audio data are filtered out from the received current audio data as the target audio data of the target receiving terminal, and then, the selected target audio data is sent to the target receiving terminal through the network. The target receiving terminal performs voice decoding and mixing on the target audio data, and finally plays the mixed audio signal, thereby realizing multi-person voice calls and improving the call quality of multi-person voice calls.

[0204] From the above, it can be seen that in this embodiment, after obtaining the current routing parameters and packet loss rate of the target receiving terminal in the current voice call, the server adjusts the current routing parameters according to the packet loss rate to obtain the target routing parameters. When receiving the current audio data sent by multiple sending terminals, the audio features of the current audio data are obtained, and then, based on the audio features and the target routing parameters, at least one target audio data corresponding to the target receiving terminal is screened out from the current audio data, and then the target audio data is sent to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal; since the scheme can adjust the routing parameters by the packet loss rate that characterizes the transmission quality of the receiver, the routing parameters can be adaptively adjusted or adjusted in different network environments, thereby adapting to different network environments and reducing the occurrence of call freezes or failures in call grabbing, and therefore, the call quality of multi-person voice calls can be improved.

[0205] In order to better implement the above method, an embodiment of the present invention also provides a multi-person voice call device (i.e., a first multi-person voice call device). The first multi-person voice call device can be integrated in a server. The server can be a single server or a server cluster composed of multiple servers.

[0206] For example, Figure 8 As shown, the first multi-person voice call device may include a first acquisition unit 401, an adjustment unit 402, a second acquisition unit 403, a screening unit 404 and a sending unit 405, as follows:

[0207] (1) a first acquisition unit 401;

[0208] The first acquisition unit 401 is used to acquire the current audio selection parameter and network packet loss parameter of the target receiving terminal in the current voice call.

[0209] For example, the first acquisition unit 401 can be specifically used to obtain the current audio selection parameter of the target receiving terminal in the current voice call, receive the current network packet loss parameter of the current period corresponding to the current audio selection parameter returned by the target receiving terminal or the predicted network packet loss parameter of the next period after the current period, and use the current network packet loss parameter or the predicted network packet loss parameter as the network packet loss parameter of the target receiving terminal.

[0210] (2) adjustment unit 402;

[0211] The adjusting unit 402 is used to adjust the current audio selection parameter according to the network packet loss parameter to obtain the target audio selection parameter.

[0212] For example, the adjustment unit 402 may be specifically configured to compare the network packet loss parameter with a preset packet loss parameter threshold, and adjust the current audio selection parameter according to the comparison result to obtain the target audio selection parameter.

[0213] (3) a second acquisition unit 403;

[0214] The second acquisition unit 403 is used to obtain the audio features of the current audio data when the current audio data sent by multiple sending terminals is received.

[0215] For example, the second acquisition unit 403 can be specifically used to, when receiving current audio data sent by multiple sending terminals, receive audio features corresponding to the current audio data sent by the sending terminals, or perform feature extraction on the current audio data to obtain the audio features corresponding to the current audio data, or send the current audio data to a third-party server so that the third-party server can perform feature extraction on the current audio data and receive the audio features corresponding to the current audio data returned by the third-party server.

[0216] (4) screening unit 404;

[0217] The screening unit 404 is configured to screen out at least one target audio data corresponding to a target receiving terminal from the current audio data based on the audio feature and the target audio selection parameter.

[0218] For example, the screening unit 404 can be specifically used to screen out at least one audio data having a noise energy value less than or equal to a target noise energy threshold in the current audio data, obtain candidate audio data corresponding to the target receiving terminal, sort the candidate audio data corresponding to the target receiving terminal based on the speech energy value, and screen out at least one audio data from the candidate audio data according to the sorting result and the target audio quantity threshold to obtain the target audio data.

[0219] (5) sending unit 405;

[0220] The sending unit 405 is used to send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0221] For example, the sending unit 405 can be specifically used to send the target audio data to the target receiving terminal so that the target terminal can perform voice decoding on the target audio data, process the decoded data as a mixed superposition of sounds, and play the mixed audio signal, thereby maintaining the current voice call with at least one sending terminal.

[0222] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0223] From the above, it can be seen that in the embodiment of the present application, after the first acquisition unit 401 acquires the current audio selection parameter and the network packet loss parameter of the target receiving terminal in the current voice call, the adjustment unit 402 adjusts the current audio selection parameter according to the network packet loss parameter to obtain the target audio selection parameter. When the second acquisition unit 403 receives the current audio data sent by multiple sending terminals, it acquires the audio features of the current audio data. Then, the screening unit 404 filters out at least one target audio data corresponding to the target receiving terminal in the current audio data based on the audio features and the target audio selection parameter. Then, the sending unit 405 sends the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal. Since the scheme can adjust the audio selection parameter by the network packet loss parameter that characterizes the transmission quality of the receiving party, the audio selection parameter can be adaptively adjusted or adjusted in different network environments, thereby adapting to different network environments and reducing the occurrence of call freezes or failures in call grabbing. Therefore, the call quality of multi-person voice calls can be improved.

[0224] In order to better implement the above method, an embodiment of the present invention also provides a multi-person voice call device (i.e., a second multi-person voice call device), which can be integrated in a terminal, and the terminal can include a tablet computer, a laptop computer and / or a personal computer, etc.

[0225] For example, Fig. 9 As shown, the second multi-person voice call device may include a collection unit 501, a recognition unit 502, a prediction unit 503 and a call unit 504, as follows:

[0226] (1) Collection unit 501;

[0227] The collection unit 501 is used to collect the audio signal in the current voice call to obtain the current audio data.

[0228] For example, the collection unit 501 can be specifically used to collect the audio signal emitted by the target object corresponding to the sending terminal through a sound collection device, and perform audio encoding on the collected audio signal to obtain the current audio data, or it can also collect the audio signal within a preset range around the sending terminal, and perform audio encoding on the collected audio signal to obtain the current audio data.

[0229] (2) identification unit 502;

[0230] The recognition unit 502 is used to recognize the speech energy value corresponding to the audio signal in the current audio data.

[0231] For example, the recognition unit 502 can be specifically used to perform frequency domain transformation on the current audio data to obtain the power spectrum value of the frequency point in each frame of audio, fuse the power spectrum values ​​to obtain the initial speech energy value corresponding to each frame of audio, and smooth the initial speech energy value to obtain the speech energy value corresponding to the speech signal.

[0232] (3) prediction unit 503;

[0233] The prediction unit 503 is used to predict the noise energy value in the current audio data, and use the noise energy value and the speech energy value as the audio features of the current audio data.

[0234] For example, the prediction unit 503 can be specifically used to compare the power spectrum values ​​to obtain the initial noise power spectrum value of the frequency point, calculate the speech existence probability of the frequency point based on the initial noise power spectrum value, and fuse the speech existence probability with the power spectrum value to obtain the noise energy value in the current audio data.

[0235] (4) a conversation unit 504;

[0236] The call unit 504 is used to send the current audio data and audio features to the server, so that the server determines the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal.

[0237] For example, the call unit 504 can be specifically used to send the current audio data and audio features to the server, or to compress the current audio data and audio features and send the compressed current audio data and compressed audio features to the server, or to encrypt the current audio data and send the encrypted current audio data and audio features to the server, so that the server can determine the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and send the target audio data to the target receiving terminal.

[0238] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0239] From the above, it can be seen that in this embodiment, after the collection unit 501 collects the audio signal in the current voice call and obtains the current audio data, the recognition unit 502 identifies the speech energy value corresponding to the audio signal in the current audio data, and then the prediction unit 503 predicts the noise energy value in the current audio data, and uses the noise energy value and the speech energy value as the audio features of the current audio data. The call unit 504 sends the current audio data and the audio features to the server, so that the server determines the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal; because this scheme can identify the speech energy value and the noise energy value as audio features after collecting the current audio data, and send the current audio data and the corresponding audio features to the server, so that the server can determine the target audio data corresponding to the target receiving terminal based on the audio features, thereby reducing the occurrence of call freezes or failures to grab the call, and therefore, the call quality of multi-person voice calls can be improved.

[0240] The embodiment of the present invention further provides an electronic device, such as Fig.10 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0241] The electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will appreciate that Fig.10The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0242] The processor 601 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 602, and calling data stored in the memory 602, the processor 601 performs various functions of the electronic device and processes data. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.

[0243] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0244] The electronic device also includes a power supply 603 for supplying power to each component. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 603 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0245] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0246] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 will run the application programs stored in the memory 602, thereby realizing various functions, as follows:

[0247] Obtain the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call, adjust the current audio selection parameters according to the network packet loss parameters, and obtain the target audio selection parameters. When receiving current audio data sent by multiple sending terminals, obtain the audio features of the current audio data, and based on the audio features and the target audio selection parameters, filter out at least one target audio data corresponding to the target receiving terminal in the current audio data, and send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0248] or

[0249] Collect the voice signal in the current voice call to obtain the current audio data, identify the voice energy value corresponding to the voice signal in the current audio data, predict the noise energy value in the current audio data, and use the noise energy value and the voice energy value as the audio features of the current audio data, and send the current audio data and the audio features to the server, so that the server can determine the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and send the target audio data to the target receiving terminal.

[0250] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0251] From the above, it can be seen that after obtaining the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call, the embodiment of the present application adjusts the current audio selection parameters according to the network packet loss parameters to obtain the target audio selection parameters. When the current audio data sent by multiple sending terminals is received, the audio features of the current audio data are obtained, and then, based on the audio features and the target audio selection parameters, at least one target audio data corresponding to the target receiving terminal is screened out from the current audio data, and then the target audio data is sent to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal; since the scheme can adjust the audio selection parameters through the network packet loss parameters that characterize the transmission quality of the receiving party, the audio selection parameters can be adaptively adjusted or adjusted in different network environments, thereby adapting to different network environments and reducing the occurrence of call freezes or failures in call grabbing, and therefore, the call quality of multi-person voice calls can be improved.

[0252] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0253] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the multi-person voice call methods provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0254] Obtain the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call, adjust the current audio selection parameters according to the network packet loss parameters, and obtain the target audio selection parameters. When receiving current audio data sent by multiple sending terminals, obtain the audio features of the current audio data, and based on the audio features and the target audio selection parameters, filter out at least one target audio data corresponding to the target receiving terminal in the current audio data, and send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

[0255] or

[0256] Collect the voice signal in the current voice call to obtain the current audio data, identify the voice energy value corresponding to the voice signal in the current audio data, predict the noise energy value in the current audio data, and use the noise energy value and the voice energy value as the audio features of the current audio data, and send the current audio data and the audio features to the server, so that the server can determine the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and send the target audio data to the target receiving terminal.

[0257] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0258] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0259] Since the instructions stored in the computer-readable storage medium can execute the steps in any one of the multi-person voice call methods provided in the embodiments of the present invention, the beneficial effects that can be achieved by any one of the multi-person voice call methods provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0260] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the above-mentioned multi-person voice call aspect or various optional implementations of the voice call aspect.

[0261] The above is a detailed introduction to a multi-person voice call method and related equipment provided in an embodiment of the present invention. The related equipment may include a multi-person voice call device, an electronic device and a computer-readable storage medium. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A multi-person voice call method, characterized in that: include: Obtain the current audio selection parameters and network packet loss parameters of the target receiving terminal in the current voice call; According to the network packet loss parameter, the current audio selection parameter is adjusted to obtain a target audio selection parameter; When receiving current audio data sent by multiple sending terminals, obtaining audio features of the current audio data; Based on the audio feature and the target audio selection parameter, screening out at least one target audio data corresponding to the target receiving terminal from the current audio data; The target audio data is sent to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

2. The multi-person voice call method according to claim 1, characterized in that: The step of adjusting the current audio selection parameter according to the network packet loss parameter to obtain the target audio selection parameter includes: Comparing the network packet loss parameter with a preset packet loss parameter threshold; According to the comparison result, the current audio selection parameter is adjusted to obtain the target audio selection parameter.

3. The multi-person voice call method according to claim 2, characterized in that: The step of adjusting the current audio selection parameter according to the comparison result to obtain the target audio selection parameter includes: When the network packet loss parameter is greater than a preset packet loss parameter threshold, adjusting the current audio selection parameter to obtain a target audio selection parameter; When the network packet loss parameter is less than or equal to the preset packet loss parameter threshold, the current audio selection parameter is replaced with a preset initial audio selection parameter to obtain a target audio selection parameter.

4. The multi-person voice call method according to claim 3, characterized in that: The current audio selection parameter includes a current audio quantity threshold and a current noise energy threshold, and the current audio selection parameter is adjusted to obtain a target audio selection parameter, including: Based on a preset parameter generation function, generating a candidate audio selection parameter corresponding to the network packet loss parameter, the candidate audio selection parameter including a candidate audio quantity threshold and a candidate noise energy threshold; Replacing the current audio quantity threshold with the candidate audio quantity threshold to obtain a target audio quantity threshold; The current noise energy threshold is replaced by the candidate noise energy threshold to obtain a target noise energy threshold, and the target audio quantity threshold and the target noise energy threshold are used as target audio selection parameters.

5. The multi-person voice call method according to claim 4, characterized in that: The preset parameter generation function includes an audio quantity generation function and a noise energy generation function, and the generation of the candidate audio selection parameter corresponding to the network packet loss parameter based on the preset parameter generation function includes: Using the audio quantity generation function to generate a candidate audio quantity threshold corresponding to the network packet loss parameter, the audio quantity generation function being a monotonically decreasing function; The noise energy generation function is used to generate a candidate noise energy threshold corresponding to the network packet loss parameter, wherein the noise energy generation function is a monotonically increasing function; The candidate audio quantity threshold and the candidate noise energy threshold are used as the candidate audio selection parameters.

6. The multi-person voice call method according to claim 4, characterized in that: The step of replacing the current audio quantity threshold with the candidate audio quantity threshold to obtain a target audio quantity threshold includes: Obtaining a preset threshold range corresponding to the candidate audio quantity threshold; When the candidate audio quantity threshold is within the preset threshold range, replacing the current audio quantity threshold with the candidate audio quantity threshold to obtain a target audio quantity threshold; When the candidate audio quantity threshold is outside the preset threshold range, the basic audio quantity threshold is screened out within the preset threshold range, the basic audio quantity threshold is used as the candidate audio quantity threshold, and the current audio quantity threshold is replaced with the candidate audio quantity threshold to obtain the target audio quantity threshold.

7. The multi-person voice call method according to claim 4, characterized in that: The audio feature includes a speech energy value and a noise energy value, and the selecting the target audio data corresponding to the target receiving terminal from the current audio data based on the audio feature and the target audio selection parameter includes: Filtering out at least one audio data whose noise energy value is less than or equal to the target noise energy threshold from the current audio data, and obtaining candidate audio data corresponding to the target receiving terminal; Based on the speech energy value, sorting the candidate audio data corresponding to the target receiving terminal; At least one audio data is screened out from the candidate audio data according to the sorting result and the target audio quantity threshold to obtain the target audio data.

8. The multi-person voice call method according to claim 7, characterized in that: The step of selecting at least one audio data from the candidate audio data according to the sorting result and the target audio quantity threshold to obtain the target audio data includes: Based on the target audio quantity threshold, determining a target sorting range corresponding to the candidate audio data; According to the sorting result, at least one audio data corresponding to the target sorting range is screened out from the candidate audio data to obtain target audio data.

9. The multi-person voice call method according to any one of claims 1 to 8, characterized in that: The obtaining of the current audio selection parameter and network packet loss parameter of the target receiving terminal in the current voice call includes: Obtain the current audio selection parameters of the target receiving terminal in the current voice call; Receiving a current network packet loss parameter of a current cycle corresponding to the current audio selection parameter or a predicted network packet loss parameter of a next cycle of the current cycle returned by the target receiving terminal; The current network packet loss parameter or the predicted network packet loss parameter is used as the network packet loss parameter of the target receiving terminal.

10. The multi-person voice call method according to any one of claims 1 to 8, characterized in that: After sending the target audio data to the target receiving terminal, the method further includes: When receiving the updated network packet loss parameter returned by the target receiving terminal based on the target audio data, using the updated network packet loss parameter as the network packet loss parameter, and using the target audio selection parameter as the current audio selection parameter; Return to the step of adjusting the current audio selection parameter according to the network packet loss parameter until the current voice call ends.

11. A multi-person voice call method, characterized in that: include: Collect the audio signal in the current voice call to obtain the current audio data; Identifying a speech energy value corresponding to the audio signal in the current audio data; Predicting a noise energy value in the current audio data, and using the noise energy value and the speech energy value as audio features of the current audio data; The current audio data and the audio features are sent to a server, so that the server determines the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal.

12. The multi-person voice call method according to claim 11, characterized in that: The step of identifying the speech energy value corresponding to the audio signal in the current audio data includes: Performing frequency domain transformation on the current audio data to obtain power spectrum values ​​of frequency points in each frame of audio; The power spectrum values ​​are fused to obtain an initial speech energy value corresponding to each frame of audio; The initial speech energy value is smoothed to obtain a speech energy value corresponding to the audio signal.

13. The multi-person voice call method according to claim 12, characterized in that: The performing frequency domain transformation on the current audio data to obtain the power spectrum value of each frequency point in the audio frame includes: Divide the current audio data into frames to obtain time domain signal data corresponding to each frame of audio; Performing frequency domain conversion on the time domain signal data to obtain frequency domain signal data corresponding to each frame of audio; The power spectrum value of each frequency point in the audio frame is determined according to the frequency domain signal data.

14. The multi-person voice call method according to claim 12, characterized in that: The predicting the noise energy value in the current audio data includes: Comparing the power spectrum values ​​to obtain the initial noise power spectrum value of the frequency point; Calculating the probability of speech existence at the frequency point according to the initial noise power spectrum value; The speech existence probability is fused with the power spectrum value to obtain the noise energy value in the current audio data.

15. A multi-person voice communication device, characterized in that: include: A first acquisition unit, configured to acquire a current audio selection parameter and a network packet loss parameter of a target receiving terminal in a current voice call; an adjusting unit, configured to adjust the current audio selection parameter according to the network packet loss parameter to obtain a target audio selection parameter; A second acquisition unit, configured to acquire audio features of the current audio data when receiving current audio data sent by multiple sending terminals; a screening unit, configured to screen out at least one target audio data corresponding to the target receiving terminal from the current audio data based on the audio feature and the target audio selection parameter; The sending unit is used to send the target audio data to the target receiving terminal to ensure the current voice call between the target receiving terminal and the sending terminal.

16. A multi-person voice communication device, characterized in that: include: A collection unit, used to collect the audio signal in the current voice call to obtain the current audio data; an identification unit, configured to identify a speech energy value corresponding to the audio signal in the current audio data; A prediction unit, configured to predict a noise energy value in the current audio data, and use the noise energy value and the speech energy value as audio features of the current audio data; The call unit is used to send the current audio data and audio features to the server, so that the server determines the target audio data corresponding to the target receiving terminal based on the audio features and the current audio data, and sends the target audio data to the target receiving terminal.

17. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to execute the steps in the multi-person voice call method according to any one of claims 1 to 14.

18. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps in the multi-person voice call method according to any one of claims 1 to 14 are implemented.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the multi-person voice call method according to any one of claims 1 to 14.

Images