Multi-channel sound mixing control method, device, equipment and medium

Abstract

The invention is suitable for the technical field of sound mixing, and particularly relates to a multi-channel sound mixing control method and device, equipment and a medium. The method comprises the following steps: acquiring an audio signal of each channel at the current moment, determining a speaking state of each channel and a transient extreme value of a corresponding audio according to the audio signal, and determining a channel with the maximum transient extreme value from all channels in the speaking state as a first target channel at the current moment and a second target channel at the current moment, and obtaining a first weight of the first target channel and a second weight of the second target channel, and carrying out sound mixing on the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain a sound mixing result. According to the technical scheme, the channel classification is realized by analyzing the audio signal, so that the channel weight needing to be enhanced and the channel weight needing to be attenuated are determined, the loss of the existing audio is avoided, the relatively important audio can be provided for enhancement, and the sound mixing definition is improved.

Description

Technical Field

[0001] The present application is applicable to the field of audio mixing technology, and in particular, relates to a multi-channel audio mixing control method, device, equipment and medium. Background Art

[0002] In conferences, stage design, concerts, releases, events and other communication scenarios of the organizing team, multiple people need to talk, and at this time, multi-channel mixing is needed to output audio. Multi-channel mixing is a way to integrate sounds from multiple sources into a stereo track or mono track. It is used in scenarios such as sound reinforcement and broadcasting, and requires multiple microphones and lavalier microphones to be cascaded as input. When there are multiple audio inputs, it is easy for multiple speakers to compete at the same time and the volume to overlap. This makes the mixed audio unclear, overflows to generate noise, and the background noise is large.

[0003] Sound is a pressure wave that is transmitted due to the pressure exerted by the vibration of an object on the air. It is collected by MIC and converted into an electrical signal, which is then sampled and quantized to obtain a digital voice signal. The superposition of voice signals is equivalent to the superposition of sound waves in the air. The mixing algorithm only needs to linearly superimpose the sampling values ​​of each signal to complete the mixing.

[0004] Multi-channel mixing is a large-scale mixing application. Weights, overflows, etc. are all factors that affect the mixing effect. At present, various mixing algorithms have their shortcomings and defects, and it is difficult to meet the application of large-scale team calls. Consider from the perspective of weight: 1) The weight uses the same value, which usually changes with the number of mixing channels. There is no overflow problem during mixing, but as the number of channels increases, the output volume also decreases. If the input of a certain channel is relatively small, the output of the entire mixing will be pulled down, and many voice details will be lost. It is suitable for mixing scenarios with fewer channels; 2) The weight is designed based on the input of the participating mixing according to its signal amplitude value, and the maximum value in each block of each channel participating in the mixing is defined. The total sum of the maximum values ​​of all the blocks of the channels is set. The weight changes with the maximum value in each block of each channel. The minimum volume of the multi-channel mixing is the weighted synthesis output. Among them, the proportion of the audio signals participating in the mixing is considered as the weight, so as to determine their proportion in the synthesized output. In this process, there is a situation where the voice after mixing is unclear, and the difference between multiple audio channels cannot be effectively expressed.

[0005] Therefore, how to optimize the audio signals of each channel in the fusion mix to avoid audio loss while improving the clarity of the mix is ​​an urgent problem to be solved. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a multi-channel mixing control method, device, equipment and medium to solve the problem of how to optimize the audio signals of each channel in the fusion mix to avoid audio loss while improving the clarity of the mix.

[0007] In a first aspect, an embodiment of the present application provides a multi-channel mixing control method, comprising: Acquire the audio signals of each channel at the current moment, and determine the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signals; Determine, from all channels in the speaking state of speaking, the channel with the largest transient maximum value as the first target channel at the current moment, and determine, from all channels in the speaking state of speaking, channels other than the first target channel as the second target channel at the current moment; A first weight corresponding to the first target channel at the current moment and a second weight corresponding to the second target channel at the current moment are obtained, and audio signals of the first target channel and the second target channel at the current moment are mixed according to the first weight and the second weight to obtain a mixing result.

[0008] In a second aspect, an embodiment of the present application provides a multi-channel audio mixing control device, comprising: A signal detection module is used to obtain the audio signals of each channel at the current moment, and determine the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signals; A channel identification module is used to determine the channel with the largest transient maximum value as the first target channel at the current moment from all channels in the speaking state, and determine the channels other than the first target channel as the second target channel at the current moment from all channels in the speaking state; A mixing processing module is used to obtain a first weight corresponding to the first target channel at the current moment and a second weight corresponding to the second target channel at the current moment, and mix the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain a mixing result.

[0009] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-channel mixing control method as described in the first aspect when executing the computer program.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the multi-channel mixing control method as described in the first aspect is implemented.

[0011] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the present application obtains the audio signals of each channel at the current moment, determines the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signal, determines the channel with the largest transient maximum value from all channels whose speech state is speech as the first target channel at the current moment, determines the channels other than the first target channel from all channels whose speech state is speech as the second target channel at the current moment, obtains the first weight corresponding to the first target channel at the current moment, and the second weight corresponding to the second target channel at the current moment, and mixes the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain the mixing result. The classification of channels is achieved by analyzing the audio signal, thereby determining the channel weights that need to be enhanced and the channel weights that need to be attenuated, avoiding the loss of existing audio, and being able to select relatively important audio for enhancement, thereby improving the clarity of the mixing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0013] Figure 1 This is a schematic diagram of an application environment of a multi-channel mixing control method provided in Example 1 of the present application; Figure 2 It is a flowchart of a multi-channel mixing control method provided in Embodiment 2 of the present application; Figure 3 It is a flowchart of a multi-channel mixing control method provided in Embodiment 3 of the present application; Figure 4 It is a flowchart of a multi-channel mixing control method provided in Embodiment 4 of the present application; Figure 5 It is a structural schematic diagram of a multi-channel mixing control device provided in Embodiment 5 of the present application; Figure 6 It is a structural diagram of a computer device provided in Example 6 of the present application. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0015] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0016] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0017] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0018] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0019] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0020] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0021] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0022] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0023] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0024] A multi-channel mixing control method provided in the first embodiment of the present application can be applied in the following aspects: Figure 1 In an application environment, N channels are mixed, N∈(1,2,3,…,n), and each channel is connected to Voice Activity Detection (VAD), so as to realize audio detection and analysis, and then the subsequent steps are performed according to the method of the present application to realize weight distribution. Among them, the multi-channel mixing control method can be run in a computer device including an audio system to realize mixing of the received multi-channel audio and output through the audio system, and the audio system includes but is not limited to professional audio equipment, traditional speakers, etc. The computer device may include but is not limited to a handheld computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud computer device, a personal digital assistant (PDA), etc.

[0025] See also Figure 2 , is a flow chart of a multi-channel mixing control method provided in Embodiment 2 of the present application. Figure 2 As shown, the multi-channel mixing control method may include the following steps: Step S201, obtaining the audio signals of each channel at the current moment, and determining the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signals.

[0026] This application is applicable to multiple channel connections for mixing, where the number of channels is N, such as Figure 1 As shown, channel 1 to channel n all provide audio signals to VAD. The audio signals are detected in VAD, and the speech state of each channel and the transient maximum value of the corresponding audio can be determined.

[0027] The speaking state can characterize the validity of the channel, that is, the audio signal of the channel in the speaking state can be mixed, while the audio signal of the channel in the non-speaking state cannot be mixed.

[0028] Transient maximums represent the maximum value of a signal strength that changes significantly in a very short period of time, which can directly affect the clarity and dynamic range of the sound. For example, in music, percussion instruments such as drums and cymbals often generate transient signals, which need to respond quickly when played back by audio equipment, otherwise the sound will be distorted. Therefore, considering the characteristics of transient maximums when mixing can effectively avoid distortion and delay, thereby providing a more realistic and natural listening experience.

[0029] For example, Figure 1 The speaking state is represented by the speaking label. When the speaking label is 1, the speaking state is determined to be speaking, and when the speaking label is 0, the speaking state is determined to be not speaking.

[0030] When no one is talking, the system searches for the user channel that recently activated the speaking tag, turns on its microphone without performing VAD, and locks the background noise atmosphere of this user channel to prevent the user from thinking that the machine is disconnected or malfunctioning.

[0031] When a single person is speaking, that is, only one speech label is 1, the voice weight of the user channel corresponding to the speech label is 1 and output.

[0032] When multiple people speak at the same time, that is, there are multiple user channels with speech labels of 1, the subsequent steps are executed to achieve weight distribution.

[0033] Optionally, determining the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signal includes: For any channel, the volume amplitude of the audio signal of the channel is detected. If the volume amplitude is greater than a preset amplitude, the speaking state of the channel is determined to be speaking; Perform transient analysis on the audio signal of the channel and determine the transient maximum value as the transient maximum value of the channel.

[0034] Among them, VAD can determine whether the channel is speaking based on the volume amplitude, human voice detection, etc. in the audio signal, and analyze the transient characteristics of the audio signal to determine the maximum amplitude of the significant change in signal strength in a very short time, and use it as the transient maximum value to characterize the importance of the audio signal.

[0035] In one embodiment, the VAD can also distinguish between human voice and other noises, so as to determine an audio signal containing human voice as speech, and a signal without human voice as non-speech.

[0036] Step S202: determine the channel with the largest transient maximum value from all channels in the speaking state as the first target channel at the current moment, and determine the channels other than the first target channel from all channels in the speaking state as the second target channel at the current moment.

[0037] Among them, according to the sorting of multiple transient maximum values ​​returned, except for the user with the largest transient maximum value, the sound of other user channels is attenuated to a certain extent. When the more user channels with speaking tags activated at the same time, the more serious the attenuation amplitude.

[0038] The channel with the largest transient maximum value is taken as the "super channel" (i.e., the first target channel), and the other channels are taken as ordinary channels (i.e., the second target channels). Different weights are matched according to the classification of the channels to weight the audio signals of the corresponding channels during mixing.

[0039] When the number of speakers changes or there is a transient maximum value ranking change, that is, the "super channel" changes, you can use the method of directly replacing the weight, or you can use a smooth transition of gradual in and out to ensure the gradual change of the two sounds and reduce the introduction of noise. For example, the first target channel at the current moment is channel 1, and the first target channel at the previous moment is channel 2. It can be seen that the first target channel has changed. At this time, channel 1 is used as the first target channel for weight assignment. If there is still sound in channel 2, it is assigned according to the weight assignment method of the second target channel.

[0040] Step S203, obtaining a first weight corresponding to the first target channel at the current moment and a second weight corresponding to the second target channel at the current moment, and mixing the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain a mixing result.

[0041] Among them, the first target channel corresponds to the first weight, the second target channel corresponds to the second weight, and the relationship between the first target channel and the first weight can be obtained by querying the mapping table, for example, the first weight of the first target channel is 1. In the present application, the first weight is greater than the second weight, and the first weight is used to enhance the audio signal of the corresponding channel, and the second weight is used to attenuate the audio signal of the corresponding channel.

[0042] The relationship between the second target channel and the second weight can be obtained through a mapping table query or through real-time calculation. During real-time calculation, the speech status of all channels needs to be considered. If there are many speaking channels, in order to reduce interference, the weight needs to be set lower to increase the attenuation of the second target channel.

[0043] The audio signal of the first target channel is adjusted using the first weight, and the audio signal of the second target channel is adjusted using the second weight, so as to form a mix and obtain a mix result. For example, the audio signals of the corresponding channels are weightedly summed using the first weight and the second weight to obtain a mix result.

[0044] Optionally, obtaining a second weight of a second target channel corresponding to the current moment includes: Determine, according to the number of all channels in the speaking state of speaking, a second weight of the second target channel at the current moment; The audio signals of the first target channel and the second target channel at the current moment are mixed according to the first weight and the second weight to obtain a mixing result, including: According to the first weight, the audio signal of the first target channel at the current moment is adjusted to obtain the first target signal at the current moment; According to the second weight, the audio signal of the second target channel at the current moment is adjusted to obtain the second target signal at the current moment; The first target signal at the current moment and the second target signal at the current moment are mixed to obtain a mixing result at the current moment.

[0045] The second weight is determined according to the number of channels in the speaking state, and the first weight is assigned a value of 1.

[0046] For example, VAD obtains the speech label (Speech_Label) and transient maximum value (Max_Value) of each channel through real-time statistics. If someone is speaking on channel a, then Speech_Label_a = 1 is output (i.e., the speech status is speaking) and the maximum value Max_Value_a is returned. If someone stops speaking, Speech_Label_a = 0 (i.e., the speech status is not speaking) and Max_Value_a=0. Compare the Max_Value_i of each channel i (i=1,2,3,...,n), and count the sum of the number of channels with speech labels to obtain Speech_Count. If the speech label Speech_Label_M of channel M is 1 and the transient maximum value Max_Value_M is the largest, then the weight of the "super channel" Max_Channel=M is ​​set to 1, and the other channels (i.e., the second target channel) are attenuated (for example, the expression of the attenuated weight is as follows: Other_Channel_pad=floor(Speech_Count / 2)*3, where the floor function is rounded down).

[0047] The embodiment of the present application obtains the audio signals of each channel at the current moment, determines the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signal, determines the channel with the largest transient maximum value from all channels with the speech state of speaking as the first target channel at the current moment, determines the channels other than the first target channel from all channels with the speech state of speaking as the second target channel at the current moment, obtains the first weight corresponding to the first target channel at the current moment, and the second weight corresponding to the second target channel at the current moment, and mixes the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain the mixing result. The classification of channels is achieved by analyzing the audio signal, thereby determining the channel weights that need to be enhanced and the channel weights that need to be attenuated, avoiding the loss of existing audio, and being able to enhance relatively important audio, thereby improving the clarity of the mixing.

[0048] See also Figure 3 , is a flow chart of a multi-channel mixing control method provided in Embodiment 3 of the present application. Figure 3 As shown, after determining the channel with the largest transient maximum value as the first target channel from all channels in the speaking state in the above step S202, the multi-channel mixing control method may include the following steps: Step S301, obtaining the first target channel at the previous moment, and detecting whether the first target channel at the current moment is the same as the first target channel at the previous moment.

[0049] Among them, the interval between two moments is defined as the sampling time interval of the microphone, so that real-time adjustment can be achieved, but the amount of calculation is large and the calculation efficiency is required to be high. Of course, a time interval can also be set. There are S audio frames in the time interval. For example, S is 100. If each audio frame is 3ms, the time interval is 300ms, which can meet the user's auditory requirements.

[0050] If the first target channel at the previous moment is channel 1, and the first channel at the current moment is channel 2, it is obvious that the two channels are not the same. Specifically, the channels are named numerically. When comparing, the names of the two channels are subtracted. If the subtraction result is zero, it means that the two channels are the same. If the subtraction result is not zero, it means that the two channels are different.

[0051] Optionally, detecting whether the first target channel at the current moment is the same as the first target channel at the previous moment includes: Obtaining a first target channel corresponding to each audio frame in a third preset time period from the current moment and before the current moment; The first target channel corresponding to each audio frame is compared with the first target channel at the previous moment. If the first target channels corresponding to all audio frames are the same as the first target channels at the previous moment, it is determined that the first target channel at the current moment is the same as the first target channel at the previous moment. If there is an audio frame corresponding to a first target channel that is different from the first target channel at the previous moment, it is determined that the first target channel at the current moment is different from the first target channel at the previous moment.

[0052] Among them, the first target channel at this moment is set to M, the system saves the last first target channel as H, and sets Result to the value of subtracting the two channels. If Result=0, it means that if the first target channel has changed, no processing is performed. When Result≠0, the first target channel has changed.

[0053] For example, the judgment condition of Result≠0 is set as: Result≠0 for 100 consecutive audio frames, and an audio frame is set to 3ms, and the 300ms is the time length of the third preset time period. In this way, continuous jumps of the first target channel can be avoided, and the position stability can be achieved to a certain extent.

[0054] Step S302, if it is detected that the first target channel at the current moment is different from the first target channel at the previous moment, the weight of the first target channel at the previous moment is smoothly transitioned from the first weight of the first target channel at the previous moment to the second weight corresponding to the preset moment within the first preset time period.

[0055] Among them, when the first target channel changes, the weight corresponding to the channel also changes, and the gradual in and out effect helps to make a smooth transition.

[0056] If the first target channel is channel H and becomes the second target channel, a fade-out operation is performed on the audio signal of the original first target channel H; if the original channel M becomes the first target channel, a fade-in operation is performed on the audio signal of channel M.

[0057] Fade-in and Fade-out adds a smooth transition at the start / end of the audio or when the volume suddenly changes to achieve a soft and smooth effect, which can increase the professionalism of the product to a certain extent. Fade-in: The sound effect starts with a low volume and slowly increases the volume to a higher expected volume; Fade-out: The sound effect starts with a high volume and slowly decreases the volume to a lower expected volume.

[0058] The smooth transition methods include linear smoothing and nonlinear smoothing, among which linear smoothing is as follows: Suppose the fade-in and fade-out duration is len, the fade-in start point (fade-out end point) constant is v, the window function can be Bartlett window or triangular window, and the weight parameter calculation formula is as follows: W(M)=((1-v)*M) / len+v In the formula, 0≤M≤len, when M changes from small to large, it is gradual entry, and when M changes from large to small, it is gradual exit; The nonlinear smoothing is as follows: The nonlinear implementation can use a Hamming window or a Blackman window. The weight calculation formula refers to the function formula of the window function. Taking the Blackman window to achieve gradual entry and exit as an example, the weight parameter calculation formula is: W(M)=0.42-0.5*cos(2*pi*M / len)+0.8*cos(4*pi*M / len) In the formula, 0≤M≤len, when M changes from small to large, it is gradual entry, and when M changes from large to small, it is gradual exit.

[0059] Optionally, after detecting whether the first target channel at the current moment is the same as the first target channel at the previous moment, the method further includes: If it is detected that the first target channel at the current moment is different from the first target channel at the previous moment, the previous weight of the first target channel at the current moment at the previous moment is obtained; Within the second preset time period, the weight of the first target channel at the current moment is smoothly transitioned from the previous weight to the first weight corresponding to the first target channel at the current moment.

[0060] If channel M becomes the first target channel, it is necessary to know its weight at the previous moment, that is, the previous weight, so as to gradually increase the first weight from the previous weight to achieve a gradual-in effect.

[0061] The embodiment of the present application sets a gradual in and out operation to help smooth the transition, making the sound more professional, effectively improving the sound quality, avoiding sudden changes, and improving the competitiveness of the product to a certain extent.

[0062] See also Figure 4 , is a flow chart of a multi-channel mixing control method provided in Embodiment 4 of the present application. Figure 4 As shown, the audio signals of the first target channel and the second target channel at the current moment are mixed in the above step S203 to obtain a mixing result, which may include the following steps: Step S401 , mixing and superimposing audio signals of the first target channel and the second target channel at the current moment to obtain an audio superposition result.

[0063] Step S402: Perform non-uniform compression on the audio superposition result to obtain a compression result as a mixing result.

[0064] Among them, overflow processing uses non-uniform compression method to alleviate the risk of overflow, and its input and output relationship is as follows: Assume that the input digital speech signal is x(t), the output of the audio superposition result is b(t), and the output mixing result is y(t). The calculation formula of the contraction factor mapping a(t) is as follows: ; In the formula, sign() is a symbol operation, Q is the sampling depth (for example, 16 bits), k is an integer power of 2, here the value is 8, j represents the audio signal of the jth channel, Indicates the number of samples of the audio signal.

[0065] The remainder part compression b(t) is calculated as follows:

[0066] The formula for calculating the mixing result is as follows: .

[0067] The embodiment of the present application adopts a non-uniform superposition algorithm and a segmented shrinkage rule to shrink the sampling values ​​after linear superposition to ensure that there is no overflow. The high-intensity signal sampling has a smaller weight to ensure the corresponding shrinkage ratio. The audio superposition factor is independent of the time t and the number of superimposed voice paths. It can meet the needs of voice superposition of more than ten or even higher paths, alleviate the problem of decreased audio clarity when the number of voice paths is large, and can ensure the stability and clarity of the audio quality under the condition of increasing the number of access voice paths.

[0068] Corresponding to the multi-channel mixing control method of the above embodiment, Figure 5The structure block diagram of the multi-channel audio mixing control device provided in the fifth embodiment of the present application is shown. For the convenience of description, only the parts related to the embodiment of the present application are shown.

[0069] See also Figure 5 , the multi-channel mixing control device comprises: The signal detection module 51 is used to obtain the audio signals of each channel at the current moment, and determine the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signals; A channel identification module 52 is used to determine the channel with the largest transient maximum value from all channels in the speaking state as the first target channel at the current moment, and determine the channels other than the first target channel from all channels in the speaking state as the second target channel at the current moment; The mixing processing module 53 is used to obtain a first weight corresponding to the first target channel at the current moment and a second weight corresponding to the second target channel at the current moment, and mix the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain a mixing result.

[0070] Optionally, the multi-channel audio mixing control device further includes: A channel comparison module is used to obtain the first target channel at the previous moment after determining the channel with the largest transient maximum value from all channels in the speaking state as the first target channel, and to detect whether the first target channel at the current moment is the same as the first target channel at the previous moment; The fade-out control module is used to smoothly transition the weight of the first target channel at the previous moment from the first weight of the first target channel at the previous moment to the second weight corresponding to the preset moment within a first preset time period if it is detected that the first target channel at the current moment is different from the first target channel at the previous moment.

[0071] Optionally, the multi-channel audio mixing control device further includes: A previous weight acquisition module is used to, after detecting whether the first target channel at the current moment is the same as the first target channel at the previous moment, if it is detected that the first target channel at the current moment is not the same as the first target channel at the previous moment, acquire the previous weight of the first target channel at the current moment at the previous moment; The gradual entry control module is used to smoothly transition the weight of the first target channel at the current moment from the previous weight to the first weight corresponding to the first target channel at the current moment within a second preset time period.

[0072] Optionally, a channel comparison module includes: A historical channel acquisition unit, used to acquire a first target channel corresponding to each audio frame in a third preset time period from the current moment and before the current moment; a first channel determination unit, configured to compare the first target channel corresponding to each audio frame with the first target channel at the previous moment, and if the first target channels corresponding to all audio frames are the same as the first target channels at the previous moment, determine that the first target channel at the current moment is the same as the first target channel at the previous moment; The second channel determination unit is configured to determine that the first target channel at the current moment is different from the first target channel at the previous moment if there is a first target channel corresponding to the audio frame that is different from the first target channel at the previous moment.

[0073] Optionally, the mixing processing module 53 includes: A second weight calculation unit, used for determining a second weight of a second target channel at a current moment according to the number of all channels in a speaking state; The audio mixing processing module 53 includes: A first adjustment unit, configured to adjust the audio signal of the first target channel at a current moment according to the first weight to obtain a first target signal at the current moment; A second adjustment unit, configured to adjust the audio signal of the second target channel at the current moment according to the second weight to obtain a second target signal at the current moment; The mixing processing unit is used to mix the first target signal at the current moment and the second target signal at the current moment to obtain the mixing result at the current moment.

[0074] Optionally, the mixing processing module 53 includes: A mixing and superposition unit, used to mix and superpose the audio signals of the first target channel and the second target channel at the current moment to obtain an audio superposition result; The audio compression unit is used to perform non-uniform compression on the audio superposition result, and the compression result is the mixing result.

[0075] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0076] Figure 6 This is a schematic diagram of the structure of a computer device provided in Example 6 of the present application. Figure 6 As shown, the computer device of this embodiment includes: at least one processor ( Figure 6Only one is shown), memory, network interface and readable storage medium. Among them, the processor of the computer device is used to provide computing and control capabilities, the memory of the computer device includes a non-volatile storage medium and an internal memory, the non-volatile storage medium stores an operating system, a readable storage medium and a database, the internal memory provides an environment for the operation of the operating system and the readable storage medium in the non-volatile storage medium, the database of the computer device is used to store corresponding calculation parameters, the network interface of the computer device is used to communicate with an external terminal through a network connection, and the readable storage medium is executed by the processor to implement the steps in any of the above-mentioned embodiments of the multi-channel mixing control method.

[0077] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 6 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0078] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0079] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0080] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0081] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.

[0082] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0083] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0084] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0086] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A multi-channel mixing control method, characterized in that: include: Acquire the audio signals of each channel at the current moment, and determine the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signals; Determine, from all channels in the speaking state of speaking, the channel with the largest transient maximum value as the first target channel at the current moment, and determine, from all channels in the speaking state of speaking, channels other than the first target channel as the second target channel at the current moment; A first weight corresponding to the first target channel at the current moment and a second weight corresponding to the second target channel at the current moment are obtained, and audio signals of the first target channel and the second target channel at the current moment are mixed according to the first weight and the second weight to obtain a mixing result.

2. The multi-channel audio mixing control method according to claim 1, characterized in that: The step of determining the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signal comprises: For any channel, detect the volume amplitude of the audio signal of the channel, and if the volume amplitude is greater than a preset amplitude, determine that the speech state of the channel is speech; Perform transient analysis on the audio signal of the channel and determine the transient maximum value as the transient maximum value of the channel.

3. The multi-channel audio mixing control method according to claim 1, characterized in that: After determining the channel with the largest transient maximum value as the first target channel from all channels in the speaking state, the method further includes: Acquire the first target channel at the previous moment, and detect whether the first target channel at the current moment is the same as the first target channel at the previous moment; If it is detected that the first target channel at the current moment is different from the first target channel at the previous moment, the weight of the first target channel at the previous moment is smoothly transitioned from the first weight of the first target channel at the previous moment to the second weight corresponding to the preset moment within a first preset time period.

4. The multi-channel mixing control method according to claim 3, characterized in that: After detecting whether the first target channel at the current moment is the same as the first target channel at the previous moment, the method further includes: If it is detected that the first target channel at the current moment is different from the first target channel at the previous moment, obtaining the previous weight of the first target channel at the current moment at the previous moment; Within a second preset time period, the weight of the first target channel at the current moment is smoothly transitioned from the previous weight to a first weight corresponding to the first target channel at the current moment.

5. The multi-channel mixing control method according to claim 3, characterized in that: The detecting whether the first target channel at the current moment is the same as the first target channel at the previous moment includes: Acquire a first target channel corresponding to each audio frame in a third preset time period from the current moment and before the current moment; The first target channel corresponding to each audio frame is compared with the first target channel at the previous moment, and if the first target channels corresponding to all audio frames are the same as the first target channel at the previous moment, it is determined that the first target channel at the current moment is the same as the first target channel at the previous moment; If there is an audio frame corresponding to a first target channel that is different from the first target channel at the previous moment, it is determined that the first target channel at the current moment is different from the first target channel at the previous moment.

6. The multi-channel mixing control method according to claim 1, characterized in that: The obtaining of the second weight of the second target channel corresponding to the current moment includes: Determine, according to the number of all channels in the speaking state of speaking, a second weight of the second target channel at the current moment; Mixing the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain a mixing result includes: According to the first weight, adjusting the audio signal of the first target channel at the current moment to obtain the first target signal at the current moment; According to the second weight, adjusting the audio signal of the second target channel at the current moment to obtain the second target signal at the current moment; The first target signal at the current moment and the second target signal at the current moment are mixed to obtain a mixing result at the current moment.

7. The multi-channel mixing control method according to any one of claims 1 to 6, characterized in that: The mixing of the audio signals of the first target channel and the second target channel at the current moment to obtain a mixing result includes: Mixing and superimposing the audio signals of the first target channel and the second target channel at the current moment to obtain an audio superposition result; The audio superposition result is non-uniformly compressed to obtain a compression result as a mixing result.

8. A multi-channel mixing control device, characterized in that: include: A signal detection module is used to obtain the audio signals of each channel at the current moment, and determine the speech state of each channel and the transient maximum value of the corresponding audio according to the audio signals; A channel identification module is used to determine the channel with the largest transient maximum value as the first target channel at the current moment from all channels in the speaking state, and determine the channels other than the first target channel as the second target channel at the current moment from all channels in the speaking state; A mixing processing module is used to obtain a first weight corresponding to the first target channel at the current moment and a second weight corresponding to the second target channel at the current moment, and mix the audio signals of the first target channel and the second target channel at the current moment according to the first weight and the second weight to obtain a mixing result.

9. A computer device, characterized in that: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-channel mixing control method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multi-channel mixing control method according to any one of claims 1 to 7 is implemented.

Images