AI-based intelligent noise reduction method, device, equipment and medium for conference audio stream

Abstract

The present application discloses an AI-based intelligent noise reduction method, device, equipment and medium for conference audio streams. The present application relates to the field of audio processing technology. The method includes: obtaining the original audio signal collected by the sender device participating in the conference; determining the target noise type of the environment in which the sender device is located based on the original audio signal; performing data preprocessing on the original audio signal to obtain a first feature quantity, and performing data preprocessing on the target noise type to obtain a second feature quantity; combining the first feature quantity and the second feature quantity to obtain a target feature vector; inputting the target feature vector into the AI ​​noise reduction model for noise reduction to obtain a target audio signal. The present application can improve the quality of audio streams in online conferences.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to an AI-based intelligent noise reduction method, device, equipment and medium for conference audio streams. Background Art

[0002] With the rapid development of Internet technology, online meetings have become an indispensable means of communication in modern devices and are widely used in many fields such as corporate collaboration, distance education and cross-border meetings. Online meetings mainly use audio and video communication technology to enable geographically dispersed meeting participants to interact in real time to improve work efficiency and communication convenience.

[0003] However, in the actual application of online meetings, while the microphone collects the voices of the meeting participants, it will inevitably capture various noises in the speaker's environment, such as background voices, ambient wind noise, and electrical noise, and send the voice and noise emitted by the meeting participants to other meeting participants, resulting in a decline in audio quality and affecting communication effectiveness.

[0004] Therefore, how to improve the audio streaming quality in online meetings is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide an AI-based conference audio stream intelligent noise reduction method, device, equipment and medium, aiming to improve the audio stream quality in online meetings.

[0006] To achieve the above objectives, the present application provides an AI-based intelligent noise reduction method for conference audio streams, and the AI-based intelligent noise reduction method for conference audio streams includes:

[0007] Obtain the original audio signal collected by the sending device participating in the conference;

[0008] Determine a target noise type of the environment in which the sending device is located based on the original audio signal;

[0009] Performing data preprocessing on the original audio signal to obtain a first feature quantity, and performing data preprocessing on the target noise type to obtain a second feature quantity;

[0010] Combining the first feature quantity and the second feature quantity to obtain a target feature vector;

[0011] The target feature vector is input into the AI ​​noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and a real noise type of the audio signal as model input data and using label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal.

[0012] In one embodiment, the step of inputting the target feature vector into an AI noise reduction model for noise reduction to obtain a target audio signal includes:

[0013] Determine a target noise reduction strategy based on the second feature quantity in the target feature vector by using an AI noise reduction model;

[0014] According to the target noise reduction strategy, the first feature quantity in the target feature vector is subjected to noise reduction processing to obtain a third feature quantity, and the third feature quantity is subjected to signal reconstruction processing to obtain a target audio signal.

[0015] In one embodiment, the method further comprises:

[0016] Acquire a sample data set, wherein the sample data set includes a plurality of sample data, and one sample data includes a segment of an audio signal, a real noise type of the audio signal, and a main human voice signal in the audio signal;

[0017] Constructing feature vectors based on the audio signals and real noise types in each piece of sample data, and constructing label data based on the main human voice signals in each piece of sample data;

[0018] The feature vector corresponding to each piece of sample data is used as model input data, and the label data corresponding to each piece of sample data is used as a model training label, and the denoising model to be trained is trained to obtain the AI ​​denoising model.

[0019] In one embodiment, the step of determining the target noise type of the environment in which the sending device is located based on the original audio signal includes:

[0020] Analyzing the original audio signal to obtain sound level fluctuations of the original audio signal;

[0021] If the sound level fluctuation is less than or equal to a preset threshold, determining that the target noise type of the environment in which the sending device is located is a steady-state noise type;

[0022] If the sound level fluctuation is greater than the preset threshold, it is determined that the target noise type of the environment in which the sending device is located is a non-steady-state noise type.

[0023] In one embodiment, after the step of inputting the target feature vector into the AI ​​noise reduction model for noise reduction to obtain the target audio signal, the method further includes:

[0024] The target audio signal is sent to a receiving device participating in the conference, so that the target audio signal is output through a speaker of the receiving device.

[0025] In addition, to achieve the above-mentioned purpose, the present application also provides an AI-based conference audio stream intelligent noise reduction device, and the AI-based conference audio stream intelligent noise reduction device includes:

[0026] An acquisition module is used to obtain the original audio signal collected by the sending device participating in the conference;

[0027] A determination module, configured to determine a target noise type of an environment in which the sending device is located based on the original audio signal;

[0028] A noise reduction module is used to perform data preprocessing on the original audio signal to obtain a first feature value, and to perform data preprocessing on the target noise type to obtain a second feature value; combining the first feature value and the second feature value to obtain a target feature vector; inputting the target feature vector into an AI noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and a real noise type of the audio signal as model input data, and using label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal.

[0029] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a program for implementing an AI-based intelligent noise reduction method for conference audio streams is stored on the computer-readable storage medium. The program for implementing an AI-based intelligent noise reduction method for conference audio streams is executed by a processor to implement the steps of the AI-based intelligent noise reduction method for conference audio streams as described above.

[0030] In addition, to achieve the above-mentioned objectives, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned AI-based conference audio stream intelligent noise reduction method.

[0031] The present application provides an AI-based intelligent noise reduction method for conference audio streams. The present application obtains the original audio signal collected by the sending device participating in the conference, determines the target noise type of the sending device's environment based on the original audio signal, performs data preprocessing on the original audio signal to obtain a first feature quantity, and performs data preprocessing on the target noise type to obtain a second feature quantity, combines the first feature quantity and the second feature quantity to obtain a target feature vector, and finally, inputs the target feature vector into an AI noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and a real noise type of the audio signal as model input data, and using label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal.

[0032] In summary, compared to the traditional method of sending the speaker's voice and the noise in the speaker's environment to other conference participants during online meetings, the present application obtains the original audio signal collected by the sending device of the meeting, and after determining the target noise type of the original audio signal, performs noise reduction processing on the original audio signal based on the target noise type and the AI ​​noise reduction model to obtain the noise-reduced target audio signal, thereby reducing the noise in the audio signal received by other conference participants, improving the audio stream instructions, and thereby improving communication effects and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0035] Figure 1 This is a flow chart of the first embodiment of the AI-based intelligent noise reduction method for conference audio streams of this application;

[0036] Figure 2 This is a schematic diagram of the noise reduction system architecture involved in an embodiment of the AI-based intelligent noise reduction method for conference audio streams of the present application;

[0037] Figure 3 A schematic diagram of a noise reduction process involved in an embodiment of an AI-based intelligent noise reduction method for conference audio streams of the present application;

[0038] Figure 4This is a schematic diagram of the module structure of the AI-based intelligent noise reduction device for conference audio streams in this application;

[0039] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the AI-based intelligent noise reduction method for conference audio streams in the embodiment of the present application.

[0040] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0041] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0042] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0043] The main solution of the present application is: obtaining the original audio signal collected by the sending device participating in the meeting; determining the target noise type of the environment in which the sending device is located based on the original audio signal; performing data preprocessing on the original audio signal to obtain a first feature quantity, and performing data preprocessing on the target noise type to obtain a second feature quantity; combining the first feature quantity and the second feature quantity to obtain a target feature vector; inputting the target feature vector into an AI noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and the actual noise type of the audio signal as model input data, and using label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal.

[0044] Currently, in the actual application of online meetings, while the microphone collects the voices of the meeting participants, it will inevitably capture various noises in the speaker's environment, such as background voices, ambient wind noise, and electrical noise, and send the voice and noise emitted by the meeting participants to other meeting participants, resulting in a decline in audio quality and affecting communication effectiveness.

[0045] This application obtains the original audio signal collected by the sending device of the conference, and after determining the target noise type of the original audio signal, performs noise reduction processing on the original audio signal based on the target noise type and the AI ​​noise reduction model to obtain the noise-reduced target audio signal, thereby reducing the noise in the audio signal received by other conference participants, improving the audio stream instructions, and further improving the communication effect and efficiency.

[0046] It should be noted that the execution subject of the method in each embodiment of the AI-based conference audio stream intelligent noise reduction method of this application can be a noise reduction system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an AI-based conference audio stream intelligent noise reduction device that can achieve the above functions, etc. This embodiment does not specifically limit this. The following takes the noise reduction system as an example of the execution subject to illustrate this embodiment and the following embodiments.

[0047] Based on this, this application proposes a first embodiment of an AI-based conference audio stream intelligent noise reduction method, please refer to Figure 1 The AI-based conference audio stream intelligent noise reduction method includes steps S10 to S30:

[0048] Step S10, obtaining the original audio signal collected by the sending device participating in the conference;

[0049] It should be noted that the above-mentioned sending device refers to the device used by the user who turns on the microphone in the current online meeting. The noise reduction system of this application is connected to each device participating in the meeting, so after the sending device collects the sound signal (hereinafter referred to as the original audio signal for distinction), it automatically sends the collected original audio signal to the noise reduction system, and then the noise reduction system receives the original audio signal.

[0050] Step S20, determining a target noise type of the environment in which the sending device is located based on the original audio signal;

[0051] It should be noted that the noise type of the environment in which the sending device is located (hereinafter referred to as the target noise type for distinction) is determined based on the original audio signal, wherein the target noise type includes steady-state noise and non-steady-state noise types.

[0052] In a specific implementation, the step S20 may include:

[0053] Step S201, analyzing the original audio signal to obtain the sound level fluctuation of the original audio signal;

[0054] Step S202: if the sound level fluctuation is less than or equal to a preset threshold, determining that the target noise type of the environment in which the sending device is located is a steady-state noise type;

[0055] Step S203: If the sound level fluctuation is greater than the preset threshold, it is determined that the target noise type of the environment in which the sending device is located is a non-steady-state noise type.

[0056] It should be noted that the embodiment of the present application does not limit the specific size of the above-mentioned preset threshold, which can be any size used to distinguish between two types of noise. For example, in this embodiment, 3 decibels is preset as the above-mentioned preset threshold. And it should be understood that the above-mentioned original audio signal is a sound signal, that is, a sound signal of a limited time length.

[0057] The original audio signal is framed using a sliding window method to calculate the sound level of each frame, determine the maximum and minimum values ​​in the sound level of each frame, and calculate the difference between the maximum and minimum values ​​as the sound level fluctuation of the original audio signal. Then, it is determined whether the sound level fluctuation is less than or equal to a preset threshold. If so, it is determined that the target noise type of the environment in which the sender device is located is a steady-state noise type. If not, it is determined that the target noise type of the environment in which the sender device is located is a non-steady-state noise type.

[0058] Step S30: Based on the target noise type and the AI ​​noise reduction model, the original audio signal is subjected to noise reduction processing to obtain a target audio signal.

[0059] It should be noted that the above AI (Artificial Intelligence) noise reduction model is used to perform noise reduction processing on the audio signal based on the noise type to obtain a noise-reduced audio signal. The noise reduction system of this application can call the AI ​​noise reduction model.

[0060] Based on the target noise type and the AI ​​noise reduction model, the original audio signal is subjected to noise reduction processing to obtain a noise-reduced audio signal (hereinafter referred to as the target audio signal for distinction).

[0061] In this embodiment, after step S30, the AI-based conference audio stream intelligent noise reduction method of the present application further includes:

[0062] Step A10: sending the target audio signal to a receiving device participating in the conference, so that the target audio signal is output through a speaker of the receiving device.

[0063] It should be noted that the above-mentioned receiving device refers to a device used by other conference participants except the sending device.

[0064] After the target audio signal is obtained, the target audio signal is sent to a receiving device participating in the conference, so that the receiving device plays the target audio signal through a speaker.

[0065] For example, Figure 2The figure shows a schematic diagram of the noise reduction system architecture. The noise reduction system includes an AI noise reduction model. The AI ​​noise reduction model is called to perform noise reduction processing on the original audio signal obtained from the sender device a to obtain the target audio signal. The target audio signal is then sent to other receiving devices b, and the target audio signal is played through the speaker of the receiving device.

[0066] The embodiment of the present application obtains the original audio signal collected by the sending device participating in the meeting, determines the target noise type of the environment in which the sending device is located based on the original audio signal, and performs noise reduction processing on the original audio signal based on the target noise type and an AI noise reduction model to obtain a target audio signal.

[0067] In summary, compared to the traditional method of sending the speaker's voice and the noise in the speaker's environment to other conference participants during online meetings, the embodiment of the present application obtains the original audio signal collected by the sending device of the meeting, and after determining the target noise type of the original audio signal, performs noise reduction processing on the original audio signal based on the target noise type and the AI ​​noise reduction model to obtain the noise-reduced target audio signal, thereby reducing the noise in the audio signal received by other conference participants, improving the audio stream instructions, and further improving the communication effect and efficiency.

[0068] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. On this basis, the step S30 may include:

[0069] Step S301, constructing a target feature vector based on the original audio signal and the target noise type;

[0070] A feature vector (hereinafter referred to as a target feature vector for distinction) is constructed based on the original audio signal and the target noise type.

[0071] In a specific implementation, the step S301 may include:

[0072] Step B10, performing data preprocessing on the original audio signal to obtain a first feature value, and performing data preprocessing on the target noise type to obtain a second feature value;

[0073] Step B20: Combine the first feature quantity and the second feature quantity to obtain a target feature vector.

[0074] Extracting feature quantities (hereinafter referred to as first feature quantities for distinction) from the original audio signal, wherein the first feature quantities include but are not limited to spectrum features and time domain features, and are used to characterize the statistical characteristics and structural information of the original audio signal; and performing data preprocessing on the target noise type to determine a preset feature quantity corresponding to the target noise type (hereinafter referred to as second feature quantities for distinction). Then combining the first feature quantity and the second feature quantity to obtain a target feature vector.

[0075] Step S302: input the target feature vector into an AI noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and a real noise type of the audio signal as model input data and using label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal.

[0076] After obtaining the target feature vector, the target feature vector is input into a pre-trained AI noise reduction model for noise reduction processing to obtain a target audio signal after noise reduction. It should be noted that the AI ​​noise reduction model is trained using a feature vector constructed based on the audio signal and the actual noise type of the audio signal as model input data, and using label data constructed based on the main human voice signal of the audio signal as model training labels. The actual noise type of the audio signal includes but is not limited to a steady-state noise type and a non-steady-state noise type. The main human voice signal of the audio signal refers to the human voice signal with the highest clarity in the audio signal.

[0077] In a specific implementation, the step S302 may include:

[0078] Step C10, determining a target noise reduction strategy based on the second feature quantity in the target feature vector by using an AI noise reduction model;

[0079] Step C20: performing noise reduction processing on the first feature quantity in the target feature vector according to the target noise reduction strategy to obtain a third feature quantity, and performing signal reconstruction processing on the third feature quantity to obtain a target audio signal.

[0080] The target feature vector is subjected to feature separation processing by the AI ​​noise reduction model to obtain a first feature quantity and a second feature quantity, and a preset noise reduction strategy corresponding to the target noise type is determined based on the second feature quantity (hereinafter referred to as the target noise reduction strategy for distinction), and then the AI ​​noise reduction model performs noise reduction processing on the first feature quantity according to the target noise reduction strategy to obtain a processed feature quantity (hereinafter referred to as the third feature quantity for distinction), and then performs information reconstruction processing on the third feature quantity to obtain a target audio signal. It should be noted that the step of obtaining the target audio signal based on the third feature quantity includes: performing feature inverse transformation on the third feature quantity to convert it from the feature space to the representation space of the sound signal, and then performing a signal reconstruction step to further improve the quality of the sound signal to obtain the target audio signal.

[0081] In this embodiment, the AI-based intelligent noise reduction method for conference audio streams of this application also includes:

[0082] Step D10, obtaining a sample data set, wherein the sample data set includes a plurality of sample data, and one sample data includes a segment of an audio signal, a real noise type of the audio signal, and a main human voice signal in the audio signal;

[0083] Step D20, constructing feature vectors based on the audio signals and real noise types in each piece of the sample data, and constructing label data based on the main human voice signals in each piece of the sample data;

[0084] Step D30, using the feature vector corresponding to each piece of sample data as model input data, and using the label data corresponding to each piece of sample data as model training labels, training the denoising model to be trained, and obtaining the AI ​​denoising model.

[0085] A sample data set is constructed based on multiple sample data, wherein each sample data includes an audio signal, the real noise type of the audio signal, and the main human voice signal in the audio signal; after obtaining the sample data set, a feature vector is constructed based on the audio signal and the real noise type in each sample data in the sample data set, that is, data preprocessing is performed on each data in the audio signal and the real noise type to obtain the feature quantity of each data, and each feature quantity is combined to obtain a feature vector, and at the same time, label data is constructed based on the main human voice signal in each sample data; finally, the feature vector corresponding to each sample data is used as the model input data, and the label data corresponding to each sample data is used as the model training label, and the noise reduction model to be trained is trained to obtain an AI noise reduction model. In addition, the embodiment of the present application does not limit the specific type of the above-mentioned AI noise reduction model, which can be a neural network model.

[0086] For example, Figure 3The figure shows a schematic diagram of the noise reduction process. First, the original audio signal collected by the sending device participating in the meeting is obtained; the target noise type is determined based on the original audio signal; then a target feature vector is constructed based on the original audio signal and the target noise type; the target feature vector is input into the AI ​​noise reduction model to obtain a target audio signal; then the target audio signal is sent to other receiving devices to play the target audio signal through the speakers of the receiving devices.

[0087] In this way, the embodiment of the present application obtains an AI noise reduction model for noise reduction processing by pre-training the noise reduction model to be trained based on a sample data set, so as to determine the noise reduction strategy based on the noise type of the current audio signal through the AI ​​noise reduction model, and perform noise reduction on the current audio signal according to the noise reduction strategy to obtain a noise-reduced audio signal, so as to improve the clarity of the audio signal and thus improve the video stream quality of the online meeting.

[0088] The present application also provides an AI-based conference audio stream intelligent noise reduction device, please refer to Figure 4 , the AI-based conference audio stream intelligent noise reduction device includes:

[0089] Acquisition module 10, for acquiring the original audio signal collected by the sending device participating in the meeting;

[0090] A determination module 20, configured to determine a target noise type of an environment in which the sending device is located based on the original audio signal;

[0091] The noise reduction module 30 is used to perform data preprocessing on the original audio signal to obtain a first feature value, and to perform data preprocessing on the target noise type to obtain a second feature value; combining the first feature value and the second feature value to obtain a target feature vector; inputting the target feature vector into the AI ​​noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and a real noise type of the audio signal as model input data, and using label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal.

[0092] Optionally, the noise reduction module 30 is further used for:

[0093] Determine a target noise reduction strategy based on the second feature quantity in the target feature vector by using an AI noise reduction model;

[0094] According to the target noise reduction strategy, the first feature quantity in the target feature vector is subjected to noise reduction processing to obtain a third feature quantity, and the third feature quantity is subjected to signal reconstruction processing to obtain a target audio signal.

[0095] Optionally, the AI-based conference audio stream intelligent noise reduction device further includes a training module, and the training module is used to:

[0096] Acquire a sample data set, wherein the sample data set includes a plurality of sample data, and one sample data includes a segment of an audio signal, a real noise type of the audio signal, and a main human voice signal in the audio signal;

[0097] Constructing feature vectors based on the audio signals and real noise types in each piece of sample data, and constructing label data based on the main human voice signals in each piece of sample data;

[0098] The feature vector corresponding to each piece of sample data is used as model input data, and the label data corresponding to each piece of sample data is used as a model training label, and the denoising model to be trained is trained to obtain the AI ​​denoising model.

[0099] Optionally, the determining module 20 is further configured to:

[0100] Analyzing the original audio signal to obtain sound level fluctuations of the original audio signal;

[0101] If the sound level fluctuation is less than or equal to a preset threshold, determining that the target noise type of the environment in which the sending device is located is a steady-state noise type;

[0102] If the sound level fluctuation is greater than the preset threshold, it is determined that the target noise type of the environment in which the sending device is located is a non-steady-state noise type.

[0103] Optionally, the AI-based conference audio stream intelligent noise reduction device further includes an output module, and the output module is used to:

[0104] The target audio signal is sent to a receiving device participating in the conference, so that the target audio signal is output through a speaker of the receiving device.

[0105] The AI-based intelligent noise reduction device for conference audio streams provided in the embodiment of the present application adopts the AI-based intelligent noise reduction method for conference audio streams in the above embodiment, which can solve the technical problem of how to improve the quality of audio streams in online conferences. Compared with the prior art, the beneficial effects of the AI-based intelligent noise reduction device for conference audio streams provided in the embodiment of the present application are the same as the beneficial effects of the AI-based intelligent noise reduction method for conference audio streams provided in the above embodiment, and the other technical features of the AI-based intelligent noise reduction device for conference audio streams are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0106] The present application provides an AI-based intelligent noise reduction device for conference audio streams, and the AI-based intelligent noise reduction device for conference audio streams includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the AI-based intelligent noise reduction method for conference audio streams in the above-mentioned embodiment one.

[0107] Reference below Figure 5 , which shows a structural schematic diagram of an AI-based conference audio stream intelligent noise reduction device suitable for implementing an embodiment of the present application. The AI-based conference audio stream intelligent noise reduction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The AI-based conference audio stream intelligent noise reduction device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0108] like Figure 5As shown, the AI-based conference audio stream intelligent noise reduction device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the AI-based conference audio stream intelligent noise reduction device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the AI-based conference audio stream intelligent noise reduction device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an AI-based conference audio stream intelligent noise reduction device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or provided alternatively.

[0109] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0110] The AI-based intelligent noise reduction device for conference audio streams provided by this application adopts the AI-based intelligent noise reduction method for conference audio streams in the above-mentioned embodiment, which can solve the technical problem of how to improve the quality of audio streams in online meetings. Compared with the prior art, the beneficial effects of the AI-based intelligent noise reduction device for conference audio streams provided by this application are the same as the beneficial effects of the AI-based intelligent noise reduction method for conference audio streams provided by the above-mentioned embodiment, and the other technical features of the AI-based intelligent noise reduction device for conference audio streams are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0111] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0113] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the AI-based conference audio stream intelligent noise reduction method in the above-mentioned embodiment.

[0114] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0115] The above-mentioned computer-readable storage medium may be included in the AI-based intelligent noise reduction device for conference audio streams; or it may exist independently without being assembled into the AI-based intelligent noise reduction device for conference audio streams.

[0116] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the AI-based conference audio stream intelligent noise reduction device, the AI-based conference audio stream intelligent noise reduction device: obtains the original audio signal collected by the sending device participating in the meeting; determines the target noise type of the environment in which the sending device is located based on the original audio signal; based on the target noise type and the AI ​​noise reduction model, performs noise reduction processing on the original audio signal to obtain the target audio signal.

[0117] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0120] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned AI-based intelligent noise reduction method for conference audio streams, and can solve the technical problem of how to improve the quality of audio streams in online conferences. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the AI-based intelligent noise reduction method for conference audio streams provided in the above-mentioned embodiments, and will not be elaborated here.

[0121] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned AI-based intelligent noise reduction method for conference audio streams.

[0122] The computer program product provided in this application can improve the quality of audio streams in online conferences. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the AI-based conference audio stream intelligent noise reduction method provided in the above embodiment, which will not be repeated here.

[0123] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. An AI-based intelligent noise reduction method for conference audio streams, characterized in that: The AI-based conference audio stream intelligent noise reduction method includes: Obtain the original audio signal collected by the sending device participating in the conference; Analyze the original audio signal to obtain the sound level fluctuation of the original audio signal, wherein the sound level fluctuation refers to the difference between the maximum value and the minimum value of the sound level of each frame of the original audio signal; If the sound level fluctuation is less than or equal to a preset threshold, determining that the target noise type of the environment in which the sending device is located is a steady-state noise type; If the sound level fluctuation is greater than the preset threshold, determining that the target noise type of the environment in which the sending device is located is a non-steady-state noise type; Performing data preprocessing on the original audio signal to obtain a first feature quantity, and performing data preprocessing on the target noise type to obtain a second feature quantity; Combining the first feature quantity and the second feature quantity to obtain a target feature vector; Inputting the target feature vector into an AI noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and a real noise type of the audio signal as model input data and label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal; The step of inputting the target feature vector into the AI ​​noise reduction model for noise reduction to obtain the target audio signal includes: Determine a target noise reduction strategy based on the second feature quantity in the target feature vector by using an AI noise reduction model; According to the target noise reduction strategy, the first feature quantity in the target feature vector is subjected to noise reduction processing to obtain a third feature quantity, and the third feature quantity is subjected to signal reconstruction processing to obtain a target audio signal.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a sample data set, wherein the sample data set includes a plurality of sample data, and one sample data includes a segment of an audio signal, a real noise type of the audio signal, and a main human voice signal in the audio signal; Constructing feature vectors based on the audio signals and real noise types in each piece of sample data, and constructing label data based on the main human voice signals in each piece of sample data; The feature vector corresponding to each piece of sample data is used as model input data, and the label data corresponding to each piece of sample data is used as a model training label, and the denoising model to be trained is trained to obtain the AI ​​denoising model.

3. The method according to any one of claims 1 to 2, characterized in that After the step of inputting the target feature vector into the AI ​​noise reduction model for noise reduction to obtain the target audio signal, the method further includes: The target audio signal is sent to a receiving device participating in the conference, so that the target audio signal is output through a speaker of the receiving device.

4. An AI-based intelligent noise reduction device for conference audio streams, characterized in that: The AI-based conference audio stream intelligent noise reduction device includes: An acquisition module is used to obtain the original audio signal collected by the sending device participating in the conference; A determination module, configured to analyze the original audio signal to obtain the sound level fluctuation of the original audio signal, wherein the sound level fluctuation refers to the difference between the maximum value and the minimum value in the sound level of each frame of the original audio signal; if the sound level fluctuation is less than or equal to a preset threshold, it is determined that the target noise type of the environment in which the sender device is located is a steady-state noise type; if the sound level fluctuation is greater than the preset threshold, it is determined that the target noise type of the environment in which the sender device is located is a non-steady-state noise type; A noise reduction module is used to perform data preprocessing on the original audio signal to obtain a first feature value, and to perform data preprocessing on the target noise type to obtain a second feature value; combine the first feature value and the second feature value to obtain a target feature vector; input the target feature vector into an AI noise reduction model for noise reduction to obtain a target audio signal, wherein the AI ​​noise reduction model is trained using a feature vector constructed based on an audio signal and a real noise type of the audio signal as model input data, and using label data constructed based on a main human voice signal in the audio signal as a model training label, and the main human voice signal is a human voice signal with the highest clarity in the audio signal; determine a target noise reduction strategy based on the second feature value in the target feature vector through the AI ​​noise reduction model; perform noise reduction processing on the first feature value in the target feature vector according to the target noise reduction strategy to obtain a third feature value, and perform signal reconstruction processing on the third feature value to obtain a target audio signal.

5. An AI-based intelligent noise reduction device for conference audio streams, characterized in that: The AI-based conference audio stream intelligent noise reduction device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the AI-based conference audio stream intelligent noise reduction method as described in any one of claims 1 to 3.

6. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the AI-based conference audio stream intelligent noise reduction method as described in any one of claims 1 to 3 are implemented.

Images