Voice noise reduction method and device based on environment detection, equipment and storage medium

Abstract

The application relates to an artificial intelligence technology and discloses a voice noise reduction method based on environment detection, which comprises the following steps: obtaining environment sound in a target place collected by a first microphone, training a pre-constructed original detection model to obtain an environment detection model, obtaining noise-containing voice in the target place collected by a second microphone and determining a corresponding sound source direction, obtaining target voice of the sound source direction collected by a third microphone, detecting environment noise in the target voice by using the environment detection model, determining device noise in the target voice by using the noise-containing voice, reducing noise of the target voice by using the environment noise and the device noise, and obtaining a noise-reduced sound signal. The application further discloses a voice noise reduction device based on environment detection, an electronic device and a computer readable storage medium. The application can improve the voice noise reduction effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a voice noise reduction method and device based on environment detection, electronic equipment and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence, intelligent voice recognition technology is widely used, and many practical application scenarios are accompanied by environmental noise, which seriously affects the recognition of voice signals and the quality of voice.

[0003] The existing voice recognition noise reduction method realizes noise reduction by establishing a noise reduction model. However, due to the complexity of the environmental noise on site, ordinary voice noise reduction methods are difficult to eliminate some noise (low-frequency noise, random noise, etc.), resulting in poor noise reduction effect. SUMMARY

[0004] The present application provides a voice noise reduction method, device, equipment and storage medium based on environment detection to improve the effect of voice noise reduction.

[0005] In a first aspect, the present application provides a voice noise reduction method, comprising:

[0006] obtaining the environmental noise in the target place collected by the first microphone, training the pre-constructed original detection model to obtain an environment detection model using the environmental noise;

[0007] obtaining the noise-containing voice in the target place collected by the second microphone, and determining the corresponding sound source direction;

[0008] obtaining the target voice of the sound source direction collected by the third microphone;

[0009] detecting the environmental noise in the target voice using the environment detection model, determining the device noise in the target voice using the noise-containing voice, and reducing the target voice using the environmental noise and the device noise to obtain a noise-reduced voice signal.

[0010] In detail, before obtaining the environmental noise in the target place collected by the first microphone and training the pre-constructed original detection model to obtain an environment detection model using the environmental noise, the method further comprises:

[0011] obtaining a pre-constructed environment voice training set, performing data enhancement processing on the voice data in the environment voice training set to obtain a standard voice training set;

[0012] performing frequency domain conversion on the voice data in the standard voice training set to obtain a mel frequency domain data set;

[0013] The pre-constructed neural network is trained by using the mel-frequency domain data set, to obtain the original detection model.

[0014] In detail, the speech data in the standard speech training set is subjected to frequency domain conversion to obtain a mel-frequency domain data set, which includes:

[0015] The speech data in the standard speech training set is subjected to first frequency domain conversion by using a preset frequency domain function, to obtain a frequency domain data set;

[0016] The frequency domain data set is subjected to second frequency domain conversion by using a preset mel-frequency domain conversion formula, to obtain the mel-frequency domain data set.

[0017] In detail, the environment sound collected by the first microphone in the target place is obtained, and the original detection model is trained by using the environment sound to obtain an environment detection model, which includes:

[0018] A layer of neural network in the original detection model is sequentially selected as a training layer, and the neural networks in the original detection model that are not selected are frozen;

[0019] The training layer is trained by using the environment sound collected by the first microphone in the target place, until the parameters of the training layer converge, to obtain an optimized training layer;

[0020] All the converged optimized training layers are summarized to obtain the environment detection model.

[0021] In detail, the second microphone includes a preset microphone array, wherein the noise-containing speech collected by the second microphone in the target place and the corresponding sound source direction are obtained, which includes:

[0022] A plurality of groups of noise-containing speech collected by the microphone array are obtained;

[0023] A predicted sound source position set is constructed based on the plurality of groups of noise-containing speech, and a preset target function is used to search the predicted sound source position set, to obtain the sound source direction of the noise-containing speech.

[0024] In detail, the predicted sound source position set is constructed based on the plurality of groups of noise-containing speech, which includes:

[0025] The predicted sound source position set is calculated by using the following hypothetical sound source formula:

[0026]

[0027]

[0028] wherein q is a predicted sound source rectangular coordinate vector, M is the number of microphones in the microphone array, R is the distance between the microphone and the predicted sound source, andlm GCC-PHAT function of the noise-containing speech received by the first and the mth microphone, τ lm (q) is the predicted time difference of arrival of the sound source to the first and the mth microphone, r m (r) is the rectangular coordinate vector of the first microphone, r l (r) is the rectangular coordinate vector of the mth microphone, c is the speed of sound in the air.

[0029] In detail, the device noise in the target speech is determined by using the noise-containing speech, the target speech is denoised by using the environmental noise and the device noise, and a denoised sound signal is obtained.

[0030] According to the noise-containing speech and the target speech, a blocking matrix is calculated, a noise signal is obtained according to the target speech and the blocking matrix, and the noise signal is taken as the device noise.

[0031] The environmental noise and the device noise are subtracted from the signal in the target speech, and the denoised sound signal is obtained.

[0032] In a second aspect, the present application provides a voice denoising device based on environmental detection, the device comprising:

[0033] An environmental detection model construction module is configured to obtain environmental sound in a target place collected by a first microphone, and obtain an environmental detection model by training a pre-constructed original detection model using the environmental sound.

[0034] A sound source positioning module is configured to obtain noise-containing speech in the target place collected by a second microphone, and determine a corresponding sound source direction.

[0035] A target speech acquisition module is configured to obtain target speech of the sound source direction collected by a third microphone.

[0036] A voice denoising module is configured to detect environmental noise in the target speech by using the environmental detection model, determine device noise in the target speech by using the noise-containing speech, and denoise the target speech by using the environmental noise and the device noise, to obtain a denoised sound signal.

[0037] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus.

[0038] The memory is configured to store a computer program.

[0039] A processor is configured to execute a program stored in a memory to implement the steps of the method for speech noise reduction based on environment detection according to any one of the first aspect.

[0040] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is configured to be executed by a processor to implement the steps of the method for speech noise reduction based on environment detection according to any one of the first aspect.

[0041] Compared with the prior art, the above technical solution provided by the embodiments of the present application has the following advantages:

[0042] The original detection model is trained by the environment sound in the target place collected by the first microphone, and a more suitable environment detection model for the target place can be obtained, so that the accuracy of the model in detecting the environment sound can be improved. Meanwhile, the target voice in the sound source direction is collected by the third microphone, the environment noise in the target voice is detected by using the environment detection model, the device noise in the target voice is determined by using the noisy voice, the target voice is de-noised by using the environment noise and the device noise, and a de-noised sound signal is obtained, so that the target voice is de-noised from multiple dimensions, and the effect of speech de-noising is further improved. Therefore, the method, device, electronic equipment and computer readable storage medium for speech noise reduction based on environment detection provided by the present application can improve the effect of speech de-noising. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0045] Figure 1 A flowchart of a method for speech noise reduction based on environment detection provided by the embodiments of the present application is shown in the figure.

[0046] Figure 2 A module diagram of a device for speech noise reduction based on environment detection provided by the embodiments of the present application is shown in the figure.

[0047] Figure 3 A structure diagram of an electronic device for implementing the method for speech noise reduction based on environment detection provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0048] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0049] Figure 1 A flowchart of a voice noise reduction method based on environment detection provided by the embodiments of the present application is shown. In the embodiments, the voice noise reduction method based on environment detection comprises:

[0050] S1, obtaining environment sound in a target place collected by a first microphone, and training a pre-constructed original detection model to obtain an environment detection model by using the environment sound.

[0051] In the embodiments of the present application, the first microphone can be an omnidirectional microphone, and the target place can be a closed place (inside a vehicle, a conference room, etc.). For example, in a voice conference scenario, the first microphone is used to continuously collect environment sound of the conference room before and after the conference starts for 24 hours, including speaker voice, air conditioner sound, keyboard sound, etc.

[0052] Specifically, before the step of obtaining environment sound in a target place collected by a first microphone and training a pre-constructed original detection model to obtain an environment detection model by using the environment sound, the method further comprises:

[0053] obtaining a pre-constructed environment voice training set, performing data enhancement processing on voice data in the environment voice training set to obtain a standard voice training set;

[0054] performing frequency domain conversion on the voice data in the standard voice training set to obtain a mel frequency domain data set;

[0055] training a pre-constructed neural network by using the mel frequency domain data set to obtain the original detection model.

[0056] In an optional embodiment of the present application, a Mixup enhancement method can be used to perform data enhancement on the voice data in the environment voice training set to obtain an enhanced data set, and the standard voice training set is obtained by aggregating the environment voice training set and the enhanced data set. The standard training set is used as a training set, and the original training set is used as a verification set.

[0057] In the embodiment of the present application, the frequency domain signal (spectrum) of the voice has stronger consistency than the time domain signal (waveform amplitude), and the spectrum of adjacent frames has continuity after windowing and other processing, so the mel frequency domain data has better predictability than the waveform data. At the same time, the frequency domain data can be processed at the frame level, and the data volume is larger, so it is more conducive to model training.

[0058] In the embodiment of the present application, the environmental voice training set can be an indoor environmental voice set, including knocking sound, mouse click sound, keyboard typing sound, and door opening sound, etc. The pre-constructed neural network can be an improved ResNet50 network, and the improvement is that the first layer of the ResNet50 network is replaced by a batch training (BatchNormalization) layer, and then a convolutional layer (activation function is relu) and an average pooling layer are added, a fully connected layer (activation function is relu), a batch training (BatchNormalization) layer and a last fully connected layer are added after the ResNet50 network. Based on the above improvements, the model training can be accelerated, and the obtained model is more suitable for voice data, improving the accuracy of voice data detection. At the same time, because the ResNet50 network is prone to overfitting during training, the detection accuracy of the model obtained by training is low, and in the embodiment of the present application, the early stopping method (Early Stopping) can be used to terminate the training in advance before the model training enters the overfitting. The early stopping method refers to stopping training if the loss of the validation set does not decrease along with the loss of the training set within a preset training round range. In the embodiment of the present application, the MSE loss function can be used to calculate the loss.

[0059] In detail, the frequency domain conversion of the voice data in the standard voice training set to obtain the mel frequency domain data set includes:

[0060] The voice data in the standard voice training set is converted into frequency domain data set by using a preset frequency domain function for first frequency domain conversion;

[0061] The frequency domain data set is converted into the mel frequency domain data set by using a preset mel frequency domain conversion formula for second frequency domain conversion.

[0062] In an optional embodiment of the present application, the voice data in the standard voice training set is converted into frequency domain data F by using the following function:

[0063]

[0064] Wherein, f(t) is the voice data in the standard voice training set, is the Fourier transform function.

[0065] In an optional embodiment of the present application, the following Mel frequency domain conversion formula is used to convert the frequency domain data set:

[0066]

[0067] wherein f mel is the converted Mel frequency domain data, and F is the frequency domain data.

[0068] In the embodiment of the present application, different frequency domain conversion formulas are used to convert the speech data, which can improve the conversion efficiency of the Mel frequency domain data and speed up the model training.

[0069] Specifically, the environment sound in the target place collected by the first microphone is obtained, and the pre-constructed original detection model is trained using the environment sound to obtain an environment detection model, comprising:

[0070] A neural network in the original detection model is sequentially selected as a training layer, and the neural networks in the original detection model that are not selected are frozen;

[0071] The training layer is trained using the environment sound in the target place collected by the first microphone until the parameters of the training layer converge, and an optimized training layer is obtained;

[0072] All converged optimized training layers are summarized to obtain the environment detection model.

[0073] In the embodiment of the present application, the original detection model is trained based on an open source training set, and the adaptability to environment sound of different scenes (such as indoor meetings) is poor. The original detection model is fine-tuned using the environment sound data in the target place collected by the first microphone, and a more accurate model can be trained.

[0074] S2, obtaining the noise-containing speech in the target place collected by the second microphone, and determining the corresponding sound source direction.

[0075] In an optional embodiment of the present application, the second microphone comprises a preset microphone array.

[0076] Specifically, the noise-containing speech in the target place collected by the second microphone is obtained, and the corresponding sound source direction is determined, comprising:

[0077] A plurality of groups of noise-containing speech collected by the microphone array are obtained;

[0078] A set of predicted sound source positions is constructed based on the plurality of groups of noise-containing speech, a preset target function is used to search the set of predicted sound source positions, and the sound source direction of the noise-containing speech is obtained.

[0079] In the embodiment of the present application, the microphone array comprises at least two omnidirectional microphones or distributed microphones, for example, two omnidirectional microphones are arranged on both sides of a voice conference room, and a distributed microphone array is arranged in a vehicle.

[0080] In an optional embodiment of the present application, the constructing a set of predicted sound source positions based on the plurality of sets of noisy speech comprises:

[0081] The set of predicted sound source positions is calculated by using the following hypothetical sound source formula:

[0082]

[0083]

[0084] wherein q is a straight coordinate vector of a predicted sound source, M is the number of microphones in the microphone array, R lm is a GCC-PHAT function of noisy speech received by the lth and mth microphones, τ lm (q) is a time difference of arrival of the predicted sound source to the lth and mth microphones, r m is a straight coordinate vector of the lth microphone, r i is a straight coordinate vector of the mth microphone, and c is the speed of sound in air.

[0085] In the embodiment of the present application, the sum of the phase transform weighted GCC-PHAT functions of the received signals of all microphone pairs at the hypothetical sound source position is calculated, and the point with the maximum SRP (Steered Response Power) value in the entire sound source space is the sound source position.

[0086] In an optional embodiment of the present application, the preset target function is as follows:

[0087]

[0088] wherein q * is a straight coordinate vector of the sound source, and Q is a preset search space.

[0089] In the embodiment of the present application, the sound source is predicted by using the sounds collected by different microphones in the microphone array, so that the accuracy of sound source positioning can be improved.

[0090] In another optional embodiment of the present application, the GCC method (Generialized Cross-Correlation) can also be used for sound source positioning.

[0091] S3, acquiring target speech of the sound source direction collected by the third microphone.

[0092] In the embodiment of the present application, the third microphone can be a directional microphone with a motor in the base. Since the first microphone is an omnidirectional microphone, it will include a large amount of environmental sound when collecting sound, so using a directional microphone alone to collect the sound of the sound source direction can reduce the influence of environmental sound and improve the noise reduction effect.

[0093] In detail, the third microphone collects the target voice of the sound source direction through the following steps, including:

[0094] Taking the second microphone as the origin, a displacement coordinate system is constructed according to the origin;

[0095] Taking the sound source direction as an initial target point, the initial target point is mapped to the plane where the displacement coordinate system is located to obtain a standard target point;

[0096] The distance from the standard target point to the coordinate axis in the displacement coordinate system is calculated, the position coordinates of the standard target point are calculated according to the distance, and the sound signal of the position coordinates collected by the second microphone is used as the target voice.

[0097] In the embodiment of the present application, since the sound source direction and the displacement coordinate system can not be in the same plane, the initial target point is mapped to the plane where the displacement coordinate system is located for position determination. For example, the distance from the standard target point to the X-axis of the displacement coordinate system is a, and the distance to the Y-axis is b. The position coordinates of the standard target point relative to the origin are (b, a).

[0098] S4, detecting environmental noise in the target voice using the environmental detection model, and determining device noise in the target voice using the noisy voice, and using the environmental noise and the device noise to reduce noise in the target voice to obtain a noise-reduced sound signal.

[0099] In the embodiment of the present application, the device noise in the target voice is determined using the noisy voice, and the environmental noise and the device noise are used to reduce noise in the target voice to obtain a noise-reduced sound signal, including:

[0100] According to the noisy voice and the target voice, a blocking matrix is calculated, a noise signal is obtained according to the target voice and the blocking matrix, and the noise signal is used as the device noise;

[0101] The noise-reduced sound signal is obtained by subtracting the environmental noise and the device noise from the signal in the target voice.

[0102] In the embodiment of the present application, the blocking matrix can be obtained based on the blocking matrix construction method of Fourier orthogonal basis or the blocking matrix construction method based on binomial canceller. Meanwhile, since there are multiple microphones in the target place, the different microphone devices can interfere with each other, thereby forming device noise.

[0103] In the embodiment of the present application, the noisy speech and the target speech are input into a fixed beamformer, a mixed sound signal is output through the fixed beamformer, the blocking matrix is obtained based on the blocking matrix construction method of Fourier orthogonal basis, the signal in the blocking matrix is input into an adaptive noise canceller algorithm, and a noise signal is obtained. The adaptive noise canceller algorithm can include a normalized least mean square (NLMS) algorithm, a minimum mean squared error (MMSE) algorithm, and the like.

[0104] Specifically, the signal in the target speech is used to subtract the environmental noise and the device noise, that is, the speech signal in the target speech is used to subtract the environmental noise and the device noise, and a noise-reduced sound signal is obtained.

[0105] The present application trains the original detection model by the environmental sound in the target place collected by the first microphone, can obtain an environmental detection model more suitable for the target place, and can improve the accuracy of the model in detecting the environmental sound. Meanwhile, the target speech of the sound source direction is collected by the third microphone, the environmental noise in the target speech is detected by using the environmental detection model, the device noise in the target speech is determined by using the noisy speech, the target speech is de-noised by using the environmental noise and the device noise, and a noise-reduced sound signal is obtained. The target speech is de-noised from multiple dimensions, and the effect of speech de-noising is further improved. Therefore, the speech de-noising method based on environmental detection provided by the present application can improve the effect of speech de-noising.

[0106] As shown in Figure 2 The present application provides a module schematic diagram of a speech de-noising device 10 based on environmental detection, which comprises an environmental detection model construction module 11, a sound source positioning module 12, a target speech acquisition module 13, and a speech de-noising module 14.

[0107] The environmental detection model construction module 11 is used to acquire the environmental sound in the target place collected by the first microphone, and train a pre-constructed original detection model to obtain an environmental detection model by using the environmental sound.

[0108] The sound source positioning module 12 is configured to acquire the noisy speech in the target place collected by the second microphone, and determine the corresponding sound source direction;

[0109] The target speech acquisition module 13 is configured to acquire the target speech in the sound source direction collected by the third microphone;

[0110] The speech noise reduction module 14 is configured to detect the environmental noise in the target speech by using the environmental detection model, determine the device noise in the target speech by using the noisy speech, and reduce the noise of the target speech by using the environmental noise and the device noise, to obtain the de-noised sound signal.

[0111] In detail, each module in the speech noise reduction device 10 based on environmental detection in the embodiment of the present application uses the same technical means as the speech noise reduction method based on environmental detection in the embodiment of the present application, and can produce the same technical effects, which will not be described here. Figure 1

[0112] As shown in Figure 3 The present embodiment provides an electronic device, which comprises a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 complete mutual communication through the communication bus 114;

[0113] The memory 113 is configured to store a computer program;

[0114] In an embodiment of the present application, the processor 111 is configured to implement the speech noise reduction method based on environmental detection provided by any one of the preceding method embodiments when executing the program stored in the memory 113, comprising:

[0115] Acquire the environmental sound in the target place collected by the first microphone, and train the pre-constructed original detection model to obtain an environmental detection model by using the environmental sound;

[0116] Acquire the noisy speech in the target place collected by the second microphone, and determine the corresponding sound source direction;

[0117] Acquire the target speech in the sound source direction collected by the third microphone;

[0118] Detect the environmental noise in the target speech by using the environmental detection model, determine the device noise in the target speech by using the noisy speech, and reduce the noise of the target speech by using the environmental noise and the device noise, to obtain the de-noised sound signal.

[0119] ​The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement steps of the environment detection based voice noise reduction method provided by any one of the foregoing method embodiments.

[0120] It should be noted that, in this document, the terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0121] The above is only a specific implementation of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A speech noise reduction method based on environmental detection, characterized in that, The method includes: The ambient sound in the target location is acquired by the first microphone, and the pre-constructed original detection model is trained using the ambient sound to obtain the environment detection model; wherein, the original detection model is a neural network model obtained by using an improved ResNet50 network structure with a BatchNormalization layer, and training based on a standard speech training set generated by Mixup data augmentation, and terminating the training in advance when the loss on the validation set no longer decreases during the training process using an early stopping method. Acquiring noisy speech within the target location captured by the second microphone and determining the corresponding sound source location includes: acquiring... Multiple sets of noisy speech samples are collected by a microphone array; a set of predicted sound source locations is constructed based on the multiple sets of noisy speech samples; the set of predicted sound source locations is searched using a preset objective function to obtain the sound source orientation of the noisy speech samples; wherein, constructing the set of predicted sound source locations based on the multiple sets of noisy speech samples includes: calculating the set of predicted sound source locations using the following hypothetical sound source formula: Where q is the predicted rectangular coordinate vector of the sound source, M is the number of microphones in the microphone array, and R... lm For the GCC-PHAT function of the noisy speech received by the l-th and m-th microphones, τ lm (q) is used to predict the arrival time difference from the sound source to the l-th and m-th microphones, r m Let r be the rectangular coordinate vector of the l-th microphone. l Let be the rectangular coordinate vector of the m-th microphone, and c be the speed of sound in air; Acquire the target speech from the location of the sound source, captured by the third microphone; The environmental noise in the target speech is detected using the environmental detection model, and the device noise in the target speech is determined using the noisy speech. The target speech is then denoised using the environmental noise and the device noise to obtain a denoised sound signal. The environmental noise refers to the indoor environmental sound of the target location, and the device noise refers to the sound formed by mutual interference between different microphone devices in the target location.

2. The speech denoising method based on environmental detection as described in claim 1, characterized in that, Before acquiring the ambient sound within the target location collected by the first microphone and using the ambient sound to train the pre-constructed original detection model to obtain the environmental detection model, the method further includes: Obtain a pre-constructed environmental speech training set, and perform data augmentation processing on the speech data in the environmental speech training set to obtain a standard speech training set. The speech data in the standard speech training set is transformed into the frequency domain to obtain the Mel frequency domain data set; The pre-constructed neural network is trained using the Mel frequency domain data set to obtain the original detection model.

3. The speech denoising method based on environmental detection as described in claim 2, characterized in that, The step of performing frequency domain transformation on the speech data in the standard speech training set to obtain the Mel frequency domain data set includes: The speech data in the standard speech training set is subjected to a first frequency domain transformation using a preset frequency domain function to obtain a frequency domain data set; The frequency domain data set is subjected to a second frequency domain transformation using a preset Mel frequency domain transformation formula to obtain the Mel frequency domain data set.

4. The speech denoising method based on environmental detection as described in claim 2, characterized in that, The step of acquiring ambient sound within the target location collected by the first microphone and using the ambient sound to train a pre-constructed original detection model to obtain an environmental detection model includes: One layer of the neural network in the original detection model is selected as the training layer in sequence, and the neural networks in the original detection model that have not been selected are frozen; The training layer is trained using ambient sound collected by the first microphone in the target location until the parameters of the training layer converge, thus obtaining an optimized training layer. The environment detection model is obtained by summing up all converged optimized training layers.

5. The speech denoising method based on environmental detection as described in claim 1, characterized in that, The step of determining device noise in the target speech using the noisy speech, and denoising the target speech using the environmental noise and the device noise to obtain a denoised sound signal includes: A blocking matrix is ​​calculated based on the noisy speech and the target speech; a noise signal is obtained based on the target speech and the blocking matrix; and the noise signal is used as the device noise. The noise-reduced sound signal is obtained by subtracting the environmental noise and the device noise from the signal in the target speech.

6. A speech noise reduction device based on environmental detection, characterized in that, The device includes: An environmental detection model construction module is used to acquire ambient sound in the target location collected by the first microphone, and use the ambient sound to train a pre-constructed original detection model to obtain an environmental detection model; wherein, the original detection model is a neural network model obtained by using an improved ResNet50 network structure with a BatchNormalization layer, and training based on a standard speech training set generated by Mixup data augmentation, and terminating the training early when the loss on the validation set no longer decreases during the training process using an early stopping method. A sound source localization module is used to acquire noisy speech within the target location collected by a second microphone and determine the corresponding sound source location, including: acquiring multiple sets of noisy speech collected by the microphone array; constructing a predicted sound source location set based on the multiple sets of noisy speech; searching the predicted sound source location set using a preset objective function to obtain the sound source location of the noisy speech; wherein, constructing the predicted sound source location set based on the multiple sets of noisy speech includes: calculating the predicted sound source location set using the following hypothetical sound source formula: Where q is the predicted rectangular coordinate vector of the sound source, M is the number of microphones in the microphone array, and R... lm For the GCC-PHAT function of the noisy speech received by the l-th and m-th microphones, τ lm (q) is used to predict the arrival time difference from the sound source to the l-th and m-th microphones, r m Let r be the rectangular coordinate vector of the l-th microphone. l Let be the rectangular coordinate vector of the m-th microphone, and c be the speed of sound in air; The target speech acquisition module is used to acquire the target speech from the location of the sound source, which is collected by the third microphone; The speech noise reduction module is used to detect environmental noise in the target speech using the environmental detection model, and to determine equipment noise in the target speech using the noisy speech. The target speech is then denoised using the environmental noise and the equipment noise to obtain a denoised sound signal. The environmental noise refers to the indoor environmental sound of the target location, and the equipment noise refers to the sound formed by mutual interference between different microphone devices in the target location.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in a memory, it implements the steps of the speech noise reduction method based on environmental detection as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the speech noise reduction method based on environment detection as described in any one of claims 1-5.

Images