Intelligent construction system and method for voice database in noise environment

Abstract

The invention relates to the technical field of voice databases, and discloses an intelligent construction method for a voice database in a noise environment, and the method comprises the following steps: 1, collecting voice and video samples from a plurality of noise environments; 2, preprocessing the collected voice samples, wherein the influence of background noise on voice signals is reduced through frequency spectrum subtraction; the voice data and the video data are collected in the noise environment and then uploaded to the preprocessing module, and the voice signals and the non-voice signals are distinguished by adopting the voice activity detection algorithm, so that silent fragments can be removed, the subsequent voice processing range is reduced, and the voice processing efficiency is improved. The mouth shape features of the speaker in the video data are extracted by using the convolutional neural network algorithm, and the dynamic mouth shape change is captured in combination with the optical flow method, so that the features of the speaker mouth shape can be identified and analyzed more accurately, and the recognition of the voice data is assisted.

Description

Technical Field

[0001] The present invention relates to the technical field of speech databases, and in particular to an intelligent construction system and method of a speech database in a noisy environment. Background Art

[0002] With the rapid development of artificial intelligence and speech recognition technology, the construction of speech database has become one of the key factors to improve the performance of speech recognition system. The traditional method of constructing speech database mainly relies on recording speech data in a relatively quiet environment. Although this method can obtain high-quality speech samples, in actual application scenarios, speech signals are often mixed with various background noises, such as traffic noise, crowd noise, wind noise, etc. These noises will seriously affect the accuracy of speech recognition system.

[0003] Traditional speech database construction methods usually rely on manual screening and annotation, which is not only time-consuming and labor-intensive, but also difficult to cover diverse noise scenarios. In addition, due to the diversity and unpredictability of noise, manual screening often cannot fully simulate the real-world noise environment, resulting in a decline in the performance of speech recognition systems in practical applications.

[0004] In recent years, although some studies have attempted to enhance speech databases by simulating noise, these methods often require complex signal processing technology and professional audio engineers, and the simulated noise is different from the noise in the real environment, making it difficult to fully meet the needs of practical applications. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a system and method for intelligently constructing a speech database in a noisy environment, which has the advantages of being able to automatically identify, collect and process speech data in a noisy environment, improve the performance of the speech recognition system, and solve the problems raised in the background technology.

[0007] (II) Technical solution

[0008] In order to achieve the above-mentioned purpose of being able to automatically recognize, collect and process speech data in a noisy environment and improve the performance of a speech recognition system, the present invention provides the following technical solution: a method for intelligently constructing a speech database in a noisy environment, comprising the following steps:

[0009] Step 1: Collect voice and video samples from various noise environments;

[0010] Step 2: preprocessing the collected speech samples, including reducing the influence of background noise on the speech signal by spectrum subtraction;

[0011] Step 3: Use video data to assist in identifying the speaker's mouth shape, and combine environmental sound analysis to identify the noise type and intensity;

[0012] Step 4: Use the deep learning model to learn the speech features in a noisy environment and perform real-time separation of noise and speech in the collected speech data;

[0013] Step 5: Use natural language processing technology and speech recognition algorithms to automatically transcribe and annotate the separated pure speech;

[0014] Step 6: Verify the transcribed clean voice to correct and ensure data accuracy, and output the clean voice after verification.

[0015] Preferably, the preprocessing in step 2 further includes using a sound activity detection algorithm to distinguish speech signals from non-speech signals to remove silent segments.

[0016] Preferably, in the step 2, an adaptive noise estimation method is used in the spectrum subtraction process, wherein the updating formula of the noise estimation is as follows:

[0017]

[0018] in, is the noise estimate of the current frame, β is the smoothing coefficient, Y(f,t) is the Fourier transform of the current frame, is the noise estimate of the previous frame.

[0019] Preferably, in step three, a convolutional neural network algorithm is used to extract the speaker's mouth shape features in the video data, and an optical flow method is combined to capture the dynamic changes of the mouth shape for auxiliary recognition of the speaker's mouth shape.

[0020] Preferably, the deep learning model in step 4 is a recurrent neural network or a long short-term memory network, which is used to model the time series characteristics of the speech signal.

[0021] Preferably, the detailed algorithm formula for real-time separation of noise and speech in step 4 adopts a deep neural network, and its objective function is expressed as:

[0022]

[0023] Among them, j(θ) is the cost function, s(t) is the target value of pure speech, and f θ (x(t)) is the output of the deep neural network under the parameter θ, x(t) is the noisy input speech signal, λ is the regularization parameter, and r(θ) is the regularization term used to control the complexity of the deep learning model.

[0024] Preferably, the natural language processing technology in step five further includes using a combination of a language model and an acoustic model to improve the accuracy of automatic transcription.

[0025] Preferably, step six further includes iteratively optimizing the deep learning model and natural language processing technology based on feedback of the verification results to steps four and five.

[0026] The present invention also provides an intelligent construction system for a speech database in a noisy environment, the system comprising an acquisition module, the acquisition module being used to acquire speech and video samples in a noisy environment, the acquisition module being connected to a preprocessing module, the preprocessing module being used to preprocess speech data, the preprocessing module being connected to a deep processing module, the deep processing module separating noise and speech from speech data, the deep processing module being connected to a transcription module, the transcription module transcribing and annotating the separated pure speech, the transcription module being connected to a verification module, the verification module being used to analyze and verify the transcribed pure speech, the verification module being connected to an output module, the output module being used to save and output the verified speech data.

[0027] Preferably, the verification module is also connected to the depth processing module, and the output module is configured with a cloud storage interface for uploading the verified voice data directly to a cloud database.

[0028] Compared with the prior art, the present invention provides a system and method for intelligently constructing a speech database in a noisy environment, which has the following beneficial effects:

[0029] 1. The present invention provides an intelligent construction system and method for a speech database in a noisy environment. The system collects speech data and video data in a noisy environment, uploads them to a preprocessing module, and uses a sound activity detection algorithm to distinguish speech signals from non-speech signals. This can remove silent segments and reduce the scope of subsequent speech processing. The convolutional neural network algorithm is used to extract the speaker's lip shape features in the video data, and the optical flow method is combined to capture the dynamic changes of the lip shape, which helps to more accurately identify and analyze the characteristics of the speaker's mouth shape, thereby assisting in identifying speech data, and is beneficial to improving the overall speech recognition and comprehension capabilities in a noisy environment.

[0030] 2. The present invention provides a system and method for intelligently constructing a speech database in a noisy environment. A deep learning model adopts a recurrent neural network or a long short-term memory network to capture the time dependency between data points. The long short-term memory network has a strong long-term memory capability and can retain previous information, which enables it to maintain better performance when processing long sequences of speech data. The deep learning model can automatically adjust its weights according to the input data to better adapt to the variability and dynamic characteristics of the speech signal. Through training, the deep learning model can learn to recognize speech signals in a noisy environment, and then separate noise from pure speech, thereby improving the system's ability and accuracy in processing speech in a noisy environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the process of the present invention;

[0032] Figure 2 Schematic diagram of the system architecture of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1-Figure 2 , a method for intelligently constructing a speech database in a noisy environment, comprising the following steps:

[0035] Step 1: Collect voice and video samples from various noise environments;

[0036] Step 2: preprocessing the collected speech samples, including reducing the influence of background noise on the speech signal by spectrum subtraction;

[0037] Step 3: Use video data to assist in identifying the speaker's mouth shape, and combine environmental sound analysis to identify the noise type and intensity;

[0038] Step 4: Use the deep learning model to learn the speech features in a noisy environment and perform real-time separation of noise and speech in the collected speech data;

[0039] Step 5: Use natural language processing technology and speech recognition algorithms to automatically transcribe and annotate the separated pure speech;

[0040] Step 6: Verify the transcribed clean voice to correct and ensure data accuracy, and output the clean voice after verification.

[0041] Specifically, the preprocessing in step 2 further includes using a sound activity detection algorithm to distinguish speech signals from non-speech signals, and to remove silent segments. The advantage is that by distinguishing speech signals from non-speech signals, such as background noise, silent segments, etc., the amount of data to be processed later can be greatly reduced, thereby speeding up the processing speed and reducing the demand for computing resources. Removing silent segments can reduce noise interference, improve the clarity and intelligibility of speech data, and lay a more solid foundation for subsequent audio analysis and recognition work. An effective sound activity detection algorithm can more accurately locate speech activity segments, allowing the deep learning model to focus on learning speech features rather than irrelevant background noise, thereby improving the accuracy of the entire system. Since the sound activity detection algorithm reduces the amount of data to be processed, more efficient data processing can be achieved with limited hardware resources.

[0042] Specifically, in step 2, an adaptive noise estimation method is used in the spectrum subtraction process, where the updating formula of the noise estimation is as follows:

[0043]

[0044] in, is the noise estimate of the current frame, β is the smoothing coefficient, Y(f,t) is the Fourier transform of the current frame, is the noise estimate of the previous frame. The advantage is that the noise estimate is calculated for the collected speech data through the above formula. The adaptive noise estimation allows the system to dynamically adjust the filter parameters to match the real-time changing noise environment, thereby more effectively reducing the impact of background noise on the speech signal. The adaptive technology enables the system to adapt to different noise conditions and changes. In different environments, it can provide better denoising effects.

[0045] Specifically, in step 3, a convolutional neural network algorithm is used to extract the speaker's mouth shape features in the video data, and the optical flow method is combined to capture the dynamic changes of the mouth shape, which is used to assist in the recognition of the speaker's mouth shape. The advantage is that the convolutional neural network algorithm can learn complex features from the video frames, which helps to more accurately identify and analyze the characteristics of the speaker's mouth shape. By analyzing the dynamic changes of the mouth shape, the system can better adapt to the different speaking speeds, intonations and mouth shape changes of the speaker, which helps to recognize voice information in a noisy environment. The optical flow method can track the changes in the mouth shape in real time, so that the system can instantly recognize the mouth shape in a continuous video stream. Combined with video data and voice signals, the system can understand the speech content from multiple dimensions, which helps to improve the overall voice recognition and understanding capabilities.

[0046] Specifically, the deep learning model in step 4 is a recurrent neural network or a long short-term memory network, which is used to model the time series characteristics of speech signals. The advantage is that recurrent neural networks and long short-term memory networks are specially designed to process time series data. They can capture the time dependency between data points, which is very important for continuous data that changes over time, such as speech signals. Long short-term memory networks have strong long-term memory capabilities and can retain previous information, which enables them to maintain better performance when processing long sequences of speech data. The deep learning model can automatically adjust its weights according to the input data to better adapt to the variability and dynamic characteristics of speech signals. Through training, the deep learning model can learn to recognize speech signals in a noisy environment, providing the basic conditions for the subsequent separation of pure speech and noise.

[0047] Specifically, the detailed algorithm formula for real-time separation of noise and speech in step 4 adopts a deep neural network, and its objective function is expressed as:

[0048]

[0049] Among them, j(θ) is the cost function, s(t) is the target value of pure speech, and f θ (x(t)) is the output of the deep neural network under the parameter θ, x(t) is the noisy input speech signal, λ is the regularization parameter, and r(θ) is the regularization term used to control the complexity of the deep learning model. The advantage is that it is achieved by optimizing a two-part objective function using a deep neural network. The first part of the objective function is the sum of squared errors, which measures the difference between the target value of the pure speech and the output of the neural network; the second part is a regularization term used to control the complexity of the model. The deep learning model can effectively separate pure speech from noisy speech signals in real time through its powerful feature learning ability, time series processing ability, nonlinear modeling ability and efficient optimization strategy, providing clear and pure speech for subsequent speech processing tasks.

[0050] Specifically, the natural language processing technology in step five further includes the use of a combination of a language model and an acoustic model to improve the accuracy of automatic transcription. The advantage is that the language model can understand sentence structure and semantics, while the acoustic model focuses on the physical characteristics of speech. The combination of the two enables the system to better understand the speech content, especially in complex or ambiguous speech input. This combined method can be applied to multiple languages ​​because it relies not only on the physical characteristics of the sound, but also on the inherent rules and structure of the language. These two natural language processing technologies can run in real time, and can respond to the user's speech input instantly and provide fast transcription services.

[0051] Specifically, step six further includes iterative optimization of the deep learning model and natural language processing technology based on the verification results fed back to step four and step five. The advantage is that by continuously adjusting the model parameters according to the new verification results, it can be ensured that the model is gradually improved over time. This feedback mechanism enables the deep learning model to learn the causes of previous errors and avoid similar errors in the future. The automated iterative process reduces the reliance on human experts and reduces maintenance costs. The optimized model is more resistant to the influence of noise and abnormal data, making the output of pure speech more accurate.

[0052] The present invention also provides an intelligent construction system for a speech database in a noisy environment. The system includes an acquisition module, which is used to collect speech and video samples in a noisy environment. The acquisition module is connected to a preprocessing module, which is used to preprocess speech data. The preprocessing module is connected to a deep processing module, which separates noise and speech from speech data. The deep processing module is connected to a transcription module, which transcribes and annotates the separated pure speech. The transcription module is connected to a verification module, which is used to analyze and verify the transcribed pure speech. The verification module is connected to an output module, which is used to save and output the verified speech data.

[0053] Specifically, the verification module is also connected to the deep processing module, and the output module is equipped with a cloud storage interface for uploading the verified voice data directly to the cloud database. The advantage is that this system integrates the entire process from voice data collection to preprocessing, deep processing, transcription, verification and output, realizes automated and integrated voice data processing, can quickly process a large amount of voice data, and improves the efficiency of data construction. The output module is equipped with a cloud storage interface, which can directly upload the verified voice data to the cloud database, which is convenient for data storage, management and sharing; the verification module ensures the quality of the transcribed data, and eliminates errors and inaccurate transcription results through analysis and verification, ensuring the accuracy and availability of the data; the connection between the verification module and the deep processing module allows the system to provide real-time feedback and iterative optimization based on the verification results, and continuously improve the performance of data processing; the cloud storage interface can also cooperate with the existing cloud service security measures to ensure the confidentiality and security of the data.

[0054] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent method for constructing a speech database in a noisy environment, characterized by: The following steps are involved: Step 1: Collect voice and video samples from various noise environments; Step 2: preprocessing the collected speech samples, including reducing the influence of background noise on the speech signal by spectrum subtraction; Step 3: Use video data to assist in identifying the speaker's mouth shape, and combine environmental sound analysis to identify the type and intensity of noise; Step 4: Use the deep learning model to learn the speech features in a noisy environment and perform real-time separation of noise and speech in the collected speech data; Step 5: Use natural language processing technology and speech recognition algorithms to automatically transcribe and annotate the separated pure speech; Step 6: Verify the transcribed clean voice to correct and ensure data accuracy, and output the clean voice after verification.

2. The method for intelligently constructing a speech database in a noisy environment according to claim 1, characterized in that: The preprocessing in step 2 further includes using a sound activity detection algorithm to distinguish speech signals from non-speech signals to remove silent segments.

3. The method for intelligently constructing a speech database in a noisy environment according to claim 1, characterized in that: In the step 2, an adaptive noise estimation method is used in the spectrum subtraction process, wherein the updating formula of the noise estimation is as follows: in, is the noise estimate of the current frame, β is the smoothing coefficient, Y(f,t) is the Fourier transform of the current frame, is the noise estimate of the previous frame.

4. The method for intelligently constructing a speech database in a noisy environment according to claim 1, characterized in that: In the step three, a convolutional neural network algorithm is used to extract the speaker's mouth shape features in the video data, and the optical flow method is combined to capture the dynamic changes of the mouth shape to assist in identifying the speaker's mouth shape.

5. The method for intelligently constructing a speech database in a noisy environment according to claim 1, characterized in that: The deep learning model in step 4 is a recurrent neural network or a long short-term memory network, which is used to model the time series characteristics of the speech signal.

6. The method for intelligently constructing a speech database in a noisy environment according to claim 1, characterized in that: The detailed algorithm formula for real-time separation of noise and speech in step 4 adopts a deep neural network, and its objective function is expressed as: Among them, j(θ) is the cost function, s(t) is the target value of pure speech, and f θ (x(t)) is the output of the deep neural network under the parameter θ, x(t) is the noisy input speech signal, λ is the regularization parameter, and r(θ) is the regularization term used to control the complexity of the deep learning model.

7. The method for intelligently constructing a speech database in a noisy environment according to claim 1, characterized in that: The natural language processing technology in step five further includes using a combination of a language model and an acoustic model to improve the accuracy of automatic transcription.

8. The method for intelligently constructing a speech database in a noisy environment according to claim 1, characterized in that: The step six further includes iteratively optimizing the deep learning model and natural language processing technology based on the verification results fed back to steps four and five.

9. An intelligent construction system for speech database in a noisy environment, characterized by: The system includes an acquisition module, which is used to collect voice and video samples in a noisy environment. The acquisition module is connected to a preprocessing module, which is used to preprocess voice data. The preprocessing module is connected to a deep processing module, which separates noise and voice from voice data. The deep processing module is connected to a transcription module, which transcribes and annotates the separated pure voice. The transcription module is connected to a verification module, which is used to analyze and verify the transcribed pure voice. The verification module is connected to an output module, which is used to save and output the verified voice data.

10. The intelligent construction system of a speech database in a noisy environment according to claim 9, characterized in that: The verification module is also connected to the depth processing module, and the output module is configured with a cloud storage interface for uploading the verified voice data directly to the cloud database.

Images