A laboratory strong noise environment speech recognition method based on deep learning
By employing a deep learning method with adaptive gain compensation and dual-branch feature extraction, the robustness problem of speech recognition in high-noise laboratory environments was solved, achieving accurate compensation and feature extraction for laboratory noise and improving the accuracy of speech recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PAI LAB EQUIP CO LTD
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-31
AI Technical Summary
Existing speech recognition technologies struggle to effectively identify high-frequency turbulence, broadband vibrations, and periodic electromagnetic interference unique to laboratory environments, leading to signal distortion and decreased recognition accuracy.
We employ a deep learning approach that combines adaptive gain compensation, dual-branch feature decoupling extraction, and joint front-end and back-end optimization. This approach extracts long-range semantic and local spectral features through Transformer layers and convolutional neural networks, and utilizes a joint optimization loss function for feature space mapping and parameter constraints.
It achieves accurate compensation and feature extraction for laboratory noise in extremely low signal-to-noise ratio environments, improving the robustness and accuracy of speech recognition and effectively recognizing laboratory instructions.
Smart Images
Figure CN122024712B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of speech signal processing technology, specifically to a deep learning-based speech recognition method for noisy laboratory environments. Background Technology
[0002] With the improvement of laboratory automation, non-contact voice control has become a core technology to ensure the safe operation of researchers in high-risk scenarios such as chemical synthesis and biological experiments where their hands are busy. The laboratory environment has unique acoustic heterogeneity, and the complex noise field places stringent requirements on the robustness of the voice recognition system.
[0003] Current speech recognition technology is primarily built upon deep learning architectures, utilizing acoustic and language models to convert signals into text. When dealing with environmental noise, existing technologies generally employ fixed gain adjustment or general spectral subtraction for front-end preprocessing, combined with convolutional neural networks or recurrent neural networks for feature extraction. However, in specific laboratory scenarios, existing solutions exhibit significant drawbacks. First, general gain control cannot identify and compensate for the drastic sound pressure level fluctuations caused by the start-up and shutdown of high-power equipment such as centrifuges and fume hoods, leading to clipping distortion or the speech signal being submerged by background noise. Second, traditional single-branch feature extraction architectures struggle to simultaneously decouple the continuous broadband masking noise generated by laboratory ventilation systems from the localized periodic electromagnetic interference pulses generated by precision instruments, limiting feature representation capabilities. Furthermore, because the front-end enhancement module and the back-end recognition engine are typically optimized independently, the system lacks overall correlation constraints, making it impossible to effectively correct feature space shifts generated during the enhancement process during the recognition stage.
[0004] This invention aims to solve the problem of how to achieve adaptive and accurate gain compensation, parallel decoupling extraction of global and local features, and linkage optimization of front-end and back-end models in the extremely low signal-to-noise ratio environment of laboratory-specific high-frequency turbulence, broadband vibration, and periodic electromagnetic interference, thereby constructing a speech recognition and control method with high robustness and accurate capture of experimental commands.
[0005] To address this, a deep learning-based speech recognition method for noisy laboratory environments is proposed. Summary of the Invention
[0006] The purpose of this invention is to provide a deep learning-based speech recognition method for noisy laboratory environments, which improves the robustness of recognition in extremely low signal-to-noise ratio environments through adaptive gain compensation, dual-branch feature decoupling extraction, and front-end and back-end joint optimization.
[0007] To achieve the above objectives, the present invention provides the following technical solution:
[0008] A deep learning-based speech recognition method for noisy laboratory environments includes:
[0009] The system acquires a speech signal to be identified in a laboratory environment, adjusts the energy envelope of the speech signal using a gain control method, and compensates for the signal gain in real time based on the dynamic changes of the current background sound pressure level. When adjusting the energy envelope, the system extracts the instantaneous energy envelope of the speech signal to be identified. For instantaneous energy envelopes below a preset sensitivity threshold, the system calculates the gain increment using a logarithmic weight with the current average background sound pressure level as the base. For instantaneous energy envelopes exceeding the upper limit of the preset linear dynamic range, the system calculates the overshoot decibel value and performs logarithmic compression mapping, outputting the preprocessed laboratory heterogeneous signal.
[0010] The heterogeneous laboratory signal is simultaneously input into a dual-branch feature extractor. The Transformer layer in the global feature extraction branch is used to capture the long-range semantic dependency features in the heterogeneous laboratory signal through a self-attention mechanism. The convolutional neural network layer in the local feature extraction branch is used to capture the local spectral features in the heterogeneous laboratory signal through a feature convolution operator.
[0011] A dynamic fusion weight vector is calculated through a weighted branch network; the long-range semantic dependency features are multiplied element-wise using the dynamic fusion weight vector; the local spectral features are multiplied element-wise using the inverse remainder of the dynamic fusion weight vector; the enhanced semantic features and the enhanced acoustic features are linearly weighted and summed to construct a dual-branch feature matrix; the dual-branch feature matrix is denoised using a feature space mapping function to eliminate feature space offset caused by laboratory mechanical vibration.
[0012] The denoised dual-branch feature matrix is input into the back-end recognition engine. The joint optimization loss function is used to constrain both the front-end enhancement parameters and the back-end recognition and classification parameters. The feature distribution consistency regularization term in the joint optimization loss function is used to calculate the distribution distance between the dual-branch feature matrix and the preset clean speech feature space. The step value of the gain compensation coefficient is adjusted according to the distribution distance. The target recognition result is then output.
[0013] Preferably, the gain control method includes:
[0014] The leading non-speech segment of the speech signal to be identified is acquired in real time by sliding the sensor. The average background sound pressure level of the current laboratory environment is calculated as a reference value for gain adjustment. The instantaneous energy envelope of the speech signal to be identified is extracted using a time-domain smoothing filter, and the instantaneous energy envelope is compared with a preset upper limit of the linear dynamic range. Based on the fluctuation amplitude of the average background sound pressure level, an adaptive gain compensation coefficient is calculated: when the instantaneous energy envelope is lower than a preset sensitivity threshold, the value of the gain compensation coefficient is increased according to a preset first mapping function; when the difference between the instantaneous energy envelope and the upper limit of the linear dynamic range is less than a preset safety margin threshold, the value of the gain compensation coefficient is decreased according to a preset second mapping function. The calculated gain compensation coefficient is applied to each sampling point of the speech signal to be identified, and the adjusted signal is mapped to a standard amplitude range through an energy normalization function to obtain the heterogeneous laboratory signal.
[0015] Preferably, the average background sound pressure level is calculated as follows: the leading non-speech segment is sampled using a sliding time window, the root mean square amplitude of each frame signal is extracted, and the instantaneous sound pressure level of each frame is calculated using a logarithmic transformation function based on a preset reference sound pressure value; the time-weighted average of all instantaneous sound pressure levels within the sliding time window is performed to obtain the average background sound pressure level; the first mapping function is a gain enhancement function; and the second mapping function is a nonlinear compression function.
[0016] Preferably, the feature extraction process of the dual-branch feature extractor includes: using the Transformer layer in the global feature extraction branch to calculate the association weights of the heterogeneous laboratory signal at different time spans through a multi-head self-attention mechanism, and extracting long-range semantic dependency features reflecting the contextual logic of speech, which are used to compensate for the continuous broadband masking noise generated by the laboratory ventilation system; using the convolutional neural network layer in the local feature extraction branch to perform a local spectral scan of the heterogeneous laboratory signal through a feature convolution operator with a preset stride, and extracting local spectral features reflecting instantaneous acoustic changes, which are used to identify and eliminate periodic electromagnetic interference pulses generated by experimental instruments; performing layer normalization processing on the long-range semantic dependency features and the local spectral features, and using a linear projection operator to map the two features to the same dimensional space.
[0017] Preferably, the construction process of the dual-branch feature matrix includes: concatenating the aligned long-range semantic dependency features with the local spectral features and inputting them into a preset weighted branch network; the weighted branch network calculates a dynamic fusion weight vector that matches the feature dimension through global average pooling and fully connected mapping; multiplying the long-range semantic dependency features element-wise using the dynamic fusion weight vector to obtain the enhanced semantic features; multiplying the local spectral features element-wise using the inverse remainder of the dynamic fusion weight vector to obtain the enhanced acoustic features; and linearly weighting and summing the enhanced semantic features and the enhanced acoustic features to construct the dual-branch feature matrix.
[0018] Preferably, the backend recognition engine includes an acoustic modeling unit and a concatenated temporal classification decoder. The specific recognition process includes: using the acoustic modeling unit to perform a nonlinear transformation on the input dual-branch feature matrix, mapping it to obtain the posterior probability distribution of each time frame on a preset modeling unit; the parameters of the acoustic modeling unit are corrected through gradient backpropagation using the acoustic model loss term in the joint optimization loss function; using the feature distribution consistency regularization term in the joint optimization loss function, the distribution distance between the dual-branch feature matrix and a preset clean speech feature space is calculated, and the step value of the gain compensation coefficient is adjusted according to the distribution distance; the concatenated temporal classification decoder performs path search and deduplication processing on the posterior probability distribution, and, combined with a preset laboratory dictionary and language model, outputs the final target recognition result.
[0019] Preferably, the decoding process of the concatenated temporal classification decoder includes:
[0020] Blank placeholder labels are introduced into a preset set of modeling units. An extended state path space containing the blank placeholder labels is constructed using the posterior probability distribution, aligning the length difference between the dual-branch feature matrix and the target text sequence. A forward-backward algorithm is used to search for all possible decoding paths in the extended state path space, and the probabilities of paths with the same value in the corresponding folded text sequences are accumulated and merged. The original candidate text sequence is extracted by deleting consecutively repeated labels and the blank placeholder labels. The original candidate text sequence is pruned using a beam search algorithm, and a preset laboratory terminology dictionary is introduced for word bias correction. The search width of the beam search algorithm is adaptively adjusted according to the current average background sound pressure level.
[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0022] 1. This invention, by real-time monitoring of background sound pressure level and dynamically adjusting the gain compensation coefficient using first and second mapping functions, can accurately compensate for severe sound pressure fluctuations generated by equipment such as laboratory fume hoods and centrifuges. This not only prevents signal clipping distortion under high-energy noise impact but also avoids weak voice commands being submerged in background noise, ensuring the envelope integrity and stability of the front-end input signal in extremely low signal-to-noise ratio environments.
[0023] 2. By employing a dual-branch architecture—using Transformer layers to capture long-range semantic dependencies and convolutional neural networks to capture local spectral features—this invention overcomes the limitations of traditional single-path feature extraction, which struggles to balance broadband masking and impulse interference. Combined with a dynamic weight fusion mechanism, the system can adaptively adjust the contribution of the two feature paths based on real-time noise characteristics, effectively identifying and eliminating periodic electromagnetic interference specific to the laboratory environment, while compensating for semantic distortions caused by persistent aerodynamic noise.
[0024] 3. This invention utilizes the feature distribution consistency regularization term in the joint optimization loss function to achieve feedback constraints on the front-end enhancement parameters by the back-end recognition, transforming "enhanced feature distortion" into a quantifiable feature space distance for online compensation. Combined with a connection-based temporal classification decoder that adaptively adjusts the search width based on the background sound pressure level, it maintains efficient path search capabilities even when strong interference causes the probability distribution to smooth out, ensuring the accuracy of laboratory terminology recognition. Attached Figure Description
[0025] Figure 1 This is a flowchart of a deep learning-based speech recognition method for noisy laboratory environments proposed in this invention.
[0026] Figure 2 This is a flowchart illustrating a deep learning-based speech recognition method for noisy laboratory environments proposed in this invention.
[0027] Figure 3 This is a flowchart illustrating the decoding process of the linked temporal classification decoder proposed in this invention. Detailed Implementation
[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0029] Example 1
[0030] Please see Figures 1 to 3This invention provides a deep learning-based speech recognition method for noisy laboratory environments, the technical solution of which is as follows:
[0031] A deep learning-based speech recognition method for noisy laboratory environments, such as... Figure 1 - Figure 2 As shown, it includes:
[0032] The system acquires a speech signal to be identified in a laboratory environment, adjusts the energy envelope of the speech signal using a gain control method, and compensates for the signal gain in real time based on the dynamic changes of the current background sound pressure level. When adjusting the energy envelope, the system extracts the instantaneous energy envelope of the speech signal to be identified. For instantaneous energy envelopes below a preset sensitivity threshold, the system calculates the gain increment using a logarithmic weight with the current average background sound pressure level as the base. For instantaneous energy envelopes exceeding the upper limit of the preset linear dynamic range, the system calculates the overshoot decibel value and performs logarithmic compression mapping, outputting the preprocessed laboratory heterogeneous signal.
[0033] The heterogeneous laboratory signal is simultaneously input into a dual-branch feature extractor. The Transformer layer in the global feature extraction branch is used to capture the long-range semantic dependency features in the heterogeneous laboratory signal through a self-attention mechanism. The convolutional neural network layer in the local feature extraction branch is used to capture the local spectral features in the heterogeneous laboratory signal through a feature convolution operator.
[0034] A dynamic fusion weight vector is calculated through a weighted branch network; the long-range semantic dependency features are multiplied element-wise using the dynamic fusion weight vector; the local spectral features are multiplied element-wise using the inverse remainder of the dynamic fusion weight vector; the enhanced semantic features and the enhanced acoustic features are linearly weighted and summed to construct a dual-branch feature matrix; the dual-branch feature matrix is denoised using a feature space mapping function to eliminate feature space offset caused by laboratory mechanical vibration.
[0035] The denoised dual-branch feature matrix is input into the back-end recognition engine. The joint optimization loss function is used to constrain both the front-end enhancement parameters and the back-end recognition and classification parameters. The feature distribution consistency regularization term in the joint optimization loss function is used to calculate the distribution distance between the dual-branch feature matrix and the preset clean speech feature space. The step value of the gain compensation coefficient is adjusted according to the distribution distance. The target recognition result is then output.
[0036] Furthermore, the gain control method includes:
[0037] The leading non-speech segment of the speech signal to be identified is acquired in real time by sliding the sensor. The average background sound pressure level of the current laboratory environment is calculated as a reference value for gain adjustment. The instantaneous energy envelope of the speech signal to be identified is extracted using a time-domain smoothing filter, and the instantaneous energy envelope is compared with a preset upper limit of the linear dynamic range. Based on the fluctuation amplitude of the average background sound pressure level, an adaptive gain compensation coefficient is calculated: when the instantaneous energy envelope is lower than a preset sensitivity threshold, the value of the gain compensation coefficient is increased according to a preset first mapping function; when the difference between the instantaneous energy envelope and the upper limit of the linear dynamic range is less than a preset safety margin threshold, the value of the gain compensation coefficient is decreased according to a preset second mapping function. The calculated gain compensation coefficient is applied to each sampling point of the speech signal to be identified, and the adjusted signal is mapped to a standard amplitude range through an energy normalization function to obtain the heterogeneous laboratory signal.
[0038] In this embodiment, the preset upper limit of the linear dynamic range is set to the maximum quantization value that the system can represent; the safety margin threshold is set to -3dBFS to -1dBFS.
[0039] Furthermore, the average background sound pressure level is calculated as follows: the leading non-speech segment is sampled using a sliding time window, the root mean square amplitude of each frame signal is extracted, and the instantaneous sound pressure level of each frame is calculated using a logarithmic transformation function based on a preset reference sound pressure value; the time-weighted average of all instantaneous sound pressure levels within the sliding time window is performed to obtain the average background sound pressure level; the first mapping function is a gain enhancement function; the second mapping function is a nonlinear compression function.
[0040] Specifically, in the calculation of the gain compensation coefficient, the first mapping function adopts a dynamic gain function based on exponential transformation. The calculation method is as follows: based on the absolute value of the difference between the instantaneous energy envelope and the preset sensitivity threshold, combined with the logarithmic weight with the average background sound pressure level as the base, the gain increment is calculated, and the original compensation coefficient is accumulated with the gain increment.
[0041] The second mapping function adopts a nonlinear compression function based on logarithmic compression. The calculation method is as follows: calculate the overshoot decibel value when the instantaneous energy envelope exceeds the upper limit of the linear dynamic range, and perform logarithmic compression mapping on the overshoot decibel value according to the preset compression ratio factor, thereby reducing the value of the gain compensation coefficient so that the peak value of the adjusted signal is within the upper limit of the linear dynamic range.
[0042] The preset reference sound pressure value is 20 times the standard atmospheric pressure reference value under laboratory conditions. The preset sensitivity threshold is dynamically determined by the standard deviation of the noise floor in the laboratory under silent conditions; for sudden high-energy pulse noise in the laboratory, a larger compression ratio factor (such as 8:1) is used to forcibly reduce the overshoot energy.
[0043] The specific implementation of the energy normalization function is as follows: search for the absolute maximum value of the adjusted signal within a preset frame duration, divide the sampling point value by the absolute maximum value and multiply by a preset standard amplitude constant to achieve amplitude normalization of the signal across the entire range; the preset frame duration is set to 20ms to 50ms.
[0044] The standard amplitude constant is set to 0.9 or 0.95; a small margin is reserved for backend feature extraction to prevent overflow caused by the cumulative error of floating-point operations.
[0045] This invention introduces adaptive gain control based on background sound pressure level fluctuations, utilizing exponential enhancement and logarithmic compression functions to achieve precise signal amplitude normalization. This effectively suppresses transient impulse noise and broadband sound pressure fluctuations caused by high-power laboratory equipment, prevents signal clipping, and improves the ability to capture weak commands.
[0046] Furthermore, the feature extraction process of the dual-branch feature extractor includes: using the Transformer layer in the global feature extraction branch to calculate the association weights of the heterogeneous laboratory signal at different time spans through a multi-head self-attention mechanism, and extracting long-range semantic dependency features reflecting the contextual logic of speech to compensate for the persistent broadband masking noise generated by the laboratory ventilation system; using the convolutional neural network layer in the local feature extraction branch to perform a local spectral scan of the heterogeneous laboratory signal through a feature convolution operator with a preset stride, and extracting local spectral features reflecting instantaneous acoustic changes to identify and eliminate periodic electromagnetic interference pulses generated by experimental instruments; performing layer normalization on the long-range semantic dependency features and the local spectral features, and using a linear projection operator to map the two feature paths to the same dimensional space.
[0047] In this embodiment, before entering the dual-branch feature extractor, the laboratory heterogeneous signal is first processed by frame segmentation and windowing, and 64-dimensional or 80-dimensional Mel frequency cepstral coefficients are extracted as input features.
[0048] The Transformer layer in the global feature extraction branch is composed of 4 to 12 stacked encoder blocks, wherein the number of heads in the multi-head self-attention mechanism is set to 4 or 8 to capture long-range temporal correlations spanning more than 500ms in different subspaces, thereby distinguishing speech envelopes from smooth ventilation system noise.
[0049] The convolutional neural network layer in the local feature extraction branch adopts a depth-separable convolutional structure. The feature convolution operator uses a two-dimensional convolutional kernel with a size of 3×3 or 5×5, and its preset stride is set to 1 or 2. By limiting the size of the receptive field, the convolutional neural network layer focuses on the fine-grained texture of energy abrupt changes in the time-frequency spectrum, thereby locating and suppressing periodic electromagnetic pulse interference with energy higher than a preset energy threshold.
[0050] To ensure that the dimensions of the two features are perfectly aligned, in the local feature extraction branch, the output sequence after convolution is linearly interpolated according to the preset stride of the feature convolution operator, so that its length in the time axis dimension is consistent with the length of the sequence output by the global feature extraction branch.
[0051] The linear projection operator adopts a fully connected layer structure, and its output dimension is aligned with the hidden layer dimension of the Transformer layer. When performing the layer normalization process, the mean and variance of each batch of features are calculated for standardization mapping to eliminate the problem of inconsistent numerical magnitudes between the two branches due to differences in the calculation operators.
[0052] This invention achieves precise decoupling from complex laboratory noise by constructing a dual-branch architecture using Transformer and depthwise separable convolution. Multi-head self-attention is utilized to capture long-range semantics to compensate for broadband masking in the ventilation system, while small receptive field feature convolution identifies and eliminates transient electromagnetic pulses from experimental instruments. The combination of relative position encoding and linear interpolation alignment techniques ensures the consistency of feature dimensions between global logic and local details. Layer normalization eliminates numerical magnitude differences, improving the stability of heterogeneous feature fusion and the accuracy of semantic extraction in complex noisy environments.
[0053] Furthermore, the construction process of the dual-branch feature matrix includes: concatenating the aligned long-range semantic dependency features with the local spectral features and inputting them into a preset weighted branch network; the weighted branch network calculates a dynamic fusion weight vector that matches the feature dimension through global average pooling and fully connected mapping; multiplying the long-range semantic dependency features element-wise using the dynamic fusion weight vector to obtain the enhanced semantic features; multiplying the local spectral features element-wise using the inverse remainder of the dynamic fusion weight vector to obtain the enhanced acoustic features; and linearly weighting and summing the enhanced semantic features and the enhanced acoustic features to construct the dual-branch feature matrix.
[0054] The weighted branch network has a Sigmoid activation function at the output of the fully connected mapping, which normalizes the value of each element of the dynamically fused weight vector to the interval [0, 1].
[0055] The specific calculation method of the inverse remainder is as follows: subtract the dynamic fusion weight vector from the unit constant vector to obtain the corresponding complement weight vector; the element-wise multiplication process includes: applying the dynamic fusion weight vector to the long-range semantic dependency feature, and applying the complement weight vector to the local spectral feature, so as to achieve energy complementarity between the two features in the feature space.
[0056] The dynamic weight fusion follows the mathematical relationship described below: The generated weight distribution is used to competitively adjust the two feature paths, ensuring that when the noise energy in the laboratory environment increases, the weight of semantic features with long-range masking compensation capability is automatically increased, while the contribution of local acoustic features is correspondingly reduced. The linear weighted summation refers to matrix superposition of the enhanced semantic features and enhanced acoustic features, followed by amplitude correction using a learnable scaling factor, ultimately constructing the dual-branch feature matrix.
[0057] This invention achieves nonlinear adaptive fusion of semantic and acoustic features through a weighted branch network and a sigmoid activation mechanism. By utilizing a competitive adjustment logic formed by dynamic fusion weights and their complement weight vectors, energy complementarity and conservation in the feature space are ensured. Under conditions of drastic fluctuations in laboratory noise, this embodiment can automatically balance the contribution of long-range occlusion compensation and local acoustic details. Combined with linear weighted summation of learnable scaling factors, numerical bias in the feature fusion process is effectively eliminated.
[0058] Furthermore, the backend recognition engine includes an acoustic modeling unit and a concatenated temporal classification decoder. The specific recognition process includes: using the acoustic modeling unit to perform a nonlinear transformation on the input dual-branch feature matrix to map the posterior probability distribution of each time frame on the preset modeling unit; the parameters of the acoustic modeling unit are corrected by gradient backpropagation through the acoustic model loss term in the joint optimization loss function; using the feature distribution consistency regularization term in the joint optimization loss function, the distribution distance between the dual-branch feature matrix and the preset clean speech feature space is calculated, and the step value of the gain compensation coefficient is adjusted according to the distribution distance; the concatenated temporal classification decoder is used to perform path search and deduplication processing on the posterior probability distribution, and combined with the preset laboratory dictionary and language model, the final target recognition result is output.
[0059] The preset modeling unit refers to the smallest recognizable set of symbols represented by the acoustic model output layer. In this embodiment, the preset modeling unit adopts Chinese syllables, Chinese characters, or a set of modeling units.
[0060] Furthermore, such as Figure 3 As shown, the decoding process of the concatenated temporal classification decoder includes:
[0061] Blank placeholder labels are introduced into a preset set of modeling units. An extended state path space containing the blank placeholder labels is constructed using the posterior probability distribution, aligning the length difference between the dual-branch feature matrix and the target text sequence. A forward-backward algorithm is used to search for all possible decoding paths in the extended state path space, and the probabilities of paths with the same value in the corresponding folded text sequences are accumulated and merged. The original candidate text sequence is extracted by deleting consecutively repeated labels and the blank placeholder labels. The original candidate text sequence is pruned using a beam search algorithm, and a preset laboratory terminology dictionary is introduced for word bias correction. The search width of the beam search algorithm is adaptively adjusted according to the current average background sound pressure level.
[0062] The joint optimization loss function Constructed using a weighted summation method, the formula is expressed as follows: ;in, For the acoustic model loss term, The preset smoothing factor, This is the consistency regularization term for the feature distribution. The value range is usually between 0.1 and 0.5. In a noisy laboratory environment, it is recommended to set the initial value to 0.2.
[0063] The distribution distance is quantized using Euclidean distance. The specific logic for adjusting the step value of the gain compensation coefficient is as follows: when the distribution distance exceeds a preset distance threshold, the degree of feature offset is determined to increase, and the step value is increased proportionally to accelerate the feedback compensation efficiency of the front-end gain; conversely, the step value is decreased to maintain gain stability; the distance threshold is set based on 70%-80% of the average Euclidean distance between "clean speech features" and "typical laboratory interference speech features" during the development phase.
[0064] In the connected temporal classification decoder, the rule for adaptively adjusting the search width of the beam search algorithm is as follows: establish a positive correlation mapping function between the average background sound pressure level and the search width W; when the average background sound pressure level increases, the entropy value of the posterior probability distribution increases (i.e., the uncertainty increases), and the value of W is increased to cover more candidate decoding paths.
[0065] The word bias correction employs a score-weighted mechanism. For candidate sequences matching the laboratory terminology dictionary, a preset bias weight factor is accumulated on the logarithmic probability of their paths, thereby enabling priority decoding and output of unfamiliar experimental instructions. The bias weight factor is typically set to 1.5 to 3.0.
[0066] This invention achieves coordinated constraints on front-end and back-end parameters by constructing a joint optimization loss function that includes a smoothing factor. It utilizes Euclidean distance to quantify the feature space offset in real time and dynamically adjusts the gain step value, constructing a closed-loop feedback mechanism from recognition error to front-end enhancement, effectively correcting distribution distortion caused by strong laboratory noise. This improves the recognition accuracy and recall rate of obscure scientific terms under extremely low signal-to-noise ratios.
[0067] This invention improves recognition performance in noisy laboratory environments through gain control, dual-branch feature extraction, and joint front-end and back-end optimization. First, gain control with dynamic sound pressure level compensation effectively solves the problems of signal distortion and command overload caused by high-power equipment in the laboratory. Second, a dual-branch parallel architecture of Transformer and convolutional neural network achieves deep decoupling of long-range semantic logic and local spectral details, simultaneously suppressing persistent ventilation noise and transient instrument interference. Finally, a joint optimization loss function is used to construct a front-end and back-end collaborative constraint mechanism, eliminating feature space shift caused by mechanical vibration.
[0068] Example 2
[0069] This embodiment uses the operation of a centrifuge and the interference of the exhaust system in a chemical analysis laboratory as a background to explain in detail the operation process of the deep learning-based speech recognition method for noisy laboratory environments.
[0070] During sample centrifugation, the transient impact noise generated by the equipment startup and the continuous broadband airflow noise from the fume hood constitute a complex heterogeneous signal in the laboratory. The system first calculates the average background sound pressure level of the laboratory by real-time sliding acquisition of the leading non-speech segment. When the high-speed rotation of the centrifuge causes a significant increase in ambient sound pressure, the gain control method uses a time-domain smoothing filter to extract the instantaneous energy envelope of the speech signal and compares it with a preset upper limit of the linear dynamic range. If the instantaneous energy envelope approaches saturation due to equipment noise, the system reduces the gain compensation coefficient according to a preset second mapping function to prevent signal clipping distortion; conversely, when the airflow noise masks weak commands, the first mapping function is used to increase the gain, ensuring that the output heterogeneous laboratory signal has a stable energy distribution.
[0071] The preprocessed signal enters a dual-branch feature extractor. The global feature extraction branch utilizes Transformer layers and their multi-head self-attention mechanism to identify persistent broadband masking noise patterns generated by the ventilation system across the time dimension, and extracts long-range semantic dependency features from them. Simultaneously, the local feature extraction branch performs a local scan of the spectrum through convolutional neural network layers, using feature convolution operators to accurately capture and eliminate instantaneous energy mutations caused by electromagnetic pulses from the experimental instruments, extracting local spectral features. Both feature paths are then normalized and linearly projected to the same dimensional space.
[0072] During the feature fusion stage, the weighted branch network calculates the dynamic fusion weight vector based on the real-time noise intensity distribution using global average pooling. When exhaust noise dominates, the system automatically increases the dynamic fusion weights to enhance the weights of long-range semantically dependent features, and uses the inverse remainder to reduce the contribution of acoustic features accordingly. By constructing a dual-branch feature matrix through linear weighted summation, the system effectively offsets the feature space shift caused by laboratory mechanical vibrations.
[0073] The denoised feature matrix is input into the backend recognition engine, where the acoustic modeling unit maps it to a posterior probability distribution. During decoding, the system monitors the high average background sound pressure level in real time and adaptively increases the beam search width of the sequential classification decoder to address uncertainties caused by noise interference. The decoder incorporates a laboratory dictionary containing technical terms such as "perchloric acid" and "gradient dilution," using a score-weighted mechanism for word bias correction. Simultaneously, the joint optimization loss function calculates the distribution distance based on the feature distribution consistency regularization term, providing real-time feedback and adjusting the step value of the front-end gain compensation coefficient. Ultimately, under strong noise interference, it accurately outputs the target recognition result "Start centrifuge number three."
[0074] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A deep learning-based speech recognition method for noisy laboratory environments, characterized in that, include: The system acquires a speech signal to be identified in a laboratory environment, adjusts the energy envelope of the speech signal using a gain control method, and compensates for the signal gain in real time based on the dynamic changes of the current background sound pressure level. When adjusting the energy envelope, the system extracts the instantaneous energy envelope of the speech signal to be identified. For instantaneous energy envelopes below a preset sensitivity threshold, the system calculates the gain increment using a logarithmic weight with the current average background sound pressure level as the base. For instantaneous energy envelopes exceeding the upper limit of the preset linear dynamic range, the system calculates the overshoot decibel value and performs logarithmic compression mapping, outputting the preprocessed laboratory heterogeneous signal. The heterogeneous laboratory signal is simultaneously input into a dual-branch feature extractor. The Transformer layer in the global feature extraction branch is used to capture the long-range semantic dependency features in the heterogeneous laboratory signal through a self-attention mechanism. The convolutional neural network layer in the local feature extraction branch is used to capture the local spectral features in the heterogeneous laboratory signal through a feature convolution operator. A dynamic fusion weight vector is calculated through a weighted branch network; the long-range semantic dependency features are multiplied element-wise using the dynamic fusion weight vector; the local spectral features are multiplied element-wise using the inverse remainder of the dynamic fusion weight vector; the enhanced semantic features and the enhanced acoustic features are linearly weighted and summed to construct a dual-branch feature matrix; the dual-branch feature matrix is denoised using a feature space mapping function to eliminate feature space offset caused by laboratory mechanical vibration. The denoised dual-branch feature matrix is input into the back-end recognition engine. The joint optimization loss function is used to constrain both the front-end enhancement parameters and the back-end recognition and classification parameters. The feature distribution consistency regularization term in the joint optimization loss function is used to calculate the distribution distance between the dual-branch feature matrix and the preset clean speech feature space. The step value of the gain compensation coefficient is adjusted according to the distribution distance. The target recognition result is then output.
2. The method for speech recognition in a noisy laboratory environment based on deep learning according to claim 1, characterized in that, The gain control method includes: The leading non-speech segment of the speech signal to be identified is acquired in real time by sliding the sensor. The average background sound pressure level of the current laboratory environment is calculated as a reference value for gain adjustment. The instantaneous energy envelope of the speech signal to be identified is extracted using a time-domain smoothing filter, and the instantaneous energy envelope is compared with a preset upper limit of the linear dynamic range. Based on the fluctuation amplitude of the average background sound pressure level, an adaptive gain compensation coefficient is calculated: when the instantaneous energy envelope is lower than a preset sensitivity threshold, the value of the gain compensation coefficient is increased according to a preset first mapping function; when the difference between the instantaneous energy envelope and the upper limit of the linear dynamic range is less than a preset safety margin threshold, the value of the gain compensation coefficient is decreased according to a preset second mapping function. The calculated gain compensation coefficient is applied to each sampling point of the speech signal to be identified, and the adjusted signal is mapped to a standard amplitude range through an energy normalization function to obtain the heterogeneous laboratory signal.
3. The method for speech recognition in a noisy laboratory environment based on deep learning according to claim 2, characterized in that, The average background sound pressure level is calculated as follows: the leading non-speech segment is sampled using a sliding time window, the root mean square amplitude of each frame signal is extracted, and the instantaneous sound pressure level of each frame is calculated using a logarithmic transformation function based on a preset reference sound pressure value; the time-weighted average of all instantaneous sound pressure levels within the sliding time window is performed to obtain the average background sound pressure level; the first mapping function is a gain enhancement function; the second mapping function is a nonlinear compression function.
4. The method for speech recognition in a noisy laboratory environment based on deep learning according to claim 1, characterized in that, The feature extraction process of the dual-branch feature extractor includes: using the Transformer layer in the global feature extraction branch, calculating the association weights of the heterogeneous laboratory signal at different time spans through a multi-head self-attention mechanism, and extracting long-range semantic dependency features reflecting the contextual logic of speech to compensate for the persistent broadband masking noise generated by the laboratory ventilation system; using the convolutional neural network layer in the local feature extraction branch, performing a local spectral scan of the heterogeneous laboratory signal through a feature convolution operator with a preset stride to extract local spectral features reflecting instantaneous acoustic changes to identify and eliminate periodic electromagnetic interference pulses generated by experimental instruments; performing layer normalization on the long-range semantic dependency features and the local spectral features, and mapping the two feature paths to the same dimensional space using a linear projection operator.
5. The deep learning-based speech recognition method for noisy laboratory environments according to claim 4, characterized in that, The construction process of the dual-branch feature matrix includes: concatenating the aligned long-range semantic dependency features with the local spectral features and inputting them into a preset weighted branch network; the weighted branch network calculates a dynamic fusion weight vector that matches the feature dimension through global average pooling and fully connected mapping; multiplying the long-range semantic dependency features element-wise using the dynamic fusion weight vector to obtain the enhanced semantic features; multiplying the local spectral features element-wise using the inverse remainder of the dynamic fusion weight vector to obtain the enhanced acoustic features; and linearly weighting and summing the enhanced semantic features and the enhanced acoustic features to construct the dual-branch feature matrix.
6. The deep learning-based speech recognition method for noisy laboratory environments according to claim 3, characterized in that, The backend recognition engine includes an acoustic modeling unit and a concatenated temporal classification decoder. The specific recognition process includes: using the acoustic modeling unit to perform a nonlinear transformation on the input dual-branch feature matrix, mapping it to obtain the posterior probability distribution of each time frame on a preset modeling unit; the parameters of the acoustic modeling unit are corrected through gradient backpropagation using the acoustic model loss term in the joint optimization loss function; using the feature distribution consistency regularization term in the joint optimization loss function, the distribution distance between the dual-branch feature matrix and a preset clean speech feature space is calculated, and the step value of the gain compensation coefficient is adjusted according to the distribution distance; the concatenated temporal classification decoder performs path search and deduplication processing on the posterior probability distribution, and, combined with a preset laboratory dictionary and language model, outputs the final target recognition result.
7. The method for speech recognition in a noisy laboratory environment based on deep learning according to claim 6, characterized in that, The decoding process of the concatenated temporal classification decoder includes: Blank placeholder labels are introduced into a preset set of modeling units. An extended state path space containing the blank placeholder labels is constructed using the posterior probability distribution, aligning the length difference between the dual-branch feature matrix and the target text sequence. A forward-backward algorithm is used to search for all possible decoding paths in the extended state path space, and the probabilities of paths with the same value in the corresponding folded text sequences are accumulated and merged. The original candidate text sequence is extracted by deleting consecutively repeated labels and the blank placeholder labels. The original candidate text sequence is pruned using a beam search algorithm, and a preset laboratory terminology dictionary is introduced for word bias correction. The search width of the beam search algorithm is adaptively adjusted according to the current average background sound pressure level.