Low frame rate noise adaptive speech enhancement system, method and server

Abstract

The application relates to a low-frame-rate noise adaptive speech enhancement system, method and server, and belongs to the technical field of communication, which comprises the following modules: a speech acquisition and preprocessing module, which is used for receiving and processing a call speech signal; a noise perception module, which is used for extracting and constructing feature information of background noise to form a noise environment feature vector based on the low-frame-rate speech data; a sequence memory enhancement module, which is used for combining the low-frame-rate speech data and the noise environment feature vector to perform noise adaptive speech enhancement processing; a semantic recognition and classification module, which is used for transcribing the enhanced speech signal into text and performing semantic classification; a rule decision module, which is used for performing corresponding interception operations according to preset interception rules; and an optimization module, which is used for updating and optimizing the interception rules and the classification model according to interception result feedback and combining artificial quality inspection marking. The system can be used for adaptive and fine speech enhancement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a low frame rate noise adaptive voice enhancement system, method and server. BACKGROUND

[0002] With the development of telecommunication network technology, the use frequency of voice communication in people's daily work and life continues to rise, especially in high-frequency interaction scenarios such as finance, government affairs, express delivery and logistics, and calls have gradually become an important information transmission medium. The security and recognition accuracy of user voice calls face great challenges.

[0003] The existing voice enhancement recognition system usually adopts a fixed frame rate end-to-end recognition model, combines a pre-constructed risk semantic library and an artificial rule system, and performs keyword matching or label discrimination after text transcription of the voice signal. Some systems introduce a deep learning model to classify the text, and then determine whether to prompt or intercept processing through a preset strategy.

[0004] However, the existing voice enhancement recognition system has the following shortcomings:

[0005] 1. Since the traditional voice enhancement recognition system usually adopts a fixed frame rate end-to-end recognition model, combined with a pre-set rule library and semantic library for voice analysis and interception decision. Once the rules are set, it is difficult to dynamically adjust according to the changes in the actual environment or voice model. Therefore, when the model drifts or the voice environment changes, the reaction of the existing system is usually lagging behind, lacking flexibility and self-adaptive ability;

[0006] 2. Many traditional voice recognition systems only passively record error information when a mistake is made, and do not correct it through an effective feedback mechanism. This results in the system being unable to automatically use the error information for model optimization, and the utilization rate of the misjudgment samples is extremely low, making it difficult to achieve continuous self-improvement of the system;

[0007] 3. In many existing systems, there is a lack of close coupling between the semantic classification model and the rule decision mechanism. Even if the classification model accurately identifies the voice content, the rule judgment is often separated from the model output, resulting in a lag or inconsistency in rule updates in actual application, making it difficult to cope with new voice data features;

[0008] 4. Traditional voice enhancement systems usually rely on overall retraining to handle misjudgment problems, rather than making corrections through fine-grained sample-level adjustments. This method is inefficient, costly, and cannot effectively deal with gray areas in voice content (such as accent, speech rate or ambiguous expression).

[0009] Therefore, there is an urgent need for a low frame rate noise adaptive voice enhancement system to solve the above problems. SUMMARY

[0010] The present application intends to provide a low-frame-rate noise adaptive speech enhancement system of a deep feedback sequence memory network to solve the disconnection between speech enhancement, noise suppression, semantic classification and rule decision in the prior art, and the accumulated misjudgment and performance decline in long-term operation of the system.

[0011] According to a first aspect of the present application, a low-frame-rate noise adaptive speech enhancement system is provided, comprising:

[0012] a speech collection and preprocessing module for receiving and processing a call speech signal, generating low-frame-rate speech data through downsampling and frame-level processing;

[0013] a noise perception module for extracting and constructing feature information of background noise based on the low-frame-rate speech data to form a noise environment feature vector;

[0014] a sequence memory enhancement module for combining the low-frame-rate speech data and the noise environment feature vector, and performing noise adaptive speech enhancement processing through a deep feedback sequence memory network to output an enhanced speech signal;

[0015] a semantic recognition and classification module for transcribing the enhanced speech signal into text and performing semantic classification to provide a basis for subsequent rule decision;

[0016] a rule decision module for executing a corresponding interception operation according to the label of semantic classification and a preset interception rule, and feeding back the interception result to an optimization module;

[0017] an optimization module for updating and optimizing the interception rule and the classification model according to the interception result feedback and combining artificial quality inspection annotation.

[0018] Preferably, the speech collection and preprocessing module comprises a speech signal receiving unit, a downsampling unit and a frame-level processing unit, wherein:

[0019] the speech signal receiving unit is configured to receive original audio signals in a communication link in real time;

[0020] the downsampling unit is configured to reduce the sampling rate of the original audio signals and convert them into low-frame-rate audio data at 16 kHz;

[0021] the frame-level processing unit is configured to divide the low-frame-rate audio data into small frames and perform windowing processing.

[0022] Preferably, the noise perception module comprises a noise feature extraction unit, a noise environment vector generation unit and a noise dynamic control unit, wherein:

[0023] The noise feature extraction unit is configured to convert each frame of speech signal into a frequency domain representation by using a short-time Fourier transform method, and extract a frequency domain noise feature;

[0024] The noise environment vector generation unit is configured to generate a noise environment feature vector based on the frequency domain noise feature, so as to drive noise adaptive processing of the sequence memory enhancement module.

[0025] The noise dynamic control unit is configured to adjust a noise suppression strategy in a speech enhancement process according to the noise environment feature vector.

[0026] Preferably, the sequence memory enhancement module comprises a deep feedback network unit, an enhanced signal generation unit, and a reinforcement learning unit, wherein:

[0027] The deep feedback network unit is configured to input low-frame-rate speech data and the noise environment feature vector into a deep feedback neural network for processing, and utilize long short-term memory capability to maintain context continuity of the speech.

[0028] The enhanced signal generation unit is configured to generate an enhanced speech signal according to an output of the deep feedback network.

[0029] The reinforcement learning unit is configured to adjust weights in the network according to a feedback of the enhanced speech signal, and optimize parameters of the network through a reinforcement learning mechanism.

[0030] Preferably, the semantic recognition and classification module comprises a speech-to-text unit, a model semantic classification unit, and a classification label output unit, wherein:

[0031] The speech-to-text unit is configured to transcribe the enhanced speech signal into text, and use a Transformer-based self-attention model for context decoding.

[0032] The model semantic classification unit is configured to extract deep semantic features from the text through multiple deep learning models, and perform semantic classification.

[0033] The classification label output unit is configured to convert a semantic classification result into a call category label, as an input of the rule decision module.

[0034] Preferably, the rule decision module comprises a rule matching unit, an interception judgment unit, and an operation feedback unit, wherein:

[0035] The rule matching unit is configured to match a label of the call category with a preset rule in a rule library, and judge whether interception needs to be performed.

[0036] The interception judgment unit is configured to judge whether an interception condition is met, and trigger a corresponding interception operation.

[0037] The operation feedback unit is configured to feed back the interception result to the optimization module for continuously optimizing the interception strategy.

[0038] Preferably, the optimization module comprises an error feedback processing unit, a model updating unit, and a cause feedback unit, wherein:

[0039] The error feedback processing unit is configured to receive and analyze the misjudgment feedback information from the interception operation, and generate error adjustment information.

[0040] The model updating unit is configured to update the interception rules and the classification model based on the error adjustment information, and form an error adjustment result.

[0041] The cause feedback unit is configured to feed back the error adjustment result to the system to optimize the speech enhancement and classification model.

[0042] According to a second aspect of the present application, a low-frame-rate noise adaptive speech enhancement method using the above low-frame-rate noise adaptive speech enhancement system is provided, comprising the following steps:

[0043] Real-time receiving of original speech signals, after down-sampling and frame-level processing, to generate low-frame-rate speech data;

[0044] Based on the low-frame-rate speech data, feature information of background noise is extracted and constructed to form a noise environment feature vector;

[0045] Combining the low-frame-rate speech data and the noise environment feature vector, noise adaptive speech enhancement processing is performed through a deep feedback sequence memory network to output an enhanced speech signal;

[0046] The enhanced speech signal is transcribed into text and subjected to semantic classification to provide a basis for subsequent rule decision-making;

[0047] According to the label of semantic classification and the preset interception rules, a corresponding interception operation is performed to generate an interception operation feedback result;

[0048] According to the interception operation feedback result, the interception rules and the classification model are updated and optimized in combination with artificial quality inspection labeling.

[0049] According to a third aspect of the present application, a server is provided, comprising a memory and at least one processor;

[0050] The memory stores a computer program, and the at least one processor executes the computer program stored in the memory to implement the above low-frame-rate noise adaptive speech enhancement method.

[0051] According to a fourth aspect of the present application, a computer readable storage medium is provided, in which a computer program is stored, the computer program being executed to implement the low frame rate noise adaptive speech enhancement method described above.

[0052] According to an embodiment of the present application, the low frame rate noise adaptive speech enhancement system has the following advantages:

[0053] By introducing a deep feedback sequence memory network (DFSMN) and a dynamic feedback mechanism, the system can adaptively adjust according to the changes in the noise environment during the speech enhancement process;

[0054] By using a differential misjudgment feedback matrix modeling mechanism, the system can optimize the semantic classification model and the interception rules through structured feedback of misjudgment samples. The feedback information of each misjudgment is used for system optimization, thereby improving the recognition accuracy and robustness;

[0055] By introducing a feedback-based dynamic adjustment mechanism in the system, the semantic classification model and the rule decision mechanism can work closely together. When the classification model is updated, the rule base and the interception strategy are optimized simultaneously, ensuring that the system can adjust the decision strategy in a timely manner when facing new speech samples and situations, and maintaining overall consistency and stability;

[0056] By using a negative example-driven incremental fine-tuning method, the system can automatically generate a negative sample training set and fine-tune the model boundary through a lightweight loss function;

[0057] By introducing error source analysis and error cause back-annotation mechanisms, the problem of misjudgment accumulation and rule redundancy expansion in the long-term operation of existing systems is effectively solved;

[0058] And by constructing a graph annotation model and an error distribution matrix, the system can record and trace the causal chain of each misjudgment, providing visual error data management. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a structural block diagram of a low frame rate noise adaptive speech enhancement system in an embodiment of the present application;

[0060] Figure 2 is a flowchart of a low frame rate noise adaptive speech enhancement method in an embodiment of the present application;

[0061] Figure 3 is a structural block diagram of a server in an embodiment of the present application. DETAILED DESCRIPTION

[0062] It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other in the case of no conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0063] It should be noted that the above detailed description is exemplary and is intended to provide further description of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0064] As shown in the low frame rate noise adaptive speech enhancement system in an embodiment of the present application, the system comprises: Figure 1

[0065] A speech acquisition and preprocessing module for receiving and processing the speech signal, generating low frame rate speech data through down-sampling and frame-level processing;

[0066] The module comprises a speech signal receiving unit, a down-sampling unit and a frame-level processing unit, wherein:

[0067] The speech signal receiving unit: receives the original audio signal in the communication link in real time.

[0068] The down-sampling unit: reduces the sampling rate of the original audio signal, converts it into low frame rate audio data of 16 kHz, reduces data redundancy and improves the efficiency of subsequent processing.

[0069] The frame-level processing unit: divides the low frame rate audio data into small frames and performs windowing processing (such as Hamming windowing), to ensure that the signal will not leak in the frequency domain.

[0070] A noise perception module for extracting and constructing feature information of background noise based on the low frame rate speech data to form a noise environment feature vector;

[0071] The module comprises a noise feature extraction unit, a noise environment vector generation unit and a noise dynamic control unit, wherein:

[0072] The noise feature extraction unit: converts each frame of speech signal into a frequency domain representation using the short-time Fourier transform (STFT) method, extracts the frequency domain noise feature, and the specific process is as follows:

[0073] For a discrete-time speech signal , the calculation formula of the short-time Fourier transform is:

[0074] ;

[0075] In the formula, represents the STFT result, i.e. the result of the short-time Fourier transform of the speech signal at the th time frame and the ​a complex value at a frequency bin; is the frame index; is the frequency index; is the original discrete-time speech signal; is a window function (e.g. Hamming window, Hanning window, etc.) used to frame the signal to reduce spectral leakage; is the number of points of Fourier transform (FFT size) which determines the frequency resolution; is the hop size which represents the interval of sample points between adjacent frames; is the complex exponential term representing a sinusoid with frequency ; is the imaginary unit.

[0076] For a speech signal , it is first segmented into a series of short-time frames, which can be overlapped or non-overlapped. The length of each frame is usually sample points;

[0077] A window function is applied to each frame, which helps to smooth the edges of the frame and reduce spectral leakage in the Fourier transform. A common window function is the Hamming window, whose formula is:

[0078] ;

[0079] A discrete Fourier transform (DFT) is performed on each windowed frame signal to calculate its spectrum, usually using the efficient fast Fourier transform (FFT) algorithm, so that each frame signal is converted from time-domain representation to frequency-domain representation, i.e. the spectral information of the frame is obtained.

[0080] Noise environment vector generation unit: generate noise environment feature vector based on frequency domain noise features, used to drive noise adaptive processing of sequence memory enhancement module, specifically:

[0081] The optimal estimate is obtained by minimizing the mean square error between the estimated noise power spectrum and the true noise power spectrum. First, perform voice activity detection (VAD), which aims to distinguish between speech frames and pure noise frames.

[0082] VAD can be based on multiple features, such as:

[0083] Energy threshold: the energy of pure noise frames is usually lower than that of speech frames;

[0084] Zero-crossing rate: the zero-crossing rate characteristics of speech and noise are different;

[0085] Spectral entropy: the spectrum of pure noise is usually flatter than speech, with higher spectral entropy;

[0086] Machine learning / deep learning model: train a classifier to determine whether a frame contains speech or not;

[0087] Label each frame as speech or non-speech (noise).

[0088] When the VAD determines that the current frame is non-speech (noise), use the spectral information of this frame to update the estimate of the noise. A smoothing update strategy is adopted to prevent the noise estimate from fluctuating wildly. A common update formula is:

[0089]

[0090] where is the estimated noise power at the frame and frequency ; is the noise power estimate of the previous frame; is the power spectrum (obtained by squaring the magnitude of the STFT) of the current frame (determined by the VAD as a noise frame) at frequency ; is a smoothing factor that determines how fast the update is. A larger results in a more stable noise estimate but slower response to changes in the noise; a smaller has the opposite effect.

[0091] At the beginning of the system, it is usually assumed that the first few frames are pure noise, or a pre-set initial noise power spectrum is used. The estimated noise power spectrum is taken as the output, forming a feature vector that can be used by the "sequence memory enhancement module".

[0092] The vector composed of the updated for all frequencies is the current noise environment feature vector. The dimension of this vector is usually (if considering the single-sided spectrum of real signals), where is the number of FFT points.

[0093] Noise dynamic control unit: based on the noise environment feature vector, adjust the noise suppression strategy during speech enhancement, specifically:

[0094] The noise environment feature vector mainly contains the estimated noise power spectrum In traditional speech enhancement algorithms (such as Wiener filtering, spectral subtraction), the strength of noise suppression is usually controlled by a gain factor (or called suppression factor, noise reduction factor). For the noise dynamic control unit, this gain factor can be dynamically calculated or adjusted according to the real-time noise power spectrum.​

[0095] When the noise power is large, stronger suppression is needed; when the noise power is small, weaker suppression is needed to avoid excessive noise reduction leading to speech distortion.

[0096] Global gain adjustment: by calculating the average noise energy or signal-to-noise ratio (SNR) of the entire frequency band or some key frequency bands:

[0097] If the average noise energy is high or the SNR is low: the noise dynamic control unit instructs the speech enhancement module to adopt a more aggressive noise suppression strategy, such as reducing the overall gain factor or increasing the noise suppression-related loss weight in the deep feedback sequence memory network.

[0098] If the average noise energy is low or the SNR is high: the noise dynamic control unit instructs to adopt a more moderate suppression strategy, such as increasing the overall gain factor, to reduce damage to the speech itself.

[0099] Frequency band independent gain adjustment: Since the distribution of noise at different frequencies may be different, more fine-tuned adjustment is to independently calculate the gain factor for each frequency point or frequency band.

[0100] In Wiener filtering, the gain function is usually related to the local signal-to-noise ratio. Although the deep feedback network does not directly use the Wiener filtering formula, its internal learning mechanism can perceive this local signal-to-noise ratio through the noise environment feature vector, thereby performing different degrees of suppression on different frequencies at the feature level.

[0101] To provide explicit indications about the noise energy of each frequency point in the current frame (i.e. ) for the deep feedback network, the deep feedback network will learn that at a certain frequency , the larger it is, the more suppression should be applied at that frequency.

[0102] This module can extract noise features based on real-time low-frame-rate speech data and generate a noise environment feature vector to drive the noise suppression strategy in the speech enhancement process. In this way, the system can dynamically adapt to different noise environments and perform adaptive noise suppression, significantly improving the quality of speech in complex backgrounds.

[0103] The sequence memory enhancement module is used to combine low-frame-rate speech data and noise environment feature vectors to perform noise adaptive speech enhancement processing through a deep feedback sequence memory network, and output an enhanced speech signal.

[0104] This module includes a deep feedback network unit, an enhanced signal generation unit, and a reinforcement learning unit, wherein:

[0105] Deep feedback network unit: input low frame rate speech data and noise environment feature vector into deep feedback neural network for processing, use long short term memory (LSTM-like mechanism) to maintain the context continuity of speech, specifically:

[0106] By receiving low frame rate speech data from the speech acquisition and preprocessing module. Usually the frequency domain representation of the speech signal (such as the amplitude spectrum or power spectrum after short-time Fourier transform), or further extracted features (such as mel spectrum).

[0107] At the same time, receive noise environment feature vector from noise perception module. This vector contains detailed information of the current background noise (for example, real-time estimated noise power spectrum, noise type classification).

[0108] By concatenating low frame rate speech data and noise environment feature vector, a joint input feature vector is formed, which is sent to the input layer of DFSMN. The deep feedback network can perceive the speech content and the noise environment it is in at the same time, so as to make targeted enhancement.

[0109] The fused input feature vector is first passed through multiple deep feedforward layers (usually fully connected layers or one-dimensional convolutional layers) of DFSMN. By performing complex nonlinear mapping, the original low frame rate speech and noise features are converted into higher level representations. After the deep feature extraction layer, DFSMN integrates one or more modules with long short term memory capability. The working mode of LSTM / GRU includes input gate, forget gate, output gate, which intelligently controls the inflow, outflow of information and the update of memory cell state:

[0110] Selective memory: decide which old information needs to be retained for future, which can be forgotten.

[0111] Selective update: decide how the current input updates the memory state.

[0112] Selective output: decide which part of the memory state can be output to the next layer.

[0113] Through these memory cells, DFSMN can effectively handle the time dependence of speech signals. Even if the speech is low frame rate, the sparsity of information on the time axis increases, and the memory cells can still encode the speech features and noise characteristics (such as noise persistence mode, speech speed, tone change) of the previous frame or even more distant frames into their internal state. When processing the current frame, the network combines the current input and these "memorized" context information to make more accurate enhancement decisions.

[0114] Enhanced signal generation unit: generates an enhanced speech signal according to the output of the deep feedback network.

[0115] Reinforcement learning unit: adjusts the weights in the deep feedback network according to the feedback of the enhanced speech signal, optimizes the parameters of the network through a reinforcement learning mechanism, and improves the enhancement effect.

[0116] This module uses a deep feedback sequence memory network (DFSMN) to perform low frame rate speech data enhancement processing. Through the long short-term memory (LSTM-like) mechanism, the speech signal is modeled for long-term dependence. Especially under low frame rate conditions, it can effectively maintain the context information and clarity of the speech, and improve the quality and effect of speech enhancement.

[0117] The semantic recognition and classification module is used to transcribe the enhanced speech signal into text and perform semantic classification, providing a basis for subsequent rule decision-making.

[0118] This module includes a speech-to-text unit, a model semantic classification unit, and a classification label output unit, wherein:

[0119] Speech-to-text unit: transcribes the enhanced speech signal into text, and uses a self-attention model based on Transformer for context decoding.

[0120] Model semantic classification unit: extracts deep semantic features from the text through multiple deep learning models (such as BERT, CNN+LSTM, etc.), and performs semantic classification.

[0121] Classification label output unit: converts the semantic classification result into a call category label, which is used as input for the rule decision-making module.

[0122] Rule decision-making module, for executing corresponding interception operations according to the semantic classification label and the preset interception rules, and feeding back the interception result to the optimization module.

[0123] This module matches the call type (such as advertising sales, normal business, etc.) recognized by the classification label output unit with the preset interception rule library.

[0124] For example, if it is identified as normal business, it matches the normal call release rule and does not perform interception.

[0125] This module includes a rule matching unit, an interception judgment unit, and an operation feedback unit, wherein:

[0126] Rule matching unit: matches the call category label with the preset rules in the rule library to determine whether interception needs to be performed.

[0127] Interception judgment unit: judge whether the interception condition is met, and trigger the corresponding interception operation.

[0128] Operation feedback unit: feedback the interception result to the optimization module for continuous optimization of the interception strategy.

[0129] The system realizes the bidirectional feedback and dynamic optimization of the model and rules by closely combining semantic recognition and rule decision. When executing the interception operation, the system not only feeds back the interception result, but also continuously optimizes the interception rules and classification model, thereby maintaining the adaptability and accuracy of the system and avoiding the rule lag and misjudgment problems in traditional methods.

[0130] Optimization module, for updating and optimizing the interception rules and classification model according to the interception result feedback combined with artificial quality inspection annotation.

[0131] The module includes an error feedback processing unit, a model updating unit, and a mistake feedback unit, wherein:

[0132] Error feedback processing unit: receives and analyzes the misjudgment feedback from the interception operation to generate error adjustment information.

[0133] Model updating unit: update the interception rules and classification model based on the error feedback information to improve the self-adaptive ability of the system.

[0134] Mistake feedback unit: feedback the error adjustment result to the system to optimize the speech enhancement and classification model.

[0135] The module structures the misjudgment feedback result through a differential misjudgment feedback matrix modeling mechanism. The feedback information can be fed back to the classification model and interception rules for optimization. After each misjudgment, the system adjusts the rule weight and updates the classification model to avoid misjudgment accumulation and improve the stability and recognition accuracy of the system during long-term operation.

[0136] The system introduces a negative example-driven incremental fine-tuning method, which can generate a negative sample set from misintercepted samples for small-scale lightweight adjustment, avoiding the high cost and low efficiency of overall retraining in traditional solutions and improving the training efficiency and processing capacity of the system.

[0137] As shown in Figure 2 The low-frame-rate noise adaptive speech enhancement method using the above low-frame-rate noise adaptive speech enhancement system includes the following steps:

[0138] Step 1: Real-time receive the original speech signal, after down-sampling and frame-level processing, generate low-frame-rate speech data;

[0139] In this step, after receiving the original audio signal in the communication link in real time, the original audio signal is down-sampled and converted into low-frame-rate audio data at 16 kHz, reducing data redundancy and improving the efficiency of subsequent processing. The low-frame-rate audio data is then divided into small frames and windowed (e.g., Hamming windowed) to ensure that the signal does not leak in the frequency domain.

[0140] Step 2: Based on the low-frame-rate speech data, extract and construct the feature information of the background noise to form a noise environment feature vector.

[0141] In this step, the short-time Fourier transform (STFT) method is used to convert each frame of speech signal into a frequency domain representation, and the frequency domain noise features are extracted. Then, based on the frequency domain noise features, a noise environment feature vector is generated to drive the noise adaptive processing of the sequence memory enhancement module. According to the noise environment feature vector, the noise suppression strategy in the speech enhancement process is adjusted.

[0142] Step 3: Combine the low-frame-rate speech data and the noise environment feature vector, and perform noise adaptive speech enhancement processing through a deep feedback sequence memory network to output the enhanced speech signal.

[0143] In this step, the low-frame-rate speech data and the noise environment feature vector are input into a deep feedback neural network for processing, which uses a long short-term memory (LSTM-like mechanism) to maintain the context continuity of the speech. According to the output of the deep feedback network, an enhanced speech signal is generated. Finally, the weights in the network are adjusted according to the enhanced speech signal feedback, and the parameters of the network are optimized through reinforcement learning mechanism to improve the enhancement effect.

[0144] Step 4: Transcribe the enhanced speech signal into text and perform semantic classification to provide a basis for subsequent rule decision-making.

[0145] In this step, the enhanced speech signal is transcribed into text, and a self-attention model based on Transformer is used for context decoding. Then, multiple deep learning models (such as BERT, CNN+LSTM, etc.) are used to extract deep semantic features from the text and perform semantic classification. The semantic classification result is converted into a call category label, which is used as the input of the rule decision module.

[0146] Step 5: According to the label of semantic classification and the preset interception rule, execute the corresponding interception operation to generate the interception operation feedback result.

[0147] In this step, the call category label is matched with the preset rules in the rule library to determine whether interception needs to be performed. If the interception condition is met, the corresponding interception operation is triggered. The interception result is fed back to the optimization module for continuous optimization of the interception strategy.

[0148] Step 6: Based on the feedback from the interception operation, combined with the manual quality inspection annotations, update and optimize the interception rules and classification models.

[0149] In this step, the misjudgment feedback from the interception operation is received and analyzed to generate error adjustment information. Based on the error feedback information, the interception rules and classification models are updated to improve the system's adaptive ability. The error adjustment results are fed back to the system to optimize the speech enhancement and classification models.

[0150] The method also uses the following technical solutions:

[0151] The error cause feedback mechanism is adopted. When the system makes a mistake (for example, a normal call is mistakenly intercepted, or a harassment call is mistakenly released), the graph annotation model records all relevant information nodes and their associations that lead to the mistake. This is like drawing a detailed event investigation report graph.

[0152] Record content: Each node represents a key processing link, for example:

[0153] Original call recording or text: the original data where the mistake occurred;

[0154] Output of the speech enhancement module: enhanced speech signal features;

[0155] Noise environment feature vector: noise environment information at the time of the mistake;

[0156] Intermediate features and final labels of the semantic classification module: how the semantic classifier judges the current call category (for example, is it an advertisement or normal business);

[0157] Matching path and decision result of the rule decision module: which rule did the system rely on to make the interception or release decision.

[0158] It also includes a cause-and-effect chain tracing mechanism: through the annotations with timestamps and internal states, developers can trace back from the final mistake result to the initial input data along the reverse direction of the data flow, and check whether the output of each module meets the expectations. For example, if a normal call is mistakenly judged as fraud and intercepted, the system can trace back to whether the speech enhancement introduced distortion, or the noise environment feature vector was biased, or the semantic classification model incorrectly identified the keyword, ultimately leading to the false triggering of the rule decision.

[0159] And the error analysis mechanism: through the generation of an interactive flowchart, the cause-and-effect chain of a single mistake is clearly displayed, with each node presented in a graphical form. Clicking on a node allows you to view the detailed data, intermediate features, or model output of that link.

[0160] By color (for example, red represents abnormality or high risk) or highlighting, the key nodes or abnormal values that may cause misjudgment in the decision chain are highlighted intuitively. For example, if the noise environment feature vector is abnormally high in a certain dimension, causing the semantic classifier to misjudge, this abnormal value will be highlighted on the graph.

[0161] For modules related to sequence processing (such as speech enhancement, semantic classification), a time axis view can be provided to show the processing status and results of each frame, so that the abnormality in time sequence can be found visually.

[0162] The error distribution matrix can be visualized in the form of a heat map. The rows and columns of the matrix can represent different call categories, noise types, signal-to-noise ratio ranges, etc. The color depth of the cell represents the frequency or severity of misjudgment under the specific condition. The deeper the color, the more concentrated the errors in this area, and the more prominent the problem.

[0163] In the visualization interface, the user can click on a highlighted area in the heat map to enter a more detailed report page to view the specific case list that caused this type of error, so as to analyze from both macro and micro levels.

[0164] As shown in Figure 3 A server in an embodiment of the present application includes a memory 201 and at least one processor 202.

[0165] The memory 201 stores a computer program, and the at least one processor 202 executes the computer program stored in the memory 201 to implement the low-frame-rate noise adaptive speech enhancement method described above.

[0166] In an embodiment of the present application, a computer readable storage medium is disclosed, and the computer readable storage medium stores a computer program. When the computer program is executed, the low-frame-rate noise adaptive speech enhancement method described above is implemented.

[0167] The beneficial effects of the present application are as follows:

[0168] Improve the adaptive ability of the system: by introducing a deep feedback sequence memory network (DFSMN) and a dynamic feedback mechanism, the system can adaptively adjust according to the changes in the noise environment during the speech enhancement process. In this way, the system can respond to changes in the noise environment in real time and maintain high speech enhancement accuracy in different communication scenarios, overcoming the shortcomings of traditional static rules that cannot adapt to new changes.

[0169] Provide effective misjudgment feedback mechanism: the present application proposes a differential misjudgment feedback matrix modeling mechanism, which enables the system to optimize the semantic classification model and interception rules through structured feedback of misjudgment samples. The feedback information of each misjudgment will be used for system optimization, thereby improving the recognition accuracy and robustness. This mechanism not only records the misjudgments, but also reversely promotes the improvement of the model and strategy, avoiding the problem that misjudgments cannot be effectively corrected in traditional systems.

[0170] Implement deep coupling of rules and models: the present application introduces a feedback-based dynamic adjustment mechanism in the system, so that the semantic classification model and rule decision mechanism can work closely together. When the classification model is updated, the rule base and interception strategy are optimized synchronously, ensuring that the system can adjust the decision strategy in time when facing new voice samples and situations, maintaining overall consistency and stability.

[0171] Improve the utilization rate of misintercepted samples: the present application adopts a negative example-driven incremental fine-tuning method, which enables the system to automatically generate a negative sample training set and fine-tune the model boundary through a lightweight loss function. This method can efficiently process misintercepted samples, finely adjust the discrimination accuracy of the system to speech, and thus improve the recognition ability of the system to ambiguous expressions, changes in accent, different speech rates, and other gray areas.

[0172] Enhance the long-term stability and accuracy of the system: the present application introduces error source analysis and error cause back-annotation mechanisms to effectively solve the problems of misjudgment accumulation and rule redundancy expansion in the long-term operation of existing systems. Through the feedback mechanism, the misjudgments are refined and corrected, and the strategy is updated, ensuring that the system maintains high recognition accuracy and user experience in long-term use, avoiding the problems of recognition accuracy decline and strategy failure in existing technologies.

[0173] Construct a traceable and auditable error analysis mechanism: the present application constructs a graph annotation model and an error distribution matrix, so that the system can record and trace the causal chain of each misjudgment, providing visual error data management. This not only improves the transparency of the system, but also facilitates fine-tuning by developers.

[0174] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise. In addition, it should be understood that when the terms "comprise" and / or "include" are used in the specification, they refer to the presence of a feature, step, operation, device, component, and / or combination thereof.

[0175] It is to be noted that the terms "first", "second", and the like, used in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of efficient implementation irrespective of the terms used.

[0176] Furthermore, the term "comprising" and "including" and their variants as used in the specification are intended to indicate a non-exclusive inclusion such that a process, method, system, product, or apparatus that comprises or includes a list of steps or elements can consist only of those steps or elements or can further include other steps or elements not expressly listed or inherent to such process, method, system, product, or apparatus.

[0177] For purposes of the description hereinafter, the terms "upper", "lower", "right", "left", "vertical", "horizontal", "top", "bottom", "lateral", "medial", "superior", "inferior", "proximal", "distal" and derivatives thereof shall relate to the application as it is shown in the drawings and as they are typically understood in the art unless otherwise noted. Any connection or coupling between or among components or blocks can be direct or through intervening components.

[0178] In the detailed description herein, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration various embodiments for describing the disclosure. The drawings are not necessarily to scale, the emphasis instead being placed upon illustrating the principles of the embodiments. The drawings are schematic and are intended to serve only as an aid in understanding the disclosure. In the drawings, like numerals represent like elements through the several views.

[0179] The preferred embodiments of the application are described herein with reference to the accompanying drawings. The description and drawings are illustrative of the application and are not to be construed as limiting the application. Various modifications and changes can be made thereto by those skilled in the art which fall within the scope of the present application as defined by the appended claims. Any modification or derivation should be considered as falling with the scope of the present application.

Claims

1. A low frame rate noise adaptive speech enhancement system, characterized by, Comprise: The voice acquisition and preprocessing module is used for receiving and processing the call voice signal, generating low frame rate voice data through down-sampling and frame level processing; The noise perception module is used for extracting and constructing the feature information of the background noise based on the low frame rate voice data to form a noise environment feature vector; The sequence memory enhancement module is used for combining the low frame rate voice data and the noise environment feature vector, performing noise adaptive voice enhancement processing through a deep feedback sequence memory network, and outputting the enhanced voice signal; The semantic recognition and classification module is used for transcribing the enhanced voice signal into text and performing semantic classification to provide a basis for subsequent rule decision; The rule decision module is used for executing corresponding interception operations according to the semantic classification label and the preset interception rule, and feeding back the interception result to the optimization module; The optimization module is used for updating and optimizing the interception rule and the classification model according to the interception result feedback and the manual quality inspection label.

2. The low frame rate noise-adaptive speech enhancement system of claim 1, wherein, The voice acquisition and preprocessing module comprises a voice signal receiving unit, a down-sampling unit and a frame level processing unit, wherein: The voice signal receiving unit is used for receiving the original audio signal in the communication link in real time; The down-sampling unit is used for reducing the sampling rate of the original audio signal and converting it into 16kHz low frame rate audio data; The frame level processing unit is used for dividing the low frame rate audio data into small frames and performing windowing processing.

3. The low frame rate noise-adaptive speech enhancement system of claim 1, wherein, The noise perception module comprises a noise feature extraction unit, a noise environment vector generation unit and a noise dynamic control unit, wherein: The noise feature extraction unit is used for converting each frame of voice signal into a frequency domain representation by using a short-time Fourier transform method and extracting frequency domain noise features; The noise environment vector generation unit is used for generating a noise environment feature vector based on the frequency domain noise features to drive the noise adaptive processing of the sequence memory enhancement module; The noise dynamic control unit is used for adjusting the noise suppression strategy in the voice enhancement process according to the noise environment feature vector.

4. The low frame rate noise-adaptive speech enhancement system of claim 1, wherein, The sequence memory enhancement module comprises a deep feedback network unit, an enhanced signal generation unit and a reinforcement learning unit, wherein: The deep feedback network unit is used for inputting the low frame rate voice data and the noise environment feature vector into a deep feedback neural network for processing, and using long short-term memory capability to maintain the context continuity of the voice; The enhanced signal generation unit is used for generating the enhanced voice signal according to the output of the deep feedback neural network; The reinforcement learning unit is used for adjusting the weight in the network according to the enhanced voice signal feedback, and optimizing the parameters of the deep feedback neural network through the reinforcement learning mechanism.

5. The low frame rate noise-adaptive speech enhancement system of claim 1, wherein, The semantic recognition and classification module comprises a voice-to-text unit, a model semantic classification unit and a classification label output unit, wherein: The voice-to-text unit is used for transcribing the enhanced voice signal into text and using a Transformer-based self-attention model for context decoding; The model semantic classification unit is used for extracting deep semantic features from the text through multiple deep learning models and performing semantic classification; The classification label output unit is configured to convert the semantic classification result into a call category label as an input of the rule decision module.

6. The low frame rate noise-adaptive speech enhancement system of claim 1, wherein, The rule decision module comprises a rule matching unit, an interception judgment unit, and an operation feedback unit. The rule matching unit is configured to match the call category label with preset rules in a rule library to determine whether interception needs to be performed. The interception judgment unit is configured to determine whether interception conditions are met and trigger corresponding interception operations. The operation feedback unit is configured to feed back interception results to the optimization module for continuous optimization of interception strategies.

7. The low frame rate noise-adaptive speech enhancement system of claim 1, wherein, The optimization module comprises an error feedback processing unit, a model updating unit, and a wrong cause feedback unit. The error feedback processing unit is configured to receive and analyze misjudgment feedback information from the interception operation to generate error adjustment information. The model updating unit is configured to update the interception rules and the classification model based on the error adjustment information to form an error adjustment result. The wrong cause feedback unit is configured to feed back the error adjustment result to the system to optimize the voice enhancement and classification model.

8. A low frame rate noise adaptive speech enhancement method using the low frame rate noise adaptive speech enhancement system according to any one of claims 1 to 7, characterized in that, The method comprises the following steps: Real-time reception of original voice signals, generation of low frame rate voice data after down-sampling and frame-level processing; Based on the low frame rate voice data, feature information of background noise is extracted and constructed to form a noise environment feature vector; Combined with the low frame rate voice data and the noise environment feature vector, noise adaptive voice enhancement processing is performed through a deep feedback sequence memory network to output enhanced voice signals; The enhanced voice signals are transcribed into text and subjected to semantic classification to provide a basis for subsequent rule decision; According to the label of semantic classification and the preset interception rules, corresponding interception operations are performed to generate interception operation feedback results; According to the interception operation feedback results, the interception rules and the classification model are updated and optimized in combination with artificial quality inspection labeling.

9. A server, characterized by It comprises: a memory and at least one processor; The memory stores a computer program, and the at least one processor executes the computer program stored in the memory to implement the low frame rate noise adaptive voice enhancement method of claim 8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed to implement the low frame rate noise adaptive voice enhancement method of claim 8.

Images