Low frame rate noise adaptive speech enhancement system, method and server

Abstract

The invention relates to a low-frame-rate noise adaptive speech enhancement system and method and a server, and belongs to the technical field of communication, and the system comprises a speech acquisition and preprocessing module which is used for receiving and processing a call speech signal; the noise sensing module is used for extracting and constructing feature information of background noise to form a noise environment feature vector based on the low-frame-rate voice data; the sequence memory enhancement module is used for combining the low-frame-rate voice data and the noise environment feature vector to carry out noise self-adaptive voice enhancement processing; the semantic recognition and classification module is used for transferring the enhanced voice signal into a text and carrying out semantic classification; the rule decision module is used for executing corresponding interception operation according to a preset interception rule; and the optimization module is used for updating and optimizing the interception rule and the classification model according to the interception result feedback in combination with the manual quality inspection annotation. By adopting the system, adaptive and refined speech enhancement can be carried out.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a low frame rate noise adaptive speech enhancement system, method and server. Background Art

[0002] With the development of telecommunications network technology, the frequency of voice communication in people's daily work and life continues to rise. In particular, in high-frequency interactive scenarios such as finance, government affairs, and express logistics, calls have gradually become an important medium for information transmission. However, the security and recognition accuracy of user voice calls face huge challenges.

[0003] Existing speech enhancement recognition systems typically use a fixed-frame-rate, end-to-end recognition model, combined with a pre-built risk semantic library and a manual rule-based system, to transcribe speech signals into text before performing keyword matching or label identification. Some systems incorporate deep learning models to classify text and then use pre-set policies to determine whether to issue a warning or block the call.

[0004] However, the existing speech enhancement recognition system has the following shortcomings: 1. Traditional speech enhancement recognition systems typically use end-to-end recognition models with a fixed frame rate, combined with a pre-set rule base and semantic library for speech analysis and interception decisions. Once the rules are set, these systems are difficult to dynamically adjust to changes in the actual environment or speech model. Therefore, when the model drifts or the speech environment changes, the existing systems often react slowly, lacking flexibility and adaptability.

[0005] 2. When misjudgments occur, many traditional speech recognition systems only passively record the error information and fail to implement corrections through effective feedback mechanisms. This results in the system being unable to automatically utilize error information for model optimization, resulting in extremely low utilization of misjudgment samples and difficulty achieving continuous self-improvement of the system.

[0006] 3. In many existing systems, there is a lack of tight coupling between semantic classification models and rule-based decision-making mechanisms. Even if the classification model accurately identifies speech content, the rule decisions are often separated from the model output, resulting in delayed or inconsistent rule updates in actual applications, making it difficult to adapt to new speech data characteristics.

[0007] 4. Traditional speech enhancement systems typically rely on global retraining to address misclassifications, rather than fine-tuning them through sample-level adjustments. This approach is inefficient, costly, and ineffective at addressing gray areas in speech content (such as accent, speaking speed, or ambiguous expressions).

[0008] Therefore, there is an urgent need for a low frame rate noise adaptive speech enhancement system that can solve the above problems. Summary of the Invention

[0009] The present invention aims to provide a low-frame-rate noise-adaptive speech enhancement system based on a deep feedback sequence memory network to address the disconnection between speech enhancement, noise suppression, semantic classification and rule-based decision-making in the prior art, as well as the accumulation of misjudgments and performance degradation during long-term system operation.

[0010] According to a first aspect of the present application, a low frame rate noise adaptive speech enhancement system is provided, comprising: The voice acquisition and preprocessing module is used to receive and process call voice signals and generate low-frame-rate voice data through downsampling and frame-level processing; A noise perception module, configured to extract and construct feature information of background noise based on the low frame rate speech data to form a noise environment feature vector; The sequence memory enhancement module is used to combine low-frame-rate speech data with the noise environment feature vector, perform noise-adaptive speech enhancement processing through a deep feedback sequence memory network, and output the enhanced speech signal; The semantic recognition and classification module is used to transcribe the enhanced speech signal into text and perform semantic classification to provide a basis for subsequent rule decision-making; The rule decision module is used to perform corresponding interception operations based on the semantic classification labels and preset interception rules, and feed back the interception results to the optimization module; The optimization module is used to update and optimize the interception rules and classification models based on the interception result feedback and combined with manual quality inspection and annotation.

[0011] Preferably, the voice collection and preprocessing module includes a voice signal receiving unit, a downsampling unit and a frame-level processing unit, wherein: The voice signal receiving unit is used to receive the original audio signal in the communication link in real time; The downsampling unit is used to reduce the sampling rate of the original audio signal and convert it into 16kHz low frame rate audio data; The frame-level processing unit is used to divide the low-frame-rate audio data into small frames and perform windowing processing.

[0012] Preferably, the noise perception module includes a noise feature extraction unit, a noise environment vector generation unit, and a noise dynamic control unit, wherein: The noise feature extraction unit is used to convert each frame of speech signal into a frequency domain representation using a short-time Fourier transform method to extract frequency domain noise features; The noise environment vector generating unit is used to generate a noise environment feature vector based on the frequency domain noise feature to drive the noise adaptive processing of the sequence memory enhancement module; The noise dynamic control unit is used to adjust the noise suppression strategy during the speech enhancement process according to the noise environment feature vector.

[0013] Preferably, the sequence memory enhancement module includes a deep feedback network unit, an enhancement signal generation unit, and a reinforcement learning unit, wherein: The deep feedback network unit is used to input low frame rate speech data and noise environment feature vectors into the deep feedback neural network for processing, and use long short-term memory capabilities to maintain the context continuity of the speech; The enhanced signal generating unit is configured to generate an enhanced speech signal according to an output of the deep feedback network; The reinforcement learning unit is used to adjust the weights in the network according to the enhanced speech signal feedback and optimize the parameters of the network through the reinforcement learning mechanism.

[0014] Preferably, the semantic recognition and classification module includes a speech-to-text unit, a model semantic classification unit, and a classification label output unit, wherein: The speech-to-text unit is used to transcribe the enhanced speech signal into text and perform context decoding using a Transformer-based self-attention model; The model semantic classification unit is used to extract deep semantic features from text and perform semantic classification through multiple deep learning models; The classification label output unit is used to convert the semantic classification results into call category labels as input to the rule decision module.

[0015] Preferably, the rule decision module includes a rule matching unit, an interception judgment unit, and an operation feedback unit, wherein: The rule matching unit is used to match the call category label with the preset rules in the rule library to determine whether interception is required; The interception judgment unit is used to judge whether the interception conditions are met and trigger the corresponding interception operation; The operation feedback unit is used to feed back the interception result to the optimization module for continuous optimization of the interception strategy.

[0016] Preferably, the optimization module includes an error feedback processing unit, a model updating unit, and an error cause reinjection unit, wherein: The error feedback processing unit is used to receive and analyze the misjudgment feedback information from the interception operation and generate error adjustment information; The model updating unit is configured to update the interception rules and the classification model based on the error adjustment information to form an error adjustment result; The error cause reinjection unit is used to inject the error adjustment result back into the system to optimize the speech enhancement and classification model.

[0017] According to a second aspect of the present application, a low frame rate noise adaptive speech enhancement method using the above-mentioned low frame rate noise adaptive speech enhancement system is provided, comprising the following steps: Receive the original voice signal in real time, and generate low-frame-rate voice data after downsampling and frame-level processing; Extracting and constructing feature information of background noise based on the low frame rate speech data to form a noise environment feature vector; Combining low-frame-rate speech data with the noise environment feature vector, a deep feedback sequence memory network is used to perform noise-adaptive speech enhancement processing and output the enhanced speech signal. Transcribe the enhanced speech signal into text and perform semantic classification to provide a basis for subsequent rule decisions; According to the semantic classification labels and preset interception rules, the corresponding interception operation is executed and the interception operation feedback result is generated; According to the feedback results of the interception operation, the interception rules and classification models are updated and optimized in combination with manual quality inspection and annotation.

[0018] According to a third aspect of the present application, there is provided a server, comprising: a memory and at least one processor; The memory stores a computer program, and the at least one processor executes the computer program stored in the memory to implement the above-mentioned low frame rate noise adaptive speech enhancement method.

[0019] According to a fourth aspect of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed, the low frame rate noise adaptive speech enhancement method described above is implemented.

[0020] According to one embodiment of the present application, the low frame rate noise adaptive speech enhancement system has the following advantages: By introducing the Deep Feedback Sequential Memory Network (DFSMN) and dynamic feedback mechanism, the system can adaptively adjust to changes in the noise environment during the speech enhancement process. The differential misclassification feedback matrix modeling mechanism enables the system to optimize the semantic classification model and interception rules through structured feedback on misclassified samples. The feedback information of each misclassification is used to optimize the system, thereby improving recognition accuracy and robustness.

[0021] By introducing a dynamic adjustment mechanism based on feedback into the system, the semantic classification model and the rule-based decision-making mechanism can work closely together. When the classification model is updated, the rule base and interception strategy are optimized simultaneously, ensuring that the system can adjust its decision-making strategy in a timely manner when faced with new speech samples and situations, maintaining overall consistency and stability.

[0022] Adopting a negative-example-driven incremental fine-tuning method, the system can automatically generate negative sample training sets and fine-tune the model boundaries through a lightweight loss function; By introducing error source analysis and error cause back-injection mechanisms, the problems of misjudgment accumulation and rule redundancy expansion in the long-term operation of the existing system are effectively solved; By constructing a graph annotation model and error distribution matrix, the system can record and trace the causal chain of each misjudgment and provide visual error data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a structural block diagram of a low frame rate noise adaptive speech enhancement system in one embodiment of the present application; Figure 2 This is a flow chart of a low frame rate noise adaptive speech enhancement method in one embodiment of the present application; Figure 3 This is a structural block diagram of a server in one embodiment of the present application. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] It should be noted that the above detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs.

[0026] like Figure 1 As shown, the low frame rate noise adaptive speech enhancement system in one embodiment of the present application includes: The voice acquisition and preprocessing module is used to receive and process call voice signals and generate low-frame-rate voice data through downsampling and frame-level processing; The module includes a voice signal receiving unit, a downsampling unit, and a frame-level processing unit, wherein: Voice signal receiving unit: receives the original audio signal in the communication link in real time.

[0027] Downsampling unit: reduces the sampling rate of the original audio signal and converts it into 16kHz low-frame-rate audio data, reducing data redundancy and improving the efficiency of subsequent processing.

[0028] Frame-level processing unit: Divides low-frame-rate audio data into small frames and performs windowing processing (such as Hamming window) to ensure that the signal does not leak in the frequency domain.

[0029] A noise perception module, configured to extract and construct feature information of background noise based on the low frame rate speech data to form a noise environment feature vector; The module includes a noise feature extraction unit, a noise environment vector generation unit, and a noise dynamic control unit, wherein: Noise feature extraction unit: Uses the short-time Fourier transform (STFT) method to convert each frame of speech signal into a frequency domain representation and extract frequency domain noise features, as follows: For a discrete-time speech signal , and its short-time Fourier transform calculation formula is: ; Where, Represents the STFT result, that is, in the timeframe and Complex value at frequency points; is the frame index; is the frequency index; is the original discrete-time speech signal; It is a window function (such as Hamming window, Heining window, etc.), which is used to frame the signal to reduce spectrum leakage; It is the number of Fourier transform points (FFT size), which determines the frequency resolution; is the frame shift (hopsize), which represents the sampling point interval between adjacent frames; is a complex exponential term, representing the frequency sine wave; Is an imaginary unit.

[0030] For speech signals It is first divided into a series of overlapping or non-overlapping short time frames. The length of each frame is usually sampling points;

[0031] Apply a window function to each frame , the window function helps smooth the edges of the frame and reduce spectral leakage in the Fourier transform. A common window function is the Hamming window, whose formula is:

[0032] ; A discrete Fourier transform (DFT) is performed on each frame of the windowed signal, and an efficient fast Fourier transform (FFT) algorithm is usually used to calculate its spectrum, thereby converting each frame of the signal from the time domain representation to the frequency domain representation, that is, obtaining the spectrum information of the frame.

[0033] Noise environment vector generation unit: Generates noise environment feature vectors based on frequency domain noise features to drive the noise adaptive processing of the sequence memory enhancement module. Specifically: The optimal estimate is obtained by minimizing the mean square error between the estimated noise power spectrum and the true noise power spectrum. Voice activity detection (VAD) is first performed. VAD aims to distinguish speech frames from pure noise frames.

[0034] VAD can be based on a variety of features, such as: Energy threshold: The energy of pure noise frames is usually lower than that of speech frames; Zero-crossing rate: Speech and noise have different zero-crossing rate characteristics; Spectral entropy: The spectrum of pure noise is usually flatter than that of speech and has a higher spectral entropy; Machine Learning / Deep Learning Model: Train a classifier to determine whether a frame contains speech; Label each frame as speech or non-speech (noise).

[0035] When the VAD determines that the current frame is non-speech (noise), it uses the spectral information of the frame to update the noise estimate. A smooth update strategy is used to prevent the noise estimate from fluctuating violently. The common update formula is:

[0036] ; Where, It is in Frame and frequency The estimated noise power on ; is the noise power estimate of the previous frame; Is the current Frame (determined as noise frame by VAD) at frequency The power spectrum on (obtained by the square of the magnitude of the STFT); Is a smoothing factor that determines the speed of the update. Makes the noise estimate more stable, but responds slower to noise changes; smaller The opposite is true.

[0037] At the initial stage of system startup, it is usually assumed that the first few frames are pure noise, or a preset initial noise power spectrum is used, and the estimated noise power spectrum is used as output to form a feature vector that can be used by the "sequence memory enhancement module".

[0038] After each frame update (For all frequencies ) It is the current noise environment feature vector. The dimension of this vector is usually (If we consider the one-sided spectrum of a real signal), where is the number of FFT points.

[0039] Noise dynamic control unit: adjusts the noise suppression strategy during speech enhancement based on the noise environment feature vector, specifically: The noise environment feature vector mainly contains the estimated noise power spectrum In traditional speech enhancement algorithms (such as Wiener filtering and spectral subtraction), the intensity of noise suppression is usually controlled by a gain factor (also called suppression factor or noise reduction factor). The noise dynamic control unit can dynamically calculate or adjust this gain factor according to the real-time noise power spectrum.

[0040] When the noise power is large, stronger suppression is required; when the noise power is small, weaker suppression is required to avoid excessive noise reduction and speech distortion.

[0041] Global gain adjustment: by calculating the average noise energy or signal-to-noise ratio (SNR) of the entire frequency band or certain key frequency bands: If the average noise energy is high or the SNR is low: The noise dynamics control unit instructs the speech enhancement module to adopt a more aggressive noise suppression strategy, such as reducing the overall gain factor or increasing the noise suppression-related loss weight in the deep feedback sequence memory network.

[0042] If the average noise energy is low or the SNR is high: The noise dynamics control unit will indicate the use of a milder suppression strategy, such as increasing the overall gain factor, to reduce the damage to the speech itself.

[0043] Band-independent gain adjustment: Since the distribution of noise at different frequencies may be different, a finer adjustment is to calculate the gain factor independently for each frequency point or frequency band.

[0044] In Wiener filtering, the gain function It is usually related to the local signal-to-noise ratio. Although deep feedback networks do not directly use the Wiener filter formula, their internal learning mechanism can perceive this local signal-to-noise ratio through the noise environment feature vector, thereby suppressing noise of different frequencies to varying degrees at the feature level.

[0045] Provides the deep feedback network with an explicit indication of the noise energy at each frequency point in the current frame (i.e. ), the deep feedback network will learn to superior, The larger it is, the more suppression should be applied at that frequency.

[0046] This module extracts noise features from real-time, low-frame-rate speech data and generates noise environment feature vectors to drive noise suppression strategies during speech enhancement. This allows the system to dynamically adapt to varying noise environments, performing adaptive noise suppression and significantly improving speech quality in complex environments.

[0047] The sequence memory enhancement module is used to combine low-frame-rate speech data with the noise environment feature vector, perform noise-adaptive speech enhancement processing through a deep feedback sequence memory network, and output the enhanced speech signal; The module includes a deep feedback network unit, an enhanced signal generation unit, and a reinforcement learning unit, where: Deep Feedback Network Unit: Low-frame-rate speech data and noise environment feature vectors are input into a deep feedback neural network for processing. Long-short-term memory (LSTM-like mechanism) is used to maintain the contextual continuity of the speech. Specifically: By receiving low-frame-rate speech data from the speech acquisition and preprocessing module, it is usually a frequency domain representation of the speech signal (such as the amplitude spectrum or power spectrum after short-time Fourier transform), or further extracted features (such as the Mel spectrum).

[0048] At the same time, it receives the noise environment feature vector from the noise perception module. This vector contains detailed information about the current background noise (for example, real-time estimated noise power spectrum, noise type classification).

[0049] By concatenating low-frame-rate speech data with the noise environment feature vector, a combined input feature vector is formed and fed into the DFSMN input layer. This enables the deep feedback network to simultaneously perceive both the speech content and the noise environment, thereby performing targeted enhancement.

[0050] The fused input feature vector first passes through multiple deep feed-forward layers of the DFSMN (typically fully connected layers or one-dimensional convolutional layers). By performing complex nonlinear mapping, the raw low-frame-rate speech and noise features are converted into higher-level representations. Following the deep feature extraction layer, the DFSMN integrates one or more modules with long-short-term memory capabilities. The LSTM / GRU operating method includes input gates, forget gates, and output gates, intelligently controlling the inflow and outflow of information and the updating of memory cell states.

[0051] Selective memory: Deciding which old information to keep for the future and which to forget.

[0052] Selective Update: Determines how the current input updates the memory state.

[0053] Selective output: Determines which parts of the memory state can be output to the next layer.

[0054] Through these memory units, DFSMN can effectively process the temporal dependencies of speech signals. Even if speech is delivered at a low frame rate, where information becomes increasingly sparse over time, the memory units can still encode speech features and noise characteristics (such as noise persistence patterns, speech rate, and intonation) from previous and even more distant frames into their internal state. When processing the current frame, the network combines the current input with this "memorized" contextual information to make more accurate enhancement decisions.

[0055] Enhanced signal generation unit: Generates an enhanced speech signal based on the output of the deep feedback network.

[0056] Reinforcement learning unit: Adjusts the weights in the deep feedback network based on the enhanced speech signal feedback, optimizes the network parameters through the reinforcement learning mechanism, and improves the enhancement effect.

[0057] This module uses a deep feedback sequential memory network (DFSMN) to enhance low-frame-rate speech data. It also uses a long-short-term memory (LSTM)-like mechanism to model the long-term dependencies of speech signals. This effectively preserves speech context and clarity, particularly at low frame rates, improving the quality and effectiveness of speech enhancement.

[0058] The semantic recognition and classification module is used to transcribe the enhanced speech signal into text and perform semantic classification to provide a basis for subsequent rule decision-making; This module includes a speech-to-text unit, a model semantic classification unit, and a classification label output unit, where: Speech-to-text unit: transcribes the enhanced speech signal into text and uses a Transformer-based self-attention model for contextual decoding.

[0059] Model semantic classification unit: Through multiple deep learning models (such as BERT, CNN+LSTM, etc.), deep semantic features are extracted from text and semantic classification is performed.

[0060] Classification label output unit: converts the semantic classification results into call category labels as input to the rule decision module.

[0061] The rule decision module is used to perform corresponding interception operations based on the semantic classification labels and preset interception rules, and feed back the interception results to the optimization module; This module matches the call type (such as advertising promotion, normal business, etc.) identified by the classification label output unit with the preset interception rule library.

[0062] For example, if it is identified as normal business, it matches the normal call release rule and does not intercept.

[0063] This module includes a rule matching unit, an interception judgment unit, and an operation feedback unit, among which: Rule matching unit: matches the call category label with the preset rules in the rule library to determine whether interception is required.

[0064] Interception judgment unit: judges whether the interception conditions are met and triggers the corresponding interception operation.

[0065] Operation feedback unit: Feedback the interception results to the optimization module for continuous optimization of the interception strategy.

[0066] By tightly integrating semantic recognition with rule-based decision-making, this system achieves bidirectional feedback and dynamic optimization of models and rules. When executing interception operations, the system not only provides feedback on interception results but also continuously optimizes interception rules and classification models, thereby maintaining system adaptability and accuracy and avoiding the rule lag and misjudgment problems associated with traditional methods.

[0067] The optimization module is used to update and optimize the interception rules and classification models based on the interception result feedback and combined with manual quality inspection and annotation.

[0068] This module includes an error feedback processing unit, a model update unit, and an error cause injection unit, among which: Error feedback processing unit: receives and analyzes misjudgment feedback from interception operations and generates error adjustment information.

[0069] Model updating unit: Based on error feedback information, it updates the interception rules and classification models to improve the system's adaptability.

[0070] Error Reinjection Unit: Injects error adjustment results back into the system to optimize speech enhancement and classification models.

[0071] This module structures misjudgment feedback results through a differential misjudgment feedback matrix modeling mechanism. This feedback information can be fed back into the classification model and interception rules for optimization. After each misjudgment, the system adjusts rule weights and updates the classification model to prevent misjudgment accumulation and improve the system's long-term stability and recognition accuracy.

[0072] This system introduces a negative-example-driven incremental fine-tuning method, which can use falsely intercepted samples to generate negative sample sets and perform small-scale lightweight adjustments, avoiding the high cost and low efficiency of overall retraining in traditional solutions, and improving the system's training efficiency and processing capabilities.

[0073] like Figure 2 As shown, the low frame rate noise adaptive speech enhancement method using the above low frame rate noise adaptive speech enhancement system includes the following steps: Step 1: Receive the original speech signal in real time, and generate low-frame-rate speech data after downsampling and frame-level processing; In this step, after receiving the original audio signal from the communication link in real time, the original audio signal is downsampled and converted into 16kHz low-frame-rate audio data to reduce data redundancy and improve the efficiency of subsequent processing. The low-frame-rate audio data is then divided into small frames and windowed (such as a Hamming window) to ensure that the signal does not leak in the frequency domain.

[0074] Step 2: Based on the low frame rate speech data, extract and construct feature information of background noise to form a noise environment feature vector; In this step, the Short-Time Fourier Transform (STFT) method is used to convert each frame of speech signal into a frequency-domain representation and extract frequency-domain noise features. This noise environment feature vector is then generated based on the frequency-domain noise features. This vector drives the noise adaptive processing in the sequence memory enhancement module. The noise suppression strategy during the speech enhancement process is adjusted based on the noise environment feature vector.

[0075] Step 3: Combine the low-frame-rate speech data with the noise environment feature vector, perform noise-adaptive speech enhancement processing through a deep feedback sequence memory network, and output the enhanced speech signal; In this step, low-frame-rate speech data and the noise environment feature vector are fed into a deep neural network for processing. Long-short-term memory (LSTM-like) mechanisms are used to maintain contextual continuity within the speech. Based on the output of the deep neural network, an enhanced speech signal is generated. Finally, the network weights are adjusted based on the enhanced speech signal feedback, and the network parameters are optimized through reinforcement learning to enhance the enhancement effect.

[0076] Step 4: Transcribe the enhanced speech signal into text and perform semantic classification to provide a basis for subsequent rule decisions; In this step, the enhanced speech signal is transcribed into text and contextually decoded using a Transformer-based self-attention model. Multiple deep learning models (such as BERT and CNN+LSTM) are then used to extract deep semantic features from the text and perform semantic classification. The semantic classification results are converted into call category labels, which serve as input to the rule-based decision module.

[0077] Step 5: Execute the corresponding interception operation based on the semantic classification label and the preset interception rules, and generate the interception operation feedback result; In this step, the call category label is matched against pre-set rules in the rule library to determine whether interception is necessary. If the interception conditions are met, the corresponding interception action is triggered. The interception results are then fed back to the optimization module for continuous optimization of the interception strategy.

[0078] Step 6: Based on the feedback results of the interception operation, the interception rules and classification model are updated and optimized in combination with manual quality inspection and annotation.

[0079] In this step, we receive and analyze feedback from interception operations that indicate misjudgments, generating error adjustment information. Based on this error feedback, we update interception rules and classification models to enhance the system's adaptability. The error adjustment results are then fed back into the system to optimize the speech enhancement and classification models.

[0080] This method also adopts the following technical solutions: Using an error-cause annotation mechanism, when the system misjudges a call (for example, mistakenly intercepting a legitimate call or allowing a spam call), the graph annotation model records all the relevant information nodes and their connections that led to the misjudgment. This is like drawing a detailed incident investigation report.

[0081] Record content: Each node represents a key processing link, for example: Original call recording or text: original data where the misjudgment occurred; Output of the speech enhancement module: enhanced speech signal features; Noise environment feature vector: noise environment information when misjudgment occurs; The intermediate features and final labels of the semantic classification module: How the semantic classifier determines the current call category (for example, whether it is an advertisement or normal business); The matching path and decision results of the rule decision module: The rule based on which the system makes the interception or release decision.

[0082] It also includes a causal chain traceability mechanism: using annotations with timestamps and internal states, developers can trace back the data flow from the final misjudgment result to the initial input data step by step, and check whether the output of each module meets expectations. For example, if a normal call is mistakenly identified as fraud and blocked, the system can trace back to whether it was distortion introduced by voice enhancement, biased feature vectors in a noisy environment, or incorrect keyword recognition by the semantic classification model, ultimately leading to the false triggering of the rule decision.

[0083] And the error analysis mechanism: by generating an interactive flowchart, the causal chain of a single misjudgment is clearly displayed. Each node is presented in graphical form. Clicking a node can view the detailed data, intermediate features or model output of that link.

[0084] By using color (for example, red indicates abnormality or high risk) or highlighting, we can visually highlight key nodes or outliers in the decision chain that may lead to misjudgment. For example, if a dimension of a noisy environment feature vector is abnormally high, causing the semantic classifier to misjudge, this outlier will be highlighted on the graph.

[0085] For modules involving sequence processing (such as speech enhancement and semantic classification), a timeline view can be provided to display the processing status and results of each frame, allowing for visual detection of timing anomalies.

[0086] The error distribution matrix can be visualized as a heat map. The rows and columns of the matrix can represent different dimensions, such as call types, noise types, and signal-to-noise ratio ranges. The color of the cells indicates the frequency or severity of misjudgments under those specific conditions. The darker the color, the more concentrated the errors in that area, indicating a more prominent problem.

[0087] In the visualization interface, users can click on a highlighted area in the heat map to enter a more detailed report page and view a list of specific cases that caused this type of error, thereby conducting analysis from both macro and micro levels.

[0088] like Figure 3 As shown, a server in an embodiment of the present application includes: a memory 201 and at least one processor 202; The memory 201 stores a computer program, and the at least one processor 202 executes the computer program stored in the memory 201 to implement the above-mentioned low frame rate noise adaptive speech enhancement method.

[0089] In one embodiment of the present application, a computer-readable storage medium is disclosed. The computer-readable storage medium stores a computer program. When the computer program is executed, the low frame rate noise adaptive speech enhancement method described above is implemented.

[0090] The beneficial effects of the present invention are: Improving the system's adaptability: By incorporating a deep feedback sequential memory network (DFSMN) and a dynamic feedback mechanism, this invention enables the system to adapt to changes in the noise environment during speech enhancement. This allows the system to respond to noise changes in real time and maintain high speech enhancement accuracy in various communication scenarios, overcoming the limitations of traditional static rules that are unable to adapt to new changes.

[0091] Providing an effective false positive feedback mechanism: This paper proposes a differential false positive feedback matrix modeling mechanism, enabling the system to optimize semantic classification models and interception rules through structured feedback on false positive samples. Feedback information from each false positive is used to optimize the system, thereby improving recognition accuracy and robustness. This mechanism not only records false positives but also drives improvements to models and strategies, avoiding the problem of false positives being unable to be effectively corrected in traditional systems.

[0092] Deeply coupling rules and models: This invention integrates a feedback-based dynamic adjustment mechanism into the system, enabling close coordination between the semantic classification model and the rule-based decision-making mechanism. When the classification model is updated, the rule base and interception strategy are simultaneously optimized, ensuring that the system can adjust its decision-making strategy in response to new speech samples and contexts, maintaining overall consistency and stability.

[0093] Improving the utilization of false positives: This paper employs a negative-example-driven incremental fine-tuning method, enabling the system to automatically generate a training set of negative examples and fine-tune the model boundaries using a lightweight loss function. This method efficiently handles false positives and fine-tunes the system's speech recognition accuracy, thereby improving recognition of gray areas such as ambiguous expressions, accent variations, and varying speech rates.

[0094] Enhanced long-term system stability and accuracy: This invention effectively addresses the long-term problems of misjudgment accumulation and rule redundancy in existing systems by introducing error source analysis and error-cause feedback mechanisms. This feedback mechanism refines and corrects misjudgments and updates policies, ensuring the system maintains high recognition accuracy and user experience over extended periods of use, avoiding the degradation of recognition accuracy and policy failures seen in existing technologies.

[0095] Build a traceable and auditable error analysis mechanism: This invention builds a graph annotation model and error distribution matrix, enabling the system to record and trace the causal chain of each misjudgment, providing visual error data management. This not only improves system transparency but also facilitates developers to refine it.

[0096] It should be noted that the terms used herein are intended only to describe specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0097] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0098] In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0099] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used herein to describe the spatial positional relationship of a device or feature to other devices or features as shown in the figures. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figures. For example, if the device in the drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be subsequently positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways, such as rotated 90 degrees or in other orientations, and the spatially relative descriptions used herein are interpreted accordingly.

[0100] In the above detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless the context dictates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be used, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.

[0101] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A low frame rate noise adaptive speech enhancement system, characterized in that: include: The voice acquisition and preprocessing module is used to receive and process call voice signals and generate low-frame-rate voice data through downsampling and frame-level processing; A noise perception module, configured to extract and construct feature information of background noise based on the low frame rate speech data to form a noise environment feature vector; The sequence memory enhancement module is used to combine low-frame-rate speech data with the noise environment feature vector, perform noise-adaptive speech enhancement processing through a deep feedback sequence memory network, and output the enhanced speech signal; The semantic recognition and classification module is used to transcribe the enhanced speech signal into text and perform semantic classification to provide a basis for subsequent rule decision-making; The rule decision module is used to perform corresponding interception operations based on the semantic classification labels and preset interception rules, and feed back the interception results to the optimization module; The optimization module is used to update and optimize the interception rules and classification models based on the interception result feedback and combined with manual quality inspection and annotation.

2. The low frame rate noise adaptive speech enhancement system according to claim 1, characterized in that: The speech acquisition and preprocessing module includes a speech signal receiving unit, a downsampling unit, and a frame-level processing unit, wherein: The voice signal receiving unit is used to receive the original audio signal in the communication link in real time; The downsampling unit is used to reduce the sampling rate of the original audio signal and convert it into 16kHz low frame rate audio data; The frame-level processing unit is used to divide the low-frame-rate audio data into small frames and perform windowing processing.

3. The low frame rate noise adaptive speech enhancement system according to claim 1, characterized in that: The noise perception module includes a noise feature extraction unit, a noise environment vector generation unit, and a noise dynamic control unit, wherein: The noise feature extraction unit is used to convert each frame of speech signal into a frequency domain representation using a short-time Fourier transform method to extract frequency domain noise features; The noise environment vector generating unit is used to generate a noise environment feature vector based on the frequency domain noise feature to drive the noise adaptive processing of the sequence memory enhancement module; The noise dynamic control unit is used to adjust the noise suppression strategy during the speech enhancement process according to the noise environment feature vector.

4. The low frame rate noise adaptive speech enhancement system according to claim 1, characterized in that: The sequence memory enhancement module includes a deep feedback network unit, an enhancement signal generation unit, and a reinforcement learning unit, wherein: The deep feedback network unit is used to input low frame rate speech data and noise environment feature vectors into the deep feedback neural network for processing, and use long short-term memory capabilities to maintain the context continuity of the speech; The enhanced signal generating unit is configured to generate an enhanced speech signal according to the output of the deep feedback neural network; The reinforcement learning unit is used to adjust the weights in the network according to the enhanced speech signal feedback, and optimize the parameters of the deep feedback neural network through the reinforcement learning mechanism.

5. The low frame rate noise adaptive speech enhancement system according to claim 1, characterized in that: The semantic recognition and classification module includes a speech-to-text unit, a model semantic classification unit, and a classification label output unit, wherein: The speech-to-text unit is used to transcribe the enhanced speech signal into text and perform context decoding using a Transformer-based self-attention model; The model semantic classification unit is used to extract deep semantic features from text and perform semantic classification through multiple deep learning models; The classification label output unit is used to convert the semantic classification results into call category labels as input to the rule decision module.

6. The low frame rate noise adaptive speech enhancement system according to claim 1, characterized in that: The rule decision module includes a rule matching unit, an interception judgment unit, and an operation feedback unit, wherein: The rule matching unit is used to match the call category label with the preset rules in the rule library to determine whether interception is required; The interception judgment unit is used to judge whether the interception conditions are met and trigger the corresponding interception operation; The operation feedback unit is used to feed back the interception result to the optimization module for continuous optimization of the interception strategy.

7. The low frame rate noise adaptive speech enhancement system according to claim 1, characterized in that: The optimization module includes an error feedback processing unit, a model updating unit, and an error cause re-injection unit, wherein: The error feedback processing unit is used to receive and analyze the misjudgment feedback information from the interception operation and generate error adjustment information; The model updating unit is configured to update the interception rules and the classification model based on the error adjustment information to form an error adjustment result; The error cause reinjection unit is used to inject the error adjustment result back into the system to optimize the speech enhancement and classification model.

8. A low frame rate noise adaptive speech enhancement method using the low frame rate noise adaptive speech enhancement system according to any one of claims 1 to 7, characterized in that: The steps include: Receive the original voice signal in real time, and generate low-frame-rate voice data after downsampling and frame-level processing; Extracting and constructing feature information of background noise based on the low frame rate speech data to form a noise environment feature vector; Combining low-frame-rate speech data with the noise environment feature vector, a deep feedback sequence memory network is used to perform noise-adaptive speech enhancement processing and output the enhanced speech signal. Transcribe the enhanced speech signal into text and perform semantic classification to provide a basis for subsequent rule decisions; According to the semantic classification labels and preset interception rules, the corresponding interception operation is executed and the interception operation feedback result is generated; According to the feedback results of the interception operation, the interception rules and classification models are updated and optimized in combination with manual quality inspection and annotation.

9. A server, characterized in that: include: memory and at least one processor; The memory stores a computer program, and the at least one processor executes the computer program stored in the memory to implement the low frame rate noise adaptive speech enhancement method according to claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the low frame rate noise adaptive speech enhancement method according to claim 8 is implemented.

Images