A digital human voice interaction anti-interference method, system, device and medium

By optimizing the digital human voice interaction system through speech preprocessing, hot word weighting and self-learning mechanisms, streaming speech recognition and dynamic sliding window matching technology, the problems of speech recognition accuracy and interaction fluency in noisy environments are solved, achieving more efficient noise resistance and intent matching, and improving user experience.

CN118658478BActive Publication Date: 2024-11-08HANGZHOU YILAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411137303.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-08
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

Existing digital human voice interaction technologies lack accuracy in speech recognition and fluency in noisy environments, and traditional methods perform poorly in complex noisy environments, making it difficult to meet real-time requirements.

Method used

By employing speech preprocessing, hot word weighting and self-learning mechanisms, streaming speech recognition, and dynamic sliding window matching technology, the audio signal quality is optimized, the accuracy of keyword recognition and response sensitivity are improved, and real-time intent matching is achieved.

Benefits of technology

It significantly improves the voice interaction performance of digital humans in noisy environments, provides a smoother and more natural user experience, has stronger noise resistance and robustness, and improves the accuracy of voice recognition and the interactive experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118658478B_ABST
    Figure CN118658478B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of natural language processing, and discloses a digital human voice interaction anti-interference method, system, device and medium, the method comprising: capturing audio signals, and optimizing the voice quality and sampling rate of the audio signals through voice preprocessing; constructing a language model based on text data, and introducing hot word weighting and self-learning mechanism to train the hot word probability and generalization ability of the language model; real-time recognition of streaming voice in audio signals, and extraction of keywords in streaming voice, capturing and responding to user intentions that match keywords; converting streaming voice into text strings, identifying the intention matching success rate through dynamic sliding window matching, and issuing digital human control commands after successful matching. The present invention improves the accuracy and robustness of voice recognition through the integration and optimization of advanced technologies, significantly improves the voice interaction performance of digital humans in noisy environments, and thus provides a more fluent and natural user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a digital human voice interaction anti-interference method, system, device and medium. Background Art

[0002] With the rapid development of intelligent technology and artificial intelligence, digital human voice interaction technology has become a hot research direction in the field of human-computer interaction. This technology allows users to communicate with computer systems or virtual characters through natural language, greatly enhancing user experience and interaction convenience.

[0003] However, in many application scenarios, voice interaction in traditional strong noise environments is particularly challenging, including but not limited to streets with heavy traffic, industrial production lines, large-scale public events, etc. In these occasions, background noise will significantly affect the accuracy of voice recognition and the fluency of interaction.

[0004] Existing implementation solutions and limitations:

[0005] 1. Hardware-side multi-microphone noise suppression: A multi-microphone array is used to enhance voice signals in a specific direction through spatial filtering techniques, such as beamforming algorithms, while suppressing noise in other directions. This method uses the physical position differences between microphones to estimate and eliminate background noise, which improves the performance of far-field speech recognition to a certain extent, but is still limited by recognition accuracy in complex noise environments.

[0006] 2. Noise suppression and speech enhancement based on deep learning: Blind source separation technology can improve the quality of speech signals to a certain extent by training deep neural networks to identify and separate speech and noise components. However, these methods often require large training data sets, may not work well under unseen noise types or extreme noise conditions, and cannot meet real-time requirements.

[0007] Therefore, how to provide an anti-interference method for digital human voice interaction is a problem that needs to be solved urgently. Summary of the invention

[0008] The embodiments of the present invention provide a digital human voice interaction anti-interference method, system, device and medium to solve the above problems existing in the prior art.

[0009] In order to have a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not a general review, nor is it intended to identify key / important components or to describe the scope of protection of these embodiments. Its only purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0010] According to a first aspect of an embodiment of the present invention, a digital human voice interaction anti-interference method is provided.

[0011] In one embodiment, the digital human voice interaction anti-interference method includes:

[0012] Capture audio signals and optimize the voice quality and sampling rate of audio signals through voice preprocessing;

[0013] Based on text data, we build a language model and introduce hot word weighting and self-learning mechanisms to train the hot word probability and generalization ability of the language model.

[0014] Recognize streaming speech in audio signals in real time, extract keywords from streaming speech, and capture and respond to user intentions that match keywords;

[0015] Convert streaming voice into text strings, identify the intent matching success rate through dynamic sliding window matching, and issue digital human control commands after a successful match.

[0016] In one embodiment, capturing the audio signal and optimizing the voice quality and sampling rate of the audio signal through voice preprocessing includes:

[0017] Capture audio signals containing a mixture of human voice and background noise;

[0018] Acquire a reference signal, and eliminate the echo of the audio signal using an adaptive filtering algorithm;

[0019] Based on the resampling process of the audio signal, the sampling rate is adjusted to a preset frequency.

[0020] In one embodiment, the language model is constructed based on text data, and hot word weighting and self-learning mechanisms are introduced to train the hot word probability and generalization ability of the language model, including:

[0021] Collect text data and build a hot word database containing hot words and key sentences;

[0022] Select the real-time speech recognition model as the basis to build the initial model;

[0023] The text data is divided into a training set, a validation set and a test set according to a preset ratio, and an input text sequence is set and input into the initial model; a language model is obtained through parameter calculation and iterative training;

[0024] Introducing a weighted coefficient of a hot word, adjusting the occurrence frequency of each hot word in the hot word database, and calculating the adjusted hot word probability distribution and the normalized hot word probability distribution;

[0025] A self-learning mechanism is introduced to perform incremental training on the language model using new recognition data; and based on the training recognition results, high-frequency hot words are analyzed, the training set is regularly updated, and the language model is retrained.

[0026] In one embodiment, the calculation formula for the adjusted hot word probability distribution is:

[0027]

[0028] In the formula, Represents the adjusted hot word probability distribution; Shows the probability distribution of hot words before adjustment; α Indicates the hot word weight coefficient;

[0029] The calculation formula of the normalized hot word probability distribution is:

[0030]

[0031] In the formula, Represents the normalized hot word probability distribution; Represents the sum of all hot words probability distribution.

[0032] In one embodiment, the real-time recognition of streaming speech in the audio signal, extraction of keywords in the streaming speech, and capturing and responding to user intentions matching the keywords include:

[0033] Extracting keywords contained in the streaming speech based on complete semantics and context information;

[0034] Preprocess the streaming speech, and then perform punctuation analysis, regular expression matching, and pinyin similarity comparison on the streaming speech in sequence; based on the comprehensive analysis of the results, capture the user's intention and generate a corresponding response.

[0035] In one embodiment, the converting of the streaming voice into a text string, identifying the intention matching success rate through dynamic sliding window matching, and issuing the digital human control command after the matching is successful includes:

[0036] Convert the real-time recognized streaming speech into text strings, and form a text stream through continuous reception and speech processing, and generate the recognized text content in real time;

[0037] Initialize the window parameters of the dynamic sliding window, move the sliding window in the text stream, obtain the window content at each position, and match it with the preset intention trigger keyword. If the match is successful, the intention is sent to the digital human as a control instruction. If the match fails, continue to move the sliding window.

[0038] In one embodiment, the window parameters of the sliding window include a window size and a moving rate.

[0039] According to a second aspect of an embodiment of the present invention, a digital human voice interaction anti-interference system is provided, the system comprising:

[0040] A speech preprocessing module is used to capture audio signals and optimize the speech quality and sampling rate of the audio signals through speech preprocessing;

[0041] The streaming speech recognition module is used to build a language model based on text data, and introduces hot word weighting and self-learning mechanisms to train the hot word probability and generalization ability of the language model;

[0042] Intent recognition module, used to recognize streaming speech in audio signals in real time, extract keywords from streaming speech, and capture and respond to user intent matching keywords;

[0043] The intention matching module is used to convert streaming voice into text strings, identify the intention matching success rate through dynamic sliding window matching, and issue digital human control commands after a successful match.

[0044] According to a third aspect of an embodiment of the present invention, a computer device is provided.

[0045] In some embodiments, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0046] According to a fourth aspect of embodiments of the present invention, a computer-readable storage medium is provided.

[0047] In one embodiment, the computer-readable storage medium stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0048] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0049] 1. Through the integration and optimization of advanced technologies, the accuracy and robustness of speech recognition are improved, and the voice interaction performance of digital humans in noisy environments is significantly improved, thereby providing a smoother and more natural user experience; it has stronger noise resistance, that is, through front-end noise elimination and deep learning noise suppression technology, it can more effectively reduce the interference of background noise on speech signals, and provide clearer speech signals compared to traditional single noise processing methods, thereby improving recognition accuracy.

[0050] 2. By introducing the dual strategies of hot word weighting and self-learning supervision mechanism, the system's response sensitivity to keywords or key sentences is significantly improved, making it more robust.

[0051] 3. Through the streaming real-time speech recognition mechanism, dynamic sliding window intent matching is achieved, which can align with user intent more accurately and provide a good interactive experience.

[0052] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0054] Figure 1 is a flow chart of a digital human voice interaction anti-interference method according to an exemplary embodiment;

[0055] Figure 2 is a schematic diagram of an implementation of a digital human voice interaction anti-interference method according to an exemplary embodiment;

[0056] Figure 3 is a structural schematic diagram of a digital human voice interaction anti-interference system according to an exemplary embodiment;

[0057] Figure 4 The figure is a schematic diagram showing the structure of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0058] The following description and accompanying drawings fully illustrate the specific embodiments of this article so that those skilled in the art can practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of this article includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such structure, device or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements. Each embodiment is described in a progressive manner herein, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0059] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like used herein to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this article and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description herein, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal connection of two elements, it can be a direct connection, or it can be an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0060] As used herein, the term "plurality" means two or more than two, unless otherwise specified.

[0061] In this document, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0062] In this article, the term "and / or" is a description of the association relationship between objects, indicating that three relationships may exist. For example, A and / or B means: A or B, or, A and B.

[0063] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0064] Each module in the device or system of the present application can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above modules.

[0065] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0066] Figure 1 An embodiment of the digital human voice interaction anti-interference method of the present invention is shown.

[0067] In this optional embodiment, the digital human voice interaction anti-interference method includes:

[0068] Step S101, capturing an audio signal, and optimizing the voice quality and sampling rate of the audio signal through voice preprocessing;

[0069] Step S103: construct a language model based on the text data, and introduce hot word weighting and self-learning mechanism to train the hot word probability and generalization ability of the language model;

[0070] Step S105: recognizing streaming speech in the audio signal in real time, extracting keywords in the streaming speech, and capturing and responding to user intentions matching the keywords;

[0071] Step S107: convert the streaming voice into a text string, identify the intention matching success rate through dynamic sliding window matching, and issue a digital human control command after the match is successful.

[0072] In this optional embodiment, when capturing an audio signal and optimizing the voice quality and sampling rate of the audio signal through voice preprocessing, an audio signal including a mixture of human voice and background noise can be captured; a reference signal is obtained and an adaptive filtering algorithm is used to eliminate the echo of the audio signal; and based on the resampling processing of the audio signal, the sampling rate is adjusted to a preset frequency.

[0073] In this optional embodiment, when constructing a language model based on text data, and introducing hot word weighting and self-learning mechanisms to train the hot word probability and generalization ability of the language model, text data can be collected to establish a hot word database containing hot words and key sentences; a real-time speech recognition model is selected as a basis to construct an initial model; the text data is divided into a training set, a validation set and a test set according to a preset ratio, and an input text sequence is set and input into the initial model; a language model is obtained through parameter calculation and iterative training; a hot word weighting coefficient is introduced to adjust the occurrence frequency of each hot word in the hot word database, and the adjusted hot word probability distribution and the normalized hot word probability distribution are calculated; a self-learning mechanism is introduced to perform incremental training on the language model using new recognition data; and based on the training recognition results, high-frequency hot words are analyzed, the training set is regularly updated, and the language model is retrained.

[0074] In this optional embodiment, the calculation formula of the adjusted hot word probability distribution is:

[0075]

[0076] In the formula, Represents the adjusted hot word probability distribution; Indicates the probability distribution of hot words before adjustment; α Indicates the hot word weight coefficient;

[0077] The calculation formula of the normalized hot word probability distribution is:

[0078]

[0079] In the formula, Represents the normalized hot word probability distribution; Represents the sum of all hot words probability distribution.

[0080] In this optional embodiment, the real-time recognition of streaming speech in the audio signal, extraction of keywords in the streaming speech, and capturing and responding to user intentions matching the keywords include:

[0081] Extracting keywords contained in the streaming speech based on complete semantics and context information;

[0082] Preprocess the streaming speech, and then perform punctuation analysis, regular expression matching, and pinyin similarity comparison on the streaming speech in sequence; based on the comprehensive analysis of the results, capture the user's intention and generate a corresponding response.

[0083] In this optional embodiment, when converting the streaming voice into a text string, the intention matching success rate is recognized through dynamic sliding window matching, and after the match is successful, the digital human control command is issued, the real-time recognized streaming voice can be converted into a text string, and through continuous reception and voice processing, a text stream is formed, and the recognized text content is generated in real time; the window parameters of the dynamic sliding window are initialized, and the sliding window moves in the text stream to obtain the window content of each position and match it with the preset intention trigger keyword. If the match is successful, the intention is used as a control instruction and issued to the digital human. If the match fails, the sliding window continues to be moved.

[0084] In this optional embodiment, the window parameters of the sliding window include window size and moving rate.

[0085] Figure 3 An embodiment of the digital human voice interaction anti-interference system of the present invention is shown.

[0086] In this optional embodiment, the digital human voice interaction anti-interference system includes:

[0087] The speech preprocessing module 201 is used to capture the audio signal and optimize the speech quality and sampling rate of the audio signal through speech preprocessing;

[0088] The streaming speech recognition module 203 is used to build a language model based on text data, and introduce hot word weighting and self-learning mechanism to train the hot word probability and generalization ability of the language model;

[0089] The intention recognition module 205 is used to recognize the streaming speech in the audio signal in real time, extract the keywords in the streaming speech, and capture and respond to the user intention matching the keywords;

[0090] The intention matching module 207 is used to convert the streaming voice into a text string, identify the intention matching success rate through dynamic sliding window matching, and issue a digital human control command after a successful match.

[0091] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0092] The present invention aims to improve the voice interaction performance of digital humans in noisy environments, thereby providing a smoother and more natural user experience. It mainly includes the following steps:

[0093] 1. Speech preprocessing stage

[0094] a) Voice pickup: This step is implemented through a high-performance microphone system to accurately capture the near-end human voice that contains the user's intention and the complex audio signal formed by the mixture of the near-end human voice and background noise. This process is crucial for separating effective voice signals from background noise, providing high-quality raw audio data for subsequent voice processing.

[0095] b) Echo cancellation: The composite noise signal after the sound pickup stage is transmitted to the echo cancellation processing link implemented by the WebRTC (Web Real-Time Communication) technology based on the Web end. This link uses an adaptive filtering algorithm to cancel the echo of the audio signal. The principle is to obtain a reference signal (the original audio signal played by the speaker) and compare it with the audio signal captured by the microphone. By modeling the echo path of the environment, the system can predict the echo signal and subtract it from the microphone input signal, thereby effectively eliminating the echo and other related interference from the audio signal, optimizing the audio signal, and preparing clearer input for speech recognition. The application of this technology is crucial to improving the adaptability and accuracy of voice interaction systems in changing environments.

[0096] c) Resampling: After the echo cancellation process is completed, the audio signal will be resampled, the purpose of which is to adjust the sampling rate of the audio signal to 16kHz. In the present invention, the librosa library (a special Python library for audio and music analysis) is used to resample the audio. The sampling process generally involves two steps: 1. Anti-aliasing filtering; 2. Interpolation or downsampling. This step is optimized for the subsequent streaming real-time speech recognition process, because the sampling rate of 16kHz has been proven to be an ideal choice for achieving high-accuracy speech recognition. Resampling not only helps to reduce computational complexity, but also maintains the quality of the speech signal, ensuring that the speech recognition module can process speech data efficiently and accurately.

[0097] 2. Streaming Speech Recognition Stage

[0098] Based on the existing real-time speech recognition module, the present invention introduces a dual strategy of weighting hot words (key words or phrases that need to be accurately recognized in a specific business field) and a self-learning mechanism (mainly for key sentences that need to be accurately recognized) to perform targeted optimization for specific hot words and key sentences (for example, "Lan Dudu" and "Interrupt"). The specific construction steps and training steps are as follows:

[0099] a) Build a hot word database (corpus preparation)

[0100] ①. Text data collection: Before system deployment, first prepare a corpus containing hot words and key sentences (for example, in smart assistant applications, hot words may include business terms or names).

[0101] ②、Construction of hot word library: The file is in UTF8 encoding (a variable-length character encoding) txt text format, with one hot word per line; no special characters other than spaces, line breaks, tabs, etc. appear in the corpus.

[0102] b) Build the initial model

[0103] ① Model selection: Based on the existing Transformer (deep learning model based on attention mechanism), pre-train the real-time speech recognition model (such as Alibaba real-time speech recognition model or various variants of Whisper language model based on deep learning, etc.), which can be regarded as a black box here. Assume that the language model forward network input text sequence is X = (x1, x2, x3, ..., x T ).

[0104] c) Model construction and training

[0105] ①. Training data preparation: Divide the preprocessed text into training set, validation set and test set in a ratio of 8:1:1.

[0106] ②. Forward propagation: Assume that the language model forward network input text sequence is X = (x1, x2, x3, ..., x T ), the output probability distribution is calculated through the Transformer model, and the output is (Probability distribution of recognition word occurrence), where w is the vocabulary.

[0107]

[0108] ③. Calculate the loss function: Use cross entropy loss to calculate the error between the output and the true label.

[0109]

[0110] where y i is the true distribution, Output probability distribution for the network.

[0111] ④. Back propagation: Calculate the gradient of loss to model parameters , and use the Adam optimizer (Adaptive Moment Estimation) to update the parameters and set appropriate hyperparameters (such as learning rate, etc.). The optimization algorithm formula is as follows

[0112]

[0113] Among them, θ is the model parameter and η is the learning rate.

[0114] ⑤. Iterative training: Iterative training is performed through multiple epochs (epochs / rounds, which means a complete dataset passes through the neural network once and returns once) training batches, and each epoch uses a different subset of training data.

[0115] ⑥After each training cycle, use the validation set to evaluate model performance, monitor training loss and validation loss, and prevent overfitting.

[0116] d) Hot word weighting

[0117] Weighted processing is to prioritize key words by increasing the weight of hot words. The specific implementation method is as follows:

[0118] ①. Language model adjustment: Based on the hot word database constructed in step 1a), when training the language model, adjust the frequency of hot words to increase their probability of occurrence. For hot words, we can introduce a weighting coefficient ɑ, where ɑ>1 is the hot word weight coefficient, so that the probability of each hot word is adjusted to:

[0119]

[0120] ②. Probability normalization: The adjusted probability distribution needs to be normalized to ensure that the sum of all probabilities is 1.

[0121]

[0122] in is the adjusted probability distribution of d), is the sum of all hot words probability distribution, is the normalized hot word probability distribution.

[0123] e) Self-learning mechanism part:

[0124] ①. Data collection and preprocessing: The collection must prioritize the identification of accurate key sentences (such as the intent sentences issued) and perform preprocessing (refer to the hot words section).

[0125] ②. Incremental training: The training process refers to step c), that is, incremental training of the model using new recognition data to continuously improve its performance. Assume that the existing data set is , the newly collected data is , then the goal of model updating is to minimize the new loss function : Incremental training can merge new data with original data and use the merged data for training, so that the model can quickly adapt to new text features and improve the generalization ability of the model.

[0126] f) Feedback and Improvement

[0127] The system analyzes high-frequency important words or sentences based on actual recognition results and user feedback. It is assumed that after each time interval T, the new data collected is regularly added to the training set, and the above steps are repeated to retrain or fine-tune the model to ensure that the model can maintain efficient and accurate recognition performance in different scenarios. This move not only greatly improves the system's response sensitivity to keywords or key sentences, but also significantly enhances the recognition accuracy in complex environments, such as strong noise backgrounds or accent variations. The hot word weighting mechanism ensures that these words are highly prioritized for recognition even under unfavorable conditions by increasing the weight of specific keywords or sentences in the recognition process. At the same time, the self-learning mechanism can automatically adjust the recognition strategy according to the actual recognition results and environmental conditions, further refine and optimize the recognition process, and adapt to the changing voice input characteristics and environmental noise.

[0128] 3. Intent recognition stage

[0129] a) Keyword extraction: The person who issues the command sends the intended voice sentence (for example, "Cai Xiaotong, please interrupt me") and performs real-time streaming speech recognition (based on step 2, by collecting audio streams from the microphone in real time for real-time speech recognition). Through keyword extraction, the intent keywords (for example, "interrupt") are obtained. In this process, the intent keywords are set as short as possible, and the intent command covers the complete semantics as much as possible to avoid ambiguity. Complete context information can effectively improve the accuracy of speech recognition.

[0130] b) Fuzzy matching: In order to further enhance the accuracy and flexibility of recognition, the present invention also adopts a fuzzy matching technology, which can not only process standard text data, but also elegantly deal with various non-standard inputs, such as pauses in speech, changes in speech speed, pinyin input, and regular expressions. By comprehensively using a variety of technical means such as punctuation analysis, regular expression matching, and pinyin similarity comparison, the system can accurately capture and respond to user intentions in a wide range of contexts, greatly improving the flexibility of voice command recognition and the naturalness of user interaction, and achieving the purpose of retrieval enhancement.

[0131] The specific steps are as follows:

[0132] ①. Preprocess the streaming speech recognition results: remove extra spaces, special characters, etc., standardize the text input, and achieve the purpose of text cleaning.

[0133] ②. Punctuation analysis: The punctuation segmentation part segments the text according to the punctuation marks to identify the sentence structure and pauses. The sentence structure analysis part assists in understanding the structure of the sentence by analyzing the position and type of punctuation marks, such as identifying interrogative sentences, imperative sentences, etc.

[0134] ③. Regular expression matching: Based on predefined command words, keywords, etc., design corresponding regular expression patterns, match the input text with the regular expression patterns, and identify specific commands or keywords.

[0135] ④. Pinyin similarity comparison: Convert the input text into pinyin, and identify the user's intention by calculating the similarity between the input pinyin and the preset keyword pinyin. You can use algorithms such as edit distance (such as Levenshtein distance) to calculate the similarity, allowing spelling errors or non-standard pronunciation within a certain range, and use the similarity threshold to determine whether the match is successful.

[0136] ⑤. Comprehensive analysis and response: Conduct a comprehensive analysis based on the above analysis results to determine the end-user intent, and generate a corresponding response or perform a corresponding operation based on the identified user intent.

[0137] 4. Intent matching stage

[0138] a) Streaming real-time speech recognition: First, the user's speech is streamed and recognized in real time, which means that the speech data will be converted into text strings in real time. This process relies on efficient speech-to-text conversion (speech recognition) technology, which continuously receives and processes the audio stream collected by the microphone in real time, and generates recognized text content in real time. This real-time processing capability is the basis for achieving smooth human-computer interaction.

[0139] b) Dynamic sliding window matching: Based on real-time speech recognition, this system further uses dynamic sliding window technology to analyze the recognized string. This technology dynamically moves a window in the recognized text stream to match the text in the window with the preset intent trigger keywords. The size and movement rate of the sliding window can be dynamically adjusted according to actual needs to adapt to different speech speeds and language structures, enhancing the flexibility and accuracy of the system.

[0140] The main steps are as follows:

[0141] ①. Dynamic sliding window initialization: Define a sliding window, whose size and movement rate can be dynamically adjusted according to actual needs. The window size (i.e. the number of characters or words contained in the window) and the movement rate (i.e. the step size of each window movement) are two key parameters that can be adjusted according to different application scenarios and speech speeds.

[0142] ②. Text flow analysis: The sliding window moves in the text flow identified by item a, one step at a time. For the window content at each position, the system will match it with the preset intent trigger keyword. The matching algorithm can use simple string matching or more complex pattern matching and regular expressions.

[0143] c) Intent matching and delivery: Once the text in the sliding window successfully matches the intent trigger keyword, it is considered that the user's intent has been successfully identified. At this point, the system will deliver the identified intent to other services for subsequent processing, for example, sending commands to the UE (Unreal Engine) engine to drive the digital human.

[0144] like Figure 2 The embodiments shown are for describing the present invention, which are exemplary and are only used to explain the present invention, and should not be understood as limiting the present invention.

[0145] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above method embodiment are implemented.

[0146] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] In addition, the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiment when executing the computer program.

[0148] In addition, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0149] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0150] The present invention is not limited to the structures which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A digital human voice interaction anti-interference method, characterized in that: include: Capture audio signals and optimize the voice quality and sampling rate of audio signals through voice preprocessing; Based on text data, we build a language model and introduce hot word weighting and self-learning mechanisms to train the hot word probability and generalization ability of the language model. Recognize streaming speech in audio signals in real time, extract keywords from streaming speech, and capture and respond to user intentions that match keywords; Convert streaming speech into text strings, identify the intent matching success rate through dynamic sliding window matching, and issue digital human control commands after successful matching; The language model is constructed based on text data, and hot word weighting and self-learning mechanisms are introduced to train the hot word probability and generalization ability of the language model. Collect text data and build a hot word database containing hot words and key sentences; Select the real-time speech recognition model as the basis to build the initial model; The text data is divided into a training set, a validation set and a test set according to a preset ratio, and an input text sequence is set and input into the initial model; a language model is obtained through parameter calculation and iterative training; Introducing a weighted coefficient of a hot word, adjusting the occurrence frequency of each hot word in the hot word database, and calculating the adjusted hot word probability distribution and the normalized hot word probability distribution; A self-learning mechanism is introduced to perform incremental training on the language model using new recognition data; and based on the training recognition results, high-frequency hot words are analyzed, the training set is regularly updated, and the language model is retrained.

2. The digital human voice interaction anti-interference method according to claim 1, characterized in that: The capturing of the audio signal and optimizing the voice quality and sampling rate of the audio signal through voice preprocessing includes: Capture audio signals containing a mixture of human voice and background noise; Acquire a reference signal, and eliminate the echo of the audio signal using an adaptive filtering algorithm; Based on the resampling process of the audio signal, the sampling rate is adjusted to a preset frequency.

3. The digital human voice interaction anti-interference method according to claim 2 is characterized in that: The calculation formula of the adjusted hot word probability distribution is: , In the formula, Represents the adjusted hot word probability distribution; Indicates the probability distribution of hot words before adjustment; α Indicates the hot word weight coefficient; The calculation formula of the normalized hot word probability distribution is: , In the formula, Represents the normalized hot word probability distribution; Represents the sum of all hot words probability distribution.

4. The digital human voice interaction anti-interference method according to claim 1, characterized in that: The real-time recognition of streaming speech in the audio signal, extraction of keywords in the streaming speech, and capturing and responding to user intentions matching the keywords include: Extracting keywords contained in the streaming speech based on complete semantics and context information; Preprocess the streaming speech, and then perform punctuation analysis, regular expression matching, and pinyin similarity comparison on the streaming speech in sequence; based on the comprehensive analysis of the results, capture the user's intention and generate a corresponding response.

5. The digital human voice interaction anti-interference method according to claim 4 is characterized in that: The converting of the streaming voice into a text string, identifying the intention matching success rate through dynamic sliding window matching, and issuing the digital human control command after the matching is successful include: Convert the real-time recognized streaming speech into text strings, and form a text stream through continuous reception and speech processing, and generate the recognized text content in real time; Initialize the window parameters of the dynamic sliding window, move the sliding window in the text stream, obtain the window content at each position, and match it with the preset intention trigger keyword. If the match is successful, the intention is sent to the digital human as a control instruction. If the match fails, continue to move the sliding window.

6. The digital human voice interaction anti-interference method according to claim 5, characterized in that: The window parameters of the sliding window include window size and moving rate.

7. A digital human voice interaction anti-interference system, used to implement the digital human voice interaction anti-interference method according to any one of claims 1 to 6, characterized in that: The system includes: A speech preprocessing module is used to capture audio signals and optimize the speech quality and sampling rate of the audio signals through speech preprocessing; The streaming speech recognition module is used to build a language model based on text data, and introduces hot word weighting and self-learning mechanisms to train the hot word probability and generalization ability of the language model; Intent recognition module, used to recognize streaming speech in audio signals in real time, extract keywords from streaming speech, and capture and respond to user intent matching keywords; The intention matching module is used to convert streaming voice into text strings, identify the intention matching success rate through dynamic sliding window matching, and issue digital human control commands after a successful match.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.