Intelligent voice interaction method and device

Abstract

The invention discloses an intelligent voice interaction method and device. The interaction method comprises the following steps: acquiring an original audio signal through an audio input module and converting the original audio signal into a digital audio signal; the wake-up module performs feature extraction and classification on the digital audio signals; the main control module performs keyword detection on the extracted features according to a wake-up word detection model; the audio coding and decoding module converts an original audio signal into a digital audio signal, and the main control module sends the digital audio signal to the cloud server to generate reply characters and digital audio data; the main control module sends reply characters to the interaction module to be displayed, the audio encoding and decoding module converts digital audio data into analog audio signals and sends the analog audio signals to the audio power amplifier module, the audio power amplifier module converts the analog audio signals into sound, and audio playing is completed; and the main control module re-starts the wake-up word detection model to carry out keyword detection so as to realize continuous voice dialogue. According to the design, manual operation is not needed, and low-power-consumption operation can be achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent voice control technology, specifically to an intelligent voice interaction method and device. Background Technology

[0002] With the continuous development of voice interaction and intelligent control technologies, more and more devices are equipped with voice recognition capabilities and can perform related operations based on the input voice content. To achieve real-time voice interaction and recognition, the voice interaction function will be in continuous operation. However, due to the high power consumption of voice interaction and control, the system experiences high unnecessary energy consumption. If a manual voice recognition switch is used for voice interaction, the voice input will only be executed after the switch is activated, requiring manual operation before each voice interaction. This prevents fully automated voice interaction and reduces the convenience of intelligent voice interaction. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings and problems of high power consumption and poor convenience in the existing technology, and to provide an intelligent voice interaction method and device with low power consumption and good convenience.

[0004] To achieve the above objectives, the technical solution of the present invention is: an intelligent voice interaction method, the interaction method comprising the following steps:

[0005] The audio input module acquires the raw audio signal and converts it into a digital audio signal.

[0006] The wake-up module extracts and classifies features from digital audio signals;

[0007] The main control module performs wake word detection on the extracted features based on the wake word detection model; if no wake word is detected, the intelligent voice interaction device remains in low-power standby mode; if a wake word is detected, the main control module outputs a wake-up signal and stops the wake word detection model.

[0008] The audio codec module collects the original audio signal from the audio input module in real time based on the wake-up signal, and converts the original audio signal into a digital audio signal. The main control module sends the digital audio signal to the cloud server to generate reply text and digital audio data.

[0009] The main control module sends the reply text to the interaction module for display. The audio codec module converts the digital audio data into an analog audio signal and sends it to the audio amplifier module. The audio amplifier module converts the analog audio signal into sound to complete the audio playback.

[0010] The main control module re-enables the wake word detection model to detect wake words, repeats the above steps, and achieves continuous voice dialogue.

[0011] Before performing feature extraction and classification on the digital audio signal, the digital audio signal is preprocessed and the continuous digital audio signal is divided into frame-by-frame audio signals. Then, a window function is applied to each frame of audio signal for windowing.

[0012] The preprocessing of the digital audio signal includes:

[0013] Analyze the audio characteristics of digital audio signals without speech segments, estimate the spectral characteristics of background noise, design an optimal filter by calculating the power spectrum ratio of speech to background noise, and remove background noise using the optimal filter.

[0014] The digital audio signal is delayed and filtered to match the transmission path of the echo received by the audio input module. The filter coefficients are updated to make the filter output close to the echo signal, and the predicted echo is subtracted from the digital audio signal.

[0015] High-pass filters are used to enhance the energy of the high-frequency components in digital audio signals, thereby increasing the high-frequency resolution of the digital audio signals.

[0016] The feature extraction and classification of digital audio signals includes:

[0017] Perform a short-time Fourier transform on each frame of the windowed audio signal to convert the time-domain signal to the frequency domain and obtain the spectrum of each frame;

[0018] Based on the Mel frequency scale, a set of filters is uniformly distributed in the Mel frequency domain, and then these filters in the Mel frequency domain are converted back to the linear frequency domain to obtain the corresponding Mel filter bank.

[0019] The spectrum of each frame of audio signal is passed through a Mel filter bank. The output of each filter is calculated. The logarithm of each output of the Mel filter bank is taken to obtain the logarithmic Mel energy.

[0020] The log-Mel energy is subjected to discrete cosine transform to convert it from the frequency domain to the cepstral domain, and the main spectral features are extracted.

[0021] The step of detecting wake words from the extracted features includes:

[0022] The extracted spectral features are normalized and then input into the input layer of the neural network.

[0023] The convolutional layers of a neural network perform convolution operations on the input features to extract local features:

[0024] Apply an activation function to the output of the convolutional layer;

[0025] The feature vectors are flattened and then input into a fully connected layer for global information integration.

[0026] Use a function in the output layer to convert the output of the fully connected layer into a probability distribution:

[0027] The classification decision is made based on the set probability threshold. If the probability of the wake word category is higher than the probability threshold, it is determined to be a wake word; otherwise, it is determined to be a non-wake word.

[0028] The main control module sends the digital audio signal to the cloud server to generate reply text and digital audio data, including:

[0029] The main control module starts recording and continuously reads digital audio signals until it detects multiple consecutive frames of silence or reaches the maximum recording time. Then, it stops recording and sends the recorded audio data to the cloud server. The cloud server converts the audio data into text, then inputs the text into the AI ​​model to generate reply text and digital audio data, and sends it to the main control module.

[0030] The continuous reading of digital audio signals until multiple consecutive frames of silence are detected or the maximum recording duration is reached includes:

[0031] Each frame of the digital audio signal is detected sequentially. When no data is detected in the current frame, the digital audio signal of the next frame is detected until data is detected and recording begins. Each frame of the digital audio signal is sampled and counted simultaneously.

[0032] The system detects whether the sample value of each frame of digital audio signal is lower than a certain threshold. If it is lower than the threshold, it is considered as silent and is counted. When silence is detected for multiple consecutive frames and the frame count reaches the set number of silent frames, the recording ends.

[0033] Recording ends when the number of frames in the digital audio signal reaches the set maximum recording duration.

[0034] The process of inputting text into an AI model to generate response text and digital audio data includes:

[0035] The text is fed into the AI ​​model via API. The AI ​​model converts the input text into a token sequence and maps it into a high-dimensional vector. Then, it captures the long-range dependencies and semantic associations in the input sequence. Next, it generates the output sequence step by step through autoregressive decoding. At each step, it selects the next token with the highest probability based on the current context and the tokens generated in the past. The generated token sequence is decoded into the response text of natural language text and calls the TTS service to combine sentiment parameters to synthesize human-like digital audio data.

[0036] An intelligent voice interaction device, applied to an intelligent voice interaction method, includes: a main control module, an audio input module, an audio codec module, a wake-up module, and an audio amplifier module. The output terminal of the audio input module is connected to the input terminals of the audio codec module and the wake-up module, respectively. The output terminal of the audio codec module is connected to the input terminal of the audio amplifier module. The output terminal of the wake-up module is connected to the main control module. The main control module is connected to the audio codec module and the audio amplifier module. The main control module is connected to a cloud server via a wireless module.

[0037] The audio input module is used to acquire audio signals;

[0038] The audio codec module is used to convert the acquired audio signal into a digital signal and send the digital signal to the main control module for processing; it also converts the digital audio signal processed by the main control module into an analog signal and sends the analog signal to the audio power amplifier module.

[0039] The wake-up module is used to detect the wake-up word in the audio signal and send a signal to the main control module;

[0040] The main control module is used to convert digital signals into text, transmit the text to the cloud server to generate reply text, convert the reply text into digital audio signals, switch from standby mode to working mode to start interaction according to the signal sent by the wake-up module, and switch from working mode to standby mode after the interaction ends.

[0041] The audio input module includes a microphone and an electret microphone. The microphone is connected to the input terminal of the audio codec module, and the electret microphone is connected to the input terminal of the wake-up module.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] 1. In the intelligent voice interaction method and device of this invention, audio signals are collected through an audio input module, and feature extraction and separation are performed using a wake-up module. When the main control module does not detect a wake-up word, it can achieve low-power operation. When a wake-up word is detected, the main control module switches from standby mode to working mode, transmitting the digital signal converted by the audio codec module to an external cloud server to generate response text. Voice interaction is achieved by converting the response text into audio. In use, simply speaking the wake-up word automatically activates the main control module, eliminating the need for manual operation. After interaction, the main control module returns to standby mode, thus achieving low-power operation. Therefore, this invention has low power consumption and good convenience.

[0044] 2. In the intelligent voice interaction method and device of this invention, by analyzing the correlation between digital audio signals and echoes, the suppression intensity can be dynamically adjusted to avoid excessive distortion of the digital audio signals; by preprocessing the digital audio signals, the high-frequency resolution of the speech can be enhanced, and the energy of the high-frequency components in the digital audio signals is relatively increased, making subsequent feature extraction more effective; the audio signals are converted into multiple short sequences, facilitating feature extraction for each frame separately; by applying a window function, spectral leakage can be reduced, making the audio signals appear more like periodic signals in the time domain. Therefore, the speech clarity and accuracy of this invention are high.

[0045] 3. In the intelligent voice interaction method and device of this invention, the time-varying spectral information can be obtained through short-time Fourier transform, and the energy level of the frequency component can also be obtained; the Mel scale can simulate human auditory perception, making the frequency more consistent with human auditory characteristics; by taking the logarithm of the Mel filter bank, the dynamic range of energy can be compressed, making subsequent calculations more stable and more consistent with the human auditory system's perception of sound intensity; by performing discrete cosine transform, the correlation between frequencies can be removed, making the extracted features more independent and facilitating subsequent processing by speech recognition and wake-word detection algorithms. Therefore, the speech recognition accuracy of this invention is high.

[0046] 4. In the intelligent voice interaction method and device of this invention, a neural network model is used to detect wake words by extracting features. Probability is introduced to determine whether the extracted features are wake words, which can accurately determine the triggering of wake words. By employing an AI model to achieve text response, long-distance semantic relationships can be effectively captured with limited hardware resources, improving the ability and efficiency of processing long texts. Therefore, this invention has high accuracy and fast response speed.

[0047] 5. In the intelligent voice interaction method and device of this invention, a wake-up module performs low-power standby listening before interaction. When the user speaks, the audio input module collects the audio, and the wake-up module sends a signal to the main control module. After the main control module retrieves the wake-up word, it switches from standby to working state to initiate interaction. After receiving the user's text content, the main control module performs semantic understanding and dialogue logic processing on the cloud server through a built-in algorithm or via a wireless module, generates a reply text, converts the reply text into a digital audio signal, and then converts it into an analog signal through an audio codec module, which is then sent to the audio amplifier module for playback. After interaction, the main control module enters standby state, thus achieving low-power operation. Therefore, this invention has low power consumption and good convenience. Attached Figure Description

[0048] Figure 1 This is a flowchart of an intelligent voice interaction method in this invention.

[0049] Figure 2 This is a connection block diagram of an intelligent voice interaction device according to the present invention.

[0050] Figure 3 This is the circuit diagram of the main control module in this invention.

[0051] Figure 4 This is a circuit diagram of the audio encoding / decoding module in this invention.

[0052] Figure 5 This is a circuit diagram of the audio power amplifier module in this invention.

[0053] Figure 6 This is a circuit diagram of the interactive module in this invention.

[0054] Figure 7 This is the circuit diagram of the protector in this invention.

[0055] Figure 8 This is a circuit diagram of the Type-C connector in this invention.

[0056] Figure 9 This is the circuit diagram of the serial port in this invention.

[0057] Figure 10 This is a circuit diagram of the charging circuit in this invention.

[0058] Figure 11 This is a circuit diagram of the power switching circuit in this invention.

[0059] Figure 12 This is the circuit diagram of the power switch in this invention.

[0060] Figure 13 This is a circuit diagram of the storage module in this invention.

[0061] Figure 14 This is a circuit diagram of the wake-up module in this invention.

[0062] Figure 15 This is a circuit diagram of the audio input module and the wake-up module in this invention.

[0063] Figure 16 This is the circuit diagram of the LED lamp in this invention.

[0064] Figure 17 This is the circuit diagram of the BOOT button in this invention.

[0065] In the diagram: Cloud server 1, Audio input module 2, Microphone 21, Electret microphone 22, Audio codec module 3, Wake-up module 4, Audio amplifier module 5, Interaction module 6, Power module 7, Charging circuit 71, Power switching circuit 72, Power switch 73, Wireless module 8, RF antenna 81, Main control module 9, Main control chip 91, Crystal oscillator 92, Protector 93, Type-C connector 94, Serial port 95, Storage module 10, LED light 11, BOOT button 12, Cable connection board 13. Detailed Implementation

[0066] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Example 1:

[0068] See Figure 1 A smart voice interaction method, the interaction method comprising the following steps:

[0069] The original audio signal is acquired through audio input module 2 and converted into a digital audio signal;

[0070] Wake-up module 4 performs feature extraction and classification on digital audio signals;

[0071] The main control module 9 performs wake word detection on the extracted features based on the wake word detection model; if no wake word is detected, the intelligent voice interaction device remains in low-power standby mode; if a wake word is detected, the main control module 9 outputs a wake-up signal and stops the wake word detection model.

[0072] The audio codec module 3 collects the original audio signal from the audio input module 2 in real time according to the wake-up signal, and converts the original audio signal into a digital audio signal. The main control module 9 starts recording and continuously reads the digital audio signal until it detects multiple consecutive frames of silence or reaches the maximum recording time. Then it stops recording and sends the recorded audio data to the cloud server 1. The cloud server 1 converts the audio data into text, then inputs the text into the AI ​​model to generate reply text and digital audio data, and sends it to the main control module 9.

[0073] The main control module 9 sends the reply text to the interaction module 6 for display. The audio codec module 3 converts the digital audio data into an analog audio signal and sends it to the audio amplifier module 5. The audio amplifier module 5 converts the analog audio signal into sound to complete the audio playback.

[0074] The main control module 9 re-enables the wake word detection model to detect wake words, repeats the above steps, and realizes continuous voice dialogue.

[0075] In this embodiment, the main control module 9 uses an ESP32 chip, which can run the WakeNet model. An I2S interface is configured on the ESP32 chip, and the sampling rate, bit depth, and number of channels are set. The I2S driver of the ESP32 chip is used to read digital audio signals. The ESP32 chip reads these data through the I2S interface and stores them in a memory buffer. I2S communication supports full-duplex and half-duplex communication and supports master / slave mode. Audio signals are acquired through the audio input module 2, and feature extraction and separation are performed by the wake-up module 4. When the main control module 9 does not detect a wake word, it can achieve low-power operation. When a wake word is detected, the main control module 9 switches from standby state to working state and transmits the digital signal converted by the audio codec module 3 to the external cloud server 1 to generate reply text. Voice interaction is achieved by converting the reply text into audio. When in use, simply saying the wake word will automatically put the main control module 9 into working state without manual operation. After the interaction, the main control module 9 will enter standby state, thus achieving low-power operation.

[0076] Example 2:

[0077] The basic content is the same as in Example 1, except that:

[0078] Before performing feature extraction and classification on the digital audio signal, the digital audio signal is preprocessed, the audio features of the digital audio signal without speech segments are analyzed, the spectral characteristics of the background noise are estimated, the power spectrum ratio of speech to background noise is calculated, the optimal filter is designed, and the background noise is removed by the optimal filter.

[0079] The digital audio signal is delayed and filtered to match the transmission path of the echo received by the audio input module 2. The filter coefficients are updated to make the filter output close to the echo signal, so that the digital audio signal is reduced by the predicted echo.

[0080] High-pass filters are used to enhance the energy of the high-frequency components in digital audio signals, thereby increasing the high-frequency resolution of digital audio signals.

[0081] The continuous digital audio signal is divided into frame-by-frame audio signals, and then a window function is applied to each frame of audio signal for windowing.

[0082] In this embodiment, algorithms such as spectral subtraction or Wiener filtering are used to analyze the audio features of digital audio signals without speech segments. Recursive averaging or smoothing techniques are employed to update the noise model and adapt to dynamic changes in environmental noise. In spectral subtraction, the estimated noise spectrum is subtracted from the noisy speech spectrum. In the Wiener filtering algorithm, the optimal filter is designed by calculating the power spectrum ratio of speech to noise. The filter coefficients are updated using the least mean square (LMS) or recursive least squares (RLS) algorithm to make the filter output as close as possible to the echo signal, thereby subtracting the predicted echo from the microphone input. When residual echo still exists, echo suppression (ER) is used for further attenuation. By analyzing the correlation between speech and echo, the suppression intensity is dynamically adjusted to avoid excessive distortion. Finally, speech enhancement is performed to improve speech clarity and intelligibility and improve subjective listening experience.

[0083] Example 3:

[0084] The basic content is the same as Example 2, except that:

[0085] The feature extraction and classification of digital audio signals includes:

[0086] Perform a short-time Fourier transform on each frame of the windowed audio signal to convert the time-domain signal to the frequency domain and obtain the spectrum of each frame;

[0087] Based on the Mel frequency scale, a set of filters is uniformly distributed in the Mel frequency domain, and then these filters in the Mel frequency domain are converted back to the linear frequency domain to obtain the corresponding Mel filter bank.

[0088] The spectrum of each frame of audio signal is passed through a Mel filter bank, the output value of each filter is calculated, and the logarithm of each output value of the Mel filter bank is taken to obtain the logarithmic Mel energy.

[0089] The log-Mel energy is subjected to discrete cosine transform to convert it from the frequency domain to the cepstral domain, and the main spectral features are extracted.

[0090] In this embodiment, the Mel filter bank consists of 26 filters, each of which is a triangular filter. Adjacent filters overlap. The output value of each filter is actually the sum of the energy of the audio signal in the corresponding Mel frequency range. The Hamming window function is used, and the specific formula is as follows:

[0091] ;

[0092] ;

[0093] in, The length of the frame. This is the index of the intra-frame sample;

[0094] Calculate the modulus of the short-time Fourier transform result to obtain the amplitude spectrum. For the first... The amplitude spectrum of each frequency component is:

[0095] ;

[0096] in, and These are the real and imaginary parts of the short-time Fourier transform result, respectively;

[0097] The formula for converting Mel frequency to Hertz frequency is:

[0098] ;

[0099] in, Frequency (Hertz);

[0100] The output of the triangular filter is:

[0101] ;

[0102] in, The number of triangular filters. It is the first A triangular filter at frequency Weight at each location;

[0103] The first logarithmic energy spectrum Each component for:

[0104] ;

[0105] The formula for the discrete cosine transform is:

[0106] ;

[0107] ;

[0108] in, This represents the number of coefficients in the discrete cosine transform. To determine the characteristics of the Mel-spectrum cepstral coefficients, the first 12-13 coefficients after the discrete cosine transform are taken as the Mel-spectrum cepstral coefficients.

[0109] Example 4:

[0110] The basic content is the same as in Example 1, except that:

[0111] The step of detecting wake words from the extracted features includes:

[0112] The extracted spectral features are normalized and then input into the input layer of the neural network.

[0113] The convolutional layers of a neural network perform convolution operations on the input features to extract local features:

[0114] Apply an activation function to the output of the convolutional layer;

[0115] The feature vectors are flattened and then input into a fully connected layer for global information integration.

[0116] Use a function in the output layer to convert the output of the fully connected layer into a probability distribution:

[0117] The classification decision is made based on the set probability threshold. If the probability of the wake word category is higher than the probability threshold, it is determined to be a wake word; otherwise, it is determined to be a non-wake word.

[0118] In this embodiment, the extracted spectral features are organized into a feature vector, which contains Mel-spectral cepstral coefficients at multiple time steps to capture the temporal characteristics of the audio signal. The wake-up word detection model adopts the WakeNet neural network model. In order to improve the training efficiency and stability of the neural network, the feature vector is normalized by scaling each feature value to between 0 and 1, or by standardization (subtracting the mean and dividing by the standard deviation).

[0119] The WakeNet neural network model specifically includes:

[0120] The input layer is used to receive feature vectors;

[0121] Hidden layers consist of multiple convolutional layers and fully connected layers. Convolutional layers extract local features, while fully connected layers integrate global information. The convolutional layers perform convolution operations on the input features to extract local features, as shown in the following formula:

[0122] ;

[0123] in, For the extracted local features, The element at the corresponding position in the input feature map. For convolution kernel weights, For bias terms, , It is the position index of the convolution kernel. The input feature map and the current output position The position of the element within the corresponding local region;

[0124] The ReLU activation function is applied to the extracted local features, setting the parts of the convolutional layer output that are less than 0 to 0, and only retaining features that are greater than or equal to 0. The specific formula is as follows:

[0125] ;

[0126] The processed features are directly passed to the next convolutional layer to continue participating in further feature extraction and fusion. The local features extracted through multiple rounds of convolution are input into the fully connected layer and integrated with global information. The specific formula is as follows:

[0127] ;

[0128] in, For the output of the fully connected layer, This is the weight matrix. For the input feature vector, The bias vector is used as the input and output layer for the integrated features to complete the detection of wake words;

[0129] In the output layer, the softmax function is used to convert the output of the fully connected layer into a probability distribution:

[0130] ;

[0131] in, For the number of categories, It is the natural logarithm.

[0132] Example 5:

[0133] The basic content is the same as in Example 1, except that:

[0134] The continuous reading of digital audio signals until multiple consecutive frames of silence are detected or the maximum recording duration is reached includes:

[0135] Each frame of the digital audio signal is detected sequentially. When no data is detected in the current frame, the digital audio signal of the next frame is detected until data is detected and recording begins. Each frame of the digital audio signal is sampled and counted simultaneously.

[0136] The system detects whether the sample value of each frame of digital audio signal is lower than a certain threshold. If it is lower than the threshold, it is considered as silent and is counted. When silence is detected for multiple consecutive frames and the frame count reaches the set number of silent frames, the recording ends.

[0137] Recording ends when the number of frames in the digital audio signal reaches the set maximum recording duration.

[0138] In this embodiment, each frame typically contains 20-40 milliseconds of audio data. When the sampling rate is 16 kHz and the frame length is 25 milliseconds, each frame contains 400 sampling points. A function checks each sampling point in the audio buffer to see if it is below a certain threshold. If several frames of silence are detected consecutively, it is assumed that the user has stopped speaking, and a special text frame is sent to notify the cloud server 1 that the audio stream has ended.

[0139] Example 6:

[0140] The basic content is the same as in Example 1, except that:

[0141] The process of inputting text into an AI model to generate response text and digital audio data includes:

[0142] The text is fed into the AI ​​model via API. The AI ​​model converts the input text into a token sequence and maps it into a high-dimensional vector. Then, it captures the long-range dependencies and semantic associations in the input sequence. Next, it generates the output sequence step by step through autoregressive decoding. At each step, it selects the next token with the highest probability based on the current context and the tokens generated in the past. The generated token sequence is decoded into the response text of natural language text and calls the TTS service to combine sentiment parameters to synthesize human-like digital audio data.

[0143] In this embodiment, the AI ​​model uses the DeepSeek large model. In the streaming scheme, after the DeepSeek large model generates the response text, it first formats the text into JSON format, and then sends a request via the WebSocket protocol to securely transmit the data to the specified API endpoint of the cloud server 1. The cloud server 1 receives the data and performs further processing or storage to support subsequent application operations. In the simple HTTP scheme, the main control module 9 actively requests the result. The main control module 9 uses the requests library to send HTTP requests and communicate with the DeepSeek API. After receiving the text, the main control module 9 needs to process it locally and process the response result according to the response status code. If successful, the generated text is printed; if unsuccessful, an error message is printed.

[0144] DeepSeek's large model has been optimized in version 3 to improve logical coherence and knowledge coverage. By adopting a multi-head latent attention mechanism, the key and value of multiple attention heads are projected into a low-dimensional shared latent vector space, replacing the traditional head-by-head storage method. During the inference phase, only the latent vector needs to be updated, without storing all historical KV caches, thereby reducing memory usage to 1 / 4 of the traditional Transformer architecture. This improvement enables the model to effectively capture long-distance semantic relationships with limited hardware resources when processing long documents (such as legal text summarization, long novel translation, etc.), improving the ability and efficiency of processing long texts.

[0145] To address the quadratic increase in computational complexity of the traditional Transformer architecture's full attention mechanism in long text scenarios, a dynamic hierarchical sparsity strategy is employed, combining coarse-grained token compression and fine-grained token selection. For coarse-grained compression, clustering or semantic segmentation divides long sequences into multiple sub-blocks, retaining only the global association information between key blocks to reduce redundant computation. When processing 64k-length text, it can first filter out paragraphs containing core semantics before performing fine-grained processing. For fine-grained selection, within the retained blocks, attention scoring dynamically selects locally important tokens, such as high-frequency wake words or logical connectors, ensuring local contextual accuracy. Simultaneously, the computational process is optimized for modern GPU architectures, adjusting arithmetic intensity to maximize hardware parallelism.

[0146] DeepSeek's large model introduces a dynamic routing algorithm in the MoE architecture. DeepSeek's dynamic routing algorithm adjusts the expert utilization rate by dynamically biasing. When dealing with different types of text tasks (such as text generation, code parsing, etc.), it can more rationally select the most suitable expert network for processing according to the complexity of the input task, significantly reducing computational redundancy and improving inference efficiency.

[0147] During the training of the DeepSeek large model, when the model generates a continuous text, the multi-token prediction technology can better capture the semantic relationships between the text before and after it, avoiding the situation where the generated content is disjointed, thereby improving the quality of the generated text.

[0148] DeepSeek's large model introduces an adaptive gating mechanism in its dynamic attention layer, which can dynamically adjust the weight distribution of the attention head according to the semantic complexity of the input sequence. When processing technical documents, the model will automatically enhance the attention weight of tokens related to technical terms, improve the ability to capture long-distance dependencies, and thus more accurately understand and process complex semantic relationships in the text.

[0149] In its feedforward neural network (FFN), DeepSeek employs a mixed-precision computation mode of "input layer FP16 - hidden layer FP32 - output layer FP16". This mode reduces GPU memory usage by approximately 40% while maintaining accuracy, enabling more efficient model training with limited hardware resources, thus improving training efficiency and reducing costs. This enhances computational efficiency, the ability to handle long texts, and generation speed.

[0150] Example 7:

[0151] See Figure 2 An intelligent voice interaction device includes: a main control module 9, an audio input module 2, an audio codec module 3, a wake-up module 4, and an audio amplifier module 5. The output terminal of the audio input module 2 is connected to the input terminals of the audio codec module 3 and the wake-up module 4, respectively. The output terminal of the audio codec module 3 is connected to the input terminal of the audio amplifier module 5. The output terminal of the wake-up module 4 is connected to the main control module 9. The main control module 9 is connected to the audio codec module 3 and the audio amplifier module 5. The main control module 9 is connected to a cloud server 1 via a wireless module 8.

[0152] The audio input module 2 is used to acquire audio signals;

[0153] The audio codec module 3 is used to convert the acquired audio signal into a digital signal and send the digital signal to the main control module 9 for processing; it also converts the digital audio signal processed by the main control module 9 into an analog signal and sends the analog signal to the audio power amplifier module 5.

[0154] The wake-up module 4 is used to detect the wake-up word in the audio signal and send a signal to the main control module 9;

[0155] The main control module 9 is used to convert digital signals into text, transmit the text to the cloud server 1 to generate reply text, convert the reply text into digital audio signals, switch from standby state to working state to start interaction according to the signal sent by the wake-up module 4, and switch from working state to standby state after the interaction ends.

[0156] In this embodiment, before use, the wake-up module 4 performs low-power standby listening. When the user speaks, the audio input module 2 collects the audio. After the wake-up module 4 detects the wake-up word, it sends a signal to the main control module 9, triggering the main control module 9 to switch from standby to working state and start the interaction. At the same time, the audio codec module 3 converts the audio into a digital signal and sends it to the main control module 9. After receiving the user's text content, the main control module 9 transmits it to the cloud server 1 through the built-in algorithm or through the wireless module 8 for semantic understanding and dialogue logic processing, generates reply text, and then converts the reply text into a digital audio signal. Then, it converts it into an analog signal through the audio codec module 3 and sends it to the audio amplifier module 5 for playback. The audio amplifier module 5 plays the reply voice to realize voice interaction. After the interaction ends, the main control module 9 goes into standby mode, and the wake-up module 4 continues to listen.

[0157] Example 8:

[0158] The basic content is the same as Example 7, except that:

[0159] See Figures 3 to 5 The main control module 9 includes a main control chip 91 and a crystal oscillator 92. The main control chip 91 is an ESP32-C3. Pins 29, 30, 31, and 32 of the main control chip 91 are connected to the input terminal of the crystal oscillator 92, and the output terminal of the crystal oscillator 92 is grounded. Pins 8 and 9 of the main control chip 91 are connected to the audio codec module 3 via I2C signal lines. Pins 10, 12, 13, 14, and 16 of the main control chip 91 are connected to the audio codec module 3 via an I2S bus. Pin 18 of the main control chip 91 is connected to the audio power amplifier module 5.

[0160] In this embodiment, pin 1 of crystal oscillator 92 is connected to pin 30 of main control chip 91 through third inductor L3, pin 3 is connected to pin 29 of main control chip 91, pin 1 is connected to pin 3 through twenty-third capacitor C23 and twenty-first capacitor C21 in sequence, and pins 2 and 4 of crystal oscillator 92 are grounded.

[0161] Audio codec module 3 includes an audio codec chip, model ES8311. Pins 1 and 19 of the audio codec chip are connected to the main control chip 91 via I2C signal lines. Pins 2, 6, 7, 8, and 9 of the audio codec chip are connected to the main control chip 91 via I2S signal lines. Pins 3 and 4 of the audio codec chip are both connected to the power supply terminal. One end of the ninth capacitor C9 is connected to pin 3 of the audio codec chip, and the other end is connected to the power ground terminal. One end of the tenth capacitor C10 is connected to pin 4 of the audio codec chip, and the other end is connected to the power ground terminal. Pins 20, 6, 7, 8, and 9 of the audio codec chip are connected to the main control chip 91 via I2S signal lines. Pin 21 is connected to the power ground terminal. Pins 17 and 18 of the audio codec chip are connected to the audio input module 2. Pins 14, 15, and 16 of the audio codec chip are connected to the sixth capacitor C6, the fifth capacitor C5, and the fourth capacitor C4, respectively, and then connected to the power ground terminal. Pins 12 and 13 of the audio codec chip are connected to the audio power amplifier module 5. Pin 11 of the audio codec chip is connected to the power supply terminal. Pin 11 of the audio codec chip is connected to the twelfth capacitor C12 and then connected to the power ground terminal. The power supply terminal is connected to the power module 7 through the sixth resistor R6. The power ground terminal is grounded through the fifth resistor R5.

[0162] The audio amplifier module 5 includes an audio amplifier chip, model NS4150B. Pin 3 of the audio amplifier chip is connected sequentially to resistor R9 (ninth resistor), capacitor C17 (seventeenth capacitor), and capacitor C7 (seventh capacitor). Pin 1 of the audio amplifier chip is connected to the power amplifier ground terminal via a transistor and resistor R7 (seventh resistor). Pin 2 of the audio amplifier chip is connected to the power amplifier ground terminal via capacitor C19 (nineteenth capacitor). Pins 5 and 8 of the audio amplifier chip are connected to the two input terminals of the wire-to-board connector 13. Pin 6 of the audio amplifier chip is connected to the system power supply terminal. Pin 7 of the audio amplifier chip is connected to the power amplifier ground terminal. One end of capacitor C18 (eighteenth capacitor) and capacitor C34 (thirty-fourth capacitor) is connected to the power amplifier ground terminal, and the other end is connected to the system power supply terminal. The two input terminals of the wire-to-board connector 13 are connected to the power amplifier ground terminal via capacitor C14 (fourteenth capacitor) and capacitor C15 (fifteenth capacitor). The power amplifier ground terminal is grounded via resistor R10 (tenth resistor).

[0163] Example 9:

[0164] The basic content is the same as Example 8, except that:

[0165] See Figure 3 and Figure 6 The main control chip 91 has pins 8 and 9 connected to an interaction module 6 via an I2C signal line. The interaction module 6 is used to display the text converted by the main control module 9 and the generated reply text.

[0166] In this embodiment, the interaction module 6 includes an OLED display screen, model OLED12864. Pins 8 and 9 of the main control chip 9 are connected to the SDA and SCL terminals of the OLED display screen via I2C_SDA and I2C_SCL signal lines, respectively. The VCC terminal of the OLED display screen is connected to the power module 7, and the GND terminal of the OLED display screen is grounded.

[0167] Example 10:

[0168] The basic content is the same as Example 8, except that:

[0169] See Figure 3 , Figures 7 to 9 Pins 25 and 26 of the main control chip 91 are connected to one end of the Type-C connector 94 via a protector 93. The other end of the Type-C connector 94 is connected to the power module 7. Pin 19 of the main control chip 91 is connected to the serial port 95 via a nineteenth resistor R19. Pin 20 of the main control chip 91 is connected to the serial port 95. The input terminal of the serial port 95 is connected to the power module 7.

[0170] In this embodiment, pin 2 of the main control chip 91 is connected to the power module 7 via the first inductor L1. One end of the twenty-sixth capacitor C26 and the twenty-seventh capacitor C27 is connected to the power module 7, and the other end is grounded. One end of the twenty-eighth capacitor C28 is connected to pin 3 of the main control chip 91, and the other end is grounded. Pin 7 of the main control chip 91 is connected to the power module 7 via the eleventh resistor R11. One end of the twenty-second capacitor C22 is connected to pin 7 of the main control chip 91, and the other end is grounded. Pin 11 of the main control chip 91 is connected to the thirty-third capacitor C33 and then grounded, and is also connected to the power module 7.

[0171] The protector 93 is model USBLC6-2SC6, and the Type-C connector 94 is model TYPE-C 16PIN 2MD(073). Pins 1, 3, 4, 5, and 6 of the protector 93 are connected to pins 5, 6, 7, 2, and 8 of the Type-C connector 94, respectively. Pin 2 of the protector 93 is grounded. Pins 1, 12, 13, and 14 of the Type-C connector 94 are grounded. Pins 4 and 10 of the Type-C connector 94 are grounded through the fourth resistor R4 and the tenth resistor R10, respectively. Pin 11 of the Type-C connector 94 is connected to the power module 7.

[0172] The serial port 95 can connect to a 4G module (such as a 4G communication module) to enable the device to access the 4G network. The 4G module communicates with the main control chip via UART1 (the main control chip 91 sends AT commands to control the 4G module, and the 4G module returns network data or status information), enabling the device to perform remote communication (such as data upload, cloud interaction, receiving remote commands, etc.) through the 4G network, thus expanding the device's network connectivity.

[0173] Example 11:

[0174] The basic content is the same as Example 10, except that:

[0175] See Figures 3 to 12 The power module 7 includes a charging circuit 71, a power switching circuit 72, and a power switch 73. The input terminal of the charging circuit 71 is connected to the other end of the Type-C connector 94. One output terminal of the charging circuit 71 is connected to the input terminal of the power switching circuit 72, and the other output terminal of the charging circuit 71 is connected to the line-to-board connector 13. The output terminal of the power switching circuit 72 is connected to the power switch 73 to output a 3.3V regulated power supply. The power switch 73 is connected to the input terminal of the serial port 95.

[0176] In this embodiment, the charging circuit 71 includes a charging management chip and a light-emitting diode (LED). The charging management chip is model LGS4056HDA, and the LED is model KT-0603R. Pin 4 of the charging management chip is connected to pin 11 of the wire-to-board connector 13. Pin 4 of the charging management chip is grounded through the 23rd resistor R23 and the first capacitor C1. Pin 2 of the charging management chip is connected to the 20th resistor R20 and then to pins 3 and 9 respectively, and then grounded. Pin 7 of the charging management chip is connected to the output terminal of the LED. The input terminal of the LED is connected to the input terminal of the power switching circuit 72 through the third resistor R3. Pin 5 of the charging management chip is connected to the wire-to-board connector 13.

[0177] The power switching circuit 72 includes a Schottky diode D2, a first MOSFET Q1, a second MOSFET Q2, and a third MOSFET Q3. The gate g of the third MOSFET Q3 is grounded through the twenty-first resistor R21. The drain d of the third MOSFET Q3 is connected to pin 5 of the charging management chip. The source s of the third MOSFET Q3 is connected to the power switch. The gate g of the second MOSFET Q2 is connected to the source s of the first MOSFET Q1. The drain d of the second MOSFET Q2 is connected to the input terminal of the Schottky diode D2. The output terminal of the Schottky diode D1 is connected to the third resistor R3 and pin 8 of the charging management chip. The source s of the second MOSFET Q2 is connected to the power switch 73. The drain d of the first MOSFET Q1 is grounded. The gate g of the first MOSFET Q1 is connected to the twenty-first resistor R21. The source s of the first MOSFET Q1 is connected to the power switch 73 through the twenty-second resistor R22.

[0178] The power switch 73 includes a switching switch and a voltage regulator. The switching switch is a MINI MSK12CO2, and the voltage regulator is a ME6217C33M5G. The input terminal of the switching switch is connected to the external power supply, and the output terminal of the switching switch is connected to the output terminal of the power switching circuit 72. Pins 1 and 3 of the voltage regulator are connected to the output terminal of the switching switch. Pins 1 and 3 of the voltage regulator are grounded through the thirteenth capacitor C13. Pin 5 of the voltage regulator outputs a 3.3V regulated power supply, and pin 5 of the voltage regulator is grounded through the eleventh capacitor C11.

[0179] Example 12:

[0180] The basic content is the same as Example 8, except that:

[0181] See Figure 13 Pins 21, 22, 23, and 24 of the main control chip 91 are connected to the storage module 10 via the SPI bus. The storage module 10 is used to store the program code, configuration parameters, acquired audio files, and generated response audio files required for the main control chip 91 to run.

[0182] In this embodiment, the storage module 10 includes a memory chip, model W25Q128JVSIQ. Pins 1, 2, 5, and 6 of the memory chip are connected to pins 21, 22, 23, and 24 of the main control chip via an SPI bus. Pin 3 of the memory chip is connected to the power module 7 via the fourteenth resistor R14. Pin 7 of the memory chip is connected to the power module 7 and one end of the thirtieth capacitor C30 via the fifteenth resistor R15. The other end of the thirtieth capacitor C30 is grounded. Pin 8 of the memory chip is connected to one end of the thirtieth capacitor C30. Pin 4 of the memory chip is grounded.

[0183] The memory chip is mainly used to store the program code required for the main control chip to run. When the main control chip starts up, it reads and loads the program from the memory chip into the internal memory and then begins execution; it stores the configuration parameters of the storage device. This configuration data will not be lost after the device is powered off. When the power is restored, the main control chip can read the configuration information from the memory chip and run according to the set parameters; it also stores user-related data and custom settings; and it is used to store file and directory information in the file system, which facilitates the organization and management of data, similar to how a computer hard drive stores files.

[0184] Example 13:

[0185] The basic content is the same as Example 8, except that:

[0186] See Figure 3 , Figure 14 and Figure 15 The audio input module 2 includes a microphone 21 and an electret microphone 22. The microphone 21 is connected to the input terminal of the audio codec module 3 through a second capacitor C2 and a third capacitor C3. The electret microphone 22 is connected to the input terminal of the wake-up module 4.

[0187] In this embodiment, the microphone 21 is a ZTS6216. The VDD terminal of the microphone 21 is connected to pin 15 of the audio codec chip. The wake-up module 4 uses an ASRPRO chip, model TW-ASR-PRO. Pin 20 of the ASRPRO chip is connected to the MIC terminal of the electret microphone 22 via the fifteenth capacitor C15. Pin 21 is connected to the MICL terminal of the electret microphone 22 via the twelfth capacitor C12. Pin 22 is connected to the thirteenth resistor R13 and then in parallel with the MICL terminal of the electret microphone 22. Pin 22 is connected to the fourteenth resistor R14 and then in parallel with the MIC terminal of the electret microphone 22. Pin 22 is connected to the seventeenth resistor R17 and then connected to analog ground.

[0188] Before wake-up, electret microphone 22 is in working state (low-power monitoring), continuously collecting ambient audio. The wake-up module 4 detects the wake-up word. At this time, microphone 21 and audio codec module 3 are in sleep / off state to save power. When wake-up module 4 detects the wake-up word, wake-up module 4 triggers the main control chip 91 to wake up, electret microphone 22 stops working (or only assists in noise reduction), microphone 21 and audio codec module 3 start up, and are responsible for subsequent high-fidelity voice acquisition. This ensures the function of always monitoring the wake-up word, while avoiding the high power consumption caused by continuous operation of microphone 21.

[0189] Example 14:

[0190] The basic content is the same as Example 8, except that:

[0191] See Figure 3 and Figure 16 The wireless module 8 includes a radio frequency antenna 81. Pin 1 of the main control chip 91 is connected to the radio frequency antenna 81 via the thirty-second capacitor C32 and the thirteenth resistor R13. The radio frequency antenna 81 is connected to pin 1 of the main control chip 91 via the thirty-first capacitor C31. One end of the thirty-first capacitor C31 is grounded. Pin 6 of the main control chip 91 is connected to an LED light 11 via the twenty-fourth resistor R24.

[0192] In this embodiment, the radio frequency antenna is model RFANT3216120A5T, the LED light is model WS2812 chip, and pin 6 of the main control chip 91 is connected to the DIN terminal of the WS2812 chip through the 24th resistor R24. The WS2812 chip is mainly used to receive user voice and replies as a prompt.

[0193] Example 15:

[0194] The basic content is the same as Example 7, except that:

[0195] See Figure 17 The BOOT button 12 is connected to pin 15 of the main control chip 91.

[0196] In this embodiment, the BOOT button 12 is a TS24CA tactile switch. Pin 15 of the main control chip 91 is connected to the BOOT button 12. When the main control chip 91 is powered off, press and hold the BOOT button, then power on the main control chip 91 or press the reset button, keeping the BOOT button 12 pressed for a period of time, usually a few seconds (refer to the main control chip 91's datasheet for the specific time). Then release the BOOT button 12. At this time, the main control chip 91 will enter the firmware download mode. When it is necessary to update the program (firmware) inside the controller, the main control chip 91 needs to be in a specific startup state so that the computer can burn the new program code into it through serial port 95 or other communication methods. While resetting the main control chip 91 (powering on again or manual reset), press and hold the BOOT button 12. The main control chip 91 will recognize the specific level state and enter the download mode, waiting to receive new firmware data.

[0197] Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An intelligent voice interaction method, characterized in that: The interaction method includes the following steps: The original audio signal is acquired and converted into a digital audio signal through the audio input module (2); The wake-up module (4) performs feature extraction and classification on the digital audio signal; The main control module (9) performs wake word detection on the extracted features according to the wake word detection model; if no wake word is detected, the intelligent voice interaction device remains in low power standby mode; if a wake word is detected, the main control module (9) outputs a wake-up signal and stops the wake word detection model. The audio codec module (3) collects the original audio signal from the audio input module (2) in real time according to the wake-up signal, and converts the original audio signal into a digital audio signal. The main control module (9) sends the digital audio signal to the cloud server (1) to generate reply text and digital audio data. The main control module (9) sends the reply text to the interaction module (6) for display. The audio codec module (3) converts the digital audio data into an analog audio signal and sends it to the audio amplifier module (5). The audio amplifier module (5) converts the analog audio signal into sound to complete the audio playback. The main control module (9) re-enables the wake word detection model to detect wake words, repeats the above steps, and realizes continuous voice dialogue.

2. The intelligent voice interaction method according to claim 1, characterized in that: Before performing feature extraction and classification on the digital audio signal, the digital audio signal is preprocessed and the continuous digital audio signal is divided into frame-by-frame audio signals. Then, a window function is applied to each frame of audio signal for windowing.

3. The intelligent voice interaction method according to claim 2, characterized in that: The preprocessing of the digital audio signal includes: Analyze the audio characteristics of digital audio signals without speech segments, estimate the spectral characteristics of background noise, design an optimal filter by calculating the power spectrum ratio of speech to background noise, and remove background noise using the optimal filter. The digital audio signal is delayed and filtered to match the transmission path of the echo received by the audio input module (2), and the filter coefficients are updated to make the filter output close to the echo signal, so that the digital audio signal is reduced by the predicted echo. High-pass filters are used to enhance the energy of the high-frequency components in digital audio signals, thereby increasing the high-frequency resolution of the digital audio signals.

4. The intelligent voice interaction method according to claim 2, characterized in that: The feature extraction and classification of digital audio signals includes: Perform a short-time Fourier transform on each frame of the windowed audio signal to convert the time-domain signal to the frequency domain and obtain the spectrum of each frame; Based on the Mel frequency scale, a set of filters is uniformly distributed in the Mel frequency domain, and then these filters in the Mel frequency domain are converted back to the linear frequency domain to obtain the corresponding Mel filter bank. The spectrum of each frame of audio signal is passed through a Mel filter bank, the output value of each filter is calculated, and the logarithm of each output value of the Mel filter bank is taken to obtain the logarithmic Mel energy. The log-Mel energy is subjected to discrete cosine transform to convert it from the frequency domain to the cepstral domain, and the main spectral features are extracted.

5. The intelligent voice interaction method according to claim 1, characterized in that: The step of detecting wake words from the extracted features includes: The extracted spectral features are normalized and then input into the input layer of the neural network. The convolutional layers of a neural network perform convolution operations on the input features to extract local features; Apply an activation function to the output of the convolutional layer; The feature vectors are flattened and then input into a fully connected layer for global information integration. Use a function in the output layer to convert the output of the fully connected layer into a probability distribution; The classification decision is made based on the set probability threshold. If the probability of the wake word category is higher than the probability threshold, it is determined to be a wake word; otherwise, it is determined to be a non-wake word.

6. The intelligent voice interaction method according to claim 1, characterized in that: The main control module (9) sends the digital audio signal to the cloud server to generate reply text and digital audio data, including: The main control module (9) starts recording and continuously reads digital audio signals until it detects multiple consecutive frames of silence or reaches the maximum recording time. Then it ends the recording and sends the recorded audio data to the cloud server (1). The cloud server (1) converts the audio data into text, then inputs the text into the AI ​​model to generate reply text and digital audio data, and sends them to the main control module (9).

7. The intelligent voice interaction method according to claim 6, characterized in that: The continuous reading of digital audio signals until multiple consecutive frames of silence are detected or the maximum recording duration is reached includes: Each frame of the digital audio signal is detected sequentially. When no data is detected in the current frame, the digital audio signal of the next frame is detected until data is detected and recording begins. Each frame of the digital audio signal is sampled and counted simultaneously. The system detects whether the sample value of each frame of digital audio signal is lower than a certain threshold. If it is lower than the threshold, it is considered as silent and is counted. When silence is detected for multiple consecutive frames and the frame count reaches the set number of silent frames, the recording ends. Recording ends when the number of frames in the digital audio signal reaches the set maximum recording duration.

8. The intelligent voice interaction method according to claim 7, characterized in that: The process of inputting text into an AI model to generate response text and digital audio data includes: The text is fed into the AI ​​model via API. The AI ​​model converts the input text into a token sequence and maps it into a high-dimensional vector. Then, it captures the long-range dependencies and semantic associations in the input sequence. Next, it generates the output sequence step by step through autoregressive decoding. At each step, it selects the next token with the highest probability based on the current context and the tokens generated in the past. The generated token sequence is decoded into the response text of natural language text and calls the TTS service to combine sentiment parameters to synthesize human-like digital audio data.

9. An intelligent voice interaction device, characterized in that, The intelligent voice interaction device is applied to the intelligent voice interaction method according to claim 1, comprising: a main control module (9), an audio input module (2), an audio codec module (3), a wake-up module (4), and an audio power amplifier module (5). The output end of the audio input module (2) is connected to the input ends of the audio codec module (3) and the wake-up module (4), respectively. The output end of the audio codec module (3) is connected to the input end of the audio power amplifier module (5). The output end of the wake-up module (4) is connected to the main control module (9). The main control module (9) is connected to the audio codec module (3) and the audio power amplifier module (5). The main control module (9) is connected to the cloud server (1) through a wireless module (8). The audio input module (2) is used to collect audio signals; The audio encoding / decoding module (3) is used to convert the acquired audio signal into a digital signal and send the digital signal to the main control module (9) for processing; convert the digital audio signal processed by the main control module (9) into an analog signal and send the analog signal to the audio power amplifier module (5); The wake-up module (4) is used to detect the wake-up word in the audio signal and send a signal to the main control module (9); The main control module (9) is used to convert digital signals into text and transmit the text to the cloud server (1) to generate reply text, and convert the reply text into digital audio signals; according to the signal sent by the wake-up module (4), it switches from standby state to working state to start interaction, and switches from working state to standby state after the interaction ends.

10. The intelligent voice interaction device according to claim 9, characterized in that: The audio input module (2) includes a microphone (21) and an electret microphone (22). The microphone (21) is connected to the input terminal of the audio codec module (3), and the electret microphone (22) is connected to the input terminal of the wake-up module (4).

Images