A cascaded voice interaction method for moxibustion equipment

Abstract

This invention relates to the field of voice interaction in smart devices, and discloses a cascaded voice interaction method for moxibustion devices. The method collects user voice, preprocesses it to obtain effective voice frames and acoustic feature maps; inputs the feature maps into a first-level voice processing unit, runs a low-power wake-word detection and short command recognition model, and directly outputs control commands when short commands are recognized; if no short command is recognized, it calculates a complex intent tendency score, and if the score exceeds a threshold, it wakes up a second-level voice processing unit; the second-level unit runs a locally deployed lightweight large language model to perform end-to-end voice recognition and semantic understanding, and generates interactive commands based on a moxibustion knowledge base; finally, the commands are executed and the results are broadcast. This invention, through a two-level cascaded architecture, balances low power consumption and high intelligence, achieves offline complex semantic understanding, and improves the voice interaction experience in moxibustion scenarios.

Description

Technical Field

[0001] This invention relates to the field of voice interaction in smart devices, and more particularly to a cascaded voice interaction method for moxibustion devices. Background Technology

[0002] Voice interaction technology, as an important means of human-computer interaction, enables natural language communication between users and devices through core modules such as automatic speech recognition (ASR), natural language understanding (NLU), and text-to-speech (TTS). With the development of deep learning, voice interaction systems are gradually evolving from traditional isolated word recognition to end-to-end large language model (LLM) dialogue systems.

[0003] In the fields of smart homes and healthcare, voice interaction is widely used due to its advantages of freeing up hands and convenient operation. Moxibustion, as a traditional Chinese medicine therapy that requires users to maintain a fixed posture, often makes it difficult for users to free their hands to operate the device, making voice interaction an ideal control method. However, the usage scenarios of moxibustion devices are unique: users may be in bed or semi-reclining positions, and there are complex noises around the device, such as the sound of burning moxa sticks and ambient noise. Furthermore, the interaction needs encompass two different types of tasks: real-time control (such as temperature adjustment and on / off switching) and in-depth consultation (such as acupoint lookup and symptom consultation). This places higher demands on the power consumption, real-time performance, intelligence, and environmental adaptability of voice interaction systems.

[0004] Currently, the main voice interaction technologies for moxibustion devices fall into the following categories: The first type is a single offline command solution, which uses a low-power voice recognition chip with a limited number of pre-set fixed commands. Users need to control the device according to preset words (such as "on", "off", "high", "low"). This solution has low power consumption and fast response, but it can only recognize a limited number of commands, cannot understand natural language, and cannot handle consultation-type questions such as "where should I apply moxibustion for a sore shoulder?", resulting in significant functional limitations.

[0005] The second type is the pure cloud-based large-model solution, which uploads the user's voice to a cloud server, where a large cloud model (such as the GPT series) performs speech recognition and semantic understanding and returns the results. This solution is highly intelligent and can handle complex interactions, but it relies on a network connection and cannot be used in environments without a network or with a weak network. In addition, the cloud processing latency is large (usually 1-3 seconds) and the power consumption is high (requiring continuous network communication), making it unsuitable for mobile use scenarios of moxibustion devices.

[0006] The third type is the general smart speaker solution, which integrates the moxibustion device into the smart speaker ecosystem and enables voice interaction through the speaker. While this solution can achieve a certain level of intelligence, it is not optimized for the moxibustion scenario, cannot recognize professional terms such as "Zusanli" and "warm moxibustion," and the speaker and device are separate, resulting in a cumbersome interaction process and a poor user experience.

[0007] The fourth type is the single-level wake-up solution, which uses a single wake-up word to send all voice messages to the same processing unit. However, real-time control commands require low-latency responses, and consultation commands require deep understanding. A single processing unit cannot balance power consumption and performance, resulting in either slow control response or weak consultation capabilities. Summary of the Invention

[0008] The purpose of this invention is to construct a cascaded voice interaction architecture for moxibustion devices, achieving a balance between low power consumption and high intelligence. Specifically, it addresses the following key technical issues: First, how to design a two-level heterogeneous processing architecture, enabling the first-level unit to respond to control commands in real time with extremely low power consumption, while the second-level unit initiates processing complex inquiries on demand, thus balancing real-time performance, intelligence, and low power consumption; Second, how to accurately determine whether a user's voice is a simple control or a complex inquiry, avoiding unnecessary wake-up of the second-level unit and saving power; Third, how to deploy a lightweight large language model on embedded devices to achieve complex semantic understanding in offline environments, eliminating network dependence; Fourth, how to optimize for the acoustic characteristics (moxa burning sound, ambient human voices) and environmental noise of moxibustion scenarios, improving wake-up rate and recognition rate; Fifth, how to establish a multi-level safety redundancy mechanism to ensure that the device remains controllable when voice interaction fails, guaranteeing safe use; Sixth, how to achieve personalized adaptation, allowing the interaction strategy to be dynamically optimized according to user habits.

[0009] Specifically, the present invention provides a cascaded voice interaction method for moxibustion devices, the method comprising the following steps: S1. Acquire the user's voice signal in real time through a microphone array, preprocess the voice signal, detect voice activity and extract acoustic features to obtain an effective voice frame sequence and its corresponding acoustic feature map; S2. The acoustic feature map is input to the first-level speech processing unit. The first-level speech processing unit runs a low-power wake word detection model and a short command recognition model in the field of moxibustion. When a wake word is detected, a wake signal is output and the command recognition is enabled. When a short command is recognized, the corresponding control command is directly output to the main control module for execution. S3. When the first-level speech processing unit fails to recognize the short command, it calculates the complex intent tendency score of the current speech. When the score exceeds the preset wake-up threshold, it is determined to be a complex interaction intent and a wake-up request is sent to the second-level speech processing unit. S4. In response to the wake-up request, start the secondary speech processing unit and transmit the effective speech frame sequence and its corresponding acoustic feature map to the secondary speech processing unit. S5. The secondary speech processing unit runs a locally deployed lightweight large language model, takes the acoustic feature map and effective speech frame sequence as input, performs integrated recognition and understanding through an end-to-end speech-semantic joint model, and generates interactive instructions containing intent categories and slot parameters by combining the knowledge base of moxibustion domain. S6. The interaction command is sent to the main control module for execution, and the execution result is broadcast to the user through the speech synthesis module. At the same time, the context information of this interaction is stored in the dialogue state cache for semantic disambiguation in subsequent multi-round dialogues.

[0010] A storage medium storing instructions and data for implementing a cascaded voice interaction method for moxibustion devices.

[0011] A cascaded voice interaction device for moxibustion devices includes: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a cascaded voice interaction method for moxibustion devices.

[0012] The beneficial effects provided by this invention are: First, through a two-level cascaded architecture, the first-level unit consumes less than 10mW and can standby and monitor 24 / 7, while the second-level unit starts on demand. Overall power consumption is reduced by more than 80% compared to a pure cloud solution, significantly extending the battery life of the moxibustion device. Second, through a complex intent-based scoring mechanism, it accurately identifies the user's consultation intent, achieving a 91% accuracy rate for second-level wake-up and avoiding unnecessary computational consumption. Third, by deploying a lightweight large language model locally (with less than 80 million parameters), it achieves natural language understanding in offline environments, with an end-to-end response latency of less than 300ms, eliminating network dependence and ensuring users can still use the device even without a network connection. Fourth, through specialized model training and knowledge base construction in the field of moxibustion, the accuracy rate of recognizing professional terms such as "Zusanli," "Mingmen," and "warm moxibustion" reaches 92%, and the user satisfaction rate of consultation-type interactions is increased to 95%. Fifth, through multi-level security redundancy and conflict arbitration mechanisms, emergency commands are given priority response, and backup interaction channels are automatically switched when voice fails, completely eliminating safety hazards. Sixth, through environmental adaptive optimization and personalized learning, the wake-up rate is increased from 92% to 98.5% in the environment of moxibustion combustion noise, and the interaction strategy is dynamically adjusted according to the user's usage habits, achieving a user experience that becomes smarter the more it is used. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the hardware device operation according to an embodiment of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0015] Before formally describing the present invention, a general description of the solution of the present invention will be given first to facilitate understanding.

[0016] To facilitate understanding of the specific implementation of the method of this invention, the hardware system upon which the method depends is first described. The method of this invention is implemented based on the following hardware system architecture: The moxibustion device has the following built-in functional modules: (1) Main control module: The ESP32-S3 microcontroller is used, which integrates AI acceleration instruction set and Wi-Fi / Bluetooth communication functions. It is responsible for the coordination and control of each module, data transfer and instruction arbitration.

[0017] (2) Microphone array: Dual digital microphones (such as MSM261D3526H0CM) are arranged in a linear array on the front of the moxibustion box with a spacing of 4cm. They support beamforming and noise suppression and are used to collect user voice signals.

[0018] (3) Primary speech processing unit: It adopts an ultra-low power speech coprocessor (such as CI1122), which is dedicated to wake word detection and short command recognition. The standby power consumption is less than 10mW and supports deep separable convolutional neural network inference.

[0019] (4) Secondary speech processing unit: It adopts a high-performance embedded processor (such as Rockchip RK3588S), integrates NPU computing power of 6TOPS, is used to deploy lightweight large language models and end-to-end speech recognition models, supports INT4 quantization inference, and has a power consumption of about 1.5W.

[0020] (5) Speech synthesis module: The XFS5152CE speech synthesis chip is used, which supports Chinese and English broadcasting and multiple speakers. It receives text through the UART interface and outputs speech signals to the speaker.

[0021] (6) Display and interaction module: including a 0.96-inch OLED display and touch buttons, serving as a backup channel for voice interaction, used to display status information and receive touch input.

[0022] (7) Power management module: supports dynamic voltage frequency adjustment (DVFS), can automatically switch power supply mode according to load, and cut off power supply to secondary unit when it is in sleep mode to save power consumption.

[0023] The connection relationships of the above modules are as follows: the microphone array is connected to the main control module via the PDM interface; the main control module distributes audio data to the first-level speech processing unit (via the I2S interface) and the second-level speech processing unit (via shared memory); the first-level and second-level speech processing units communicate with the main control module via the SPI interface to transmit recognition results and interactive commands; the speech synthesis module receives the broadcast text issued by the main control module via the UART interface; the display and interaction module is connected to the main control module via the I2C interface; and the power management module controls the power switches of each unit via GPIO. The main control module's built-in memory pre-stores a wake-up word model, a short command recognition model, a lightweight large language model, and a knowledge base for the moxibustion domain.

[0024] Taking a typical moxibustion process as an example: The user lies face down on the bed, receiving moxibustion on their back, and says, "Xiao Ai, Xiao Ai, turn the temperature up a bit." The microphone array collects the voice signal, which is then distributed by the main control module to the first-level voice processing unit. After being woken up, the unit recognizes the voice signal as a control command and directly returns it to the main control module to adjust the temperature. When the user says, "Xiao Ai, Xiao Ai, my shoulders feel very sore, which acupoint should I apply moxibustion to?", the first-level unit recognizes the complex interaction intent, wakes up the second-level voice processing unit, and performs voice recognition, semantic understanding, and knowledge base retrieval. After generating a treatment plan, it is broadcast to the user through the voice synthesis module. The entire process runs in real time on the embedded platform. The first-level unit's standby power consumption is <10mW, and the second-level unit starts on demand, balancing low power consumption and high intelligence.

[0025] The above is merely an exemplary description of the hardware architecture upon which the method of this invention depends. Specific selections can be made according to actual circumstances. This invention is not intended to be limited. The focus of this invention is the method described below.

[0026] Example 1 Please refer to Figure 1 The present invention provides a cascaded voice interaction method for moxibustion devices, comprising the following steps: S1. Acquire the user's voice signal in real time through a microphone array, preprocess the voice signal, detect voice activity and extract acoustic features to obtain an effective voice frame sequence and its corresponding acoustic feature map; It should be noted that the sub-steps of preprocessing the speech signal, detecting speech activity, and extracting acoustic features in step S1 include: S11. A dual-microphone array is used to collect voice signals. Adaptive beamforming is performed through a generalized sidelobe canceller algorithm to enhance the user's sound source direction signal and suppress the noise of burning moxa sticks and environmental human voice interference in the moxibustion environment. S12. Pre-emphasize the beamformed signal to compensate for the high-frequency attenuation of the voice signal; S13. Divide the pre-emphasized signal into frames according to the preset frame length and frame shift, and apply a Hamming window for windowing processing. S14. Calculate the short-time energy and zero-crossing rate for each frame of signal, use the dual-threshold method to detect speech activity, remove silent frames and pure noise frames, and obtain an effective speech frame sequence. S15. Extract Mel frequency cepstral coefficient features from the effective speech frame sequence to generate an acoustic feature map.

[0027] As one embodiment, the sub-steps of acquiring the user's voice signal in real time through a microphone array and performing preprocessing, voice activity detection, and acoustic feature extraction on the voice signal in step S1 are specifically implemented as follows: S11. Adaptive Beamforming: Employs a dual-microphone array with a 4cm spacing. The Generalized Sidelobe Canceller (GSC) algorithm is implemented as follows: A fixed beamformer uses delay-summing beamforming, with the user direction at 0° (front of the moxibustion box), and the delay corresponding to the microphone spacing... τ = d · sinθ / c , c =340m / s, d =0.04m, θ =0° τ =0, the two signals are directly added. The blocking matrix uses a differential method, subtracting the two signals to obtain the noise reference signal. The adaptive interference canceller uses the Normalized Least Mean Square (NLMS) algorithm, with a step size of... μ =0.01, filter order 64, coefficients updated in real time, subtracting noise-related components from the enhanced signal. Actual measurements show that beamforming improves the signal-to-noise ratio by 12-18 dB in a moxibustion combustion noise environment.

[0028] S12, Pre-emphasis: Employs a first-order FIR high-pass filter, transfer function H ( z )=1-0.97 z -1 This enables high-frequency enhancement.

[0029] S13, Frame Windowing: Frame length 25ms (400 points), frame shift 10ms (160 points), Hamming window coefficients are pre-calculated and stored in a lookup table.

[0030] S14. Voice activity detection: Employs a dual-threshold method, with a high energy threshold. E H Set to 3 times the average energy of the background noise, low threshold E L Set to 1.5 times the average energy of the background noise, with a high zero-crossing rate threshold. Z HSet to 25 times / frame. Background noise energy is obtained by statistical analysis of the first 10 silent frames. When the energy exceeds [a certain threshold] for 3 consecutive frames... E H Or (energy exceeding) E L And the zero-crossing rate is lower than Z H When the energy is below a certain threshold, it is determined as the start of speech; when the energy is below a certain threshold for 5 consecutive frames... E L And the zero-crossing rate is lower than Z H When the audio segment is detected, it is considered to be the end of the audio segment. After detecting the audio segment, the two frames before the start point and the two frames after the end point are retained as a buffer to avoid truncation.

[0031] S15. Acoustic Feature Extraction: A 512-point FFT is performed on each speech frame, and a Hamming window is used to reduce spectral leakage. After calculating the power spectrum, the signal is passed through 24 triangular Mel filters (frequency range 80-4000Hz), and after taking the logarithm, a DCT is performed to obtain a 13-dimensional static MFCC. The first-order difference Δ and the second-order difference ΔΔ are then calculated, finally generating a 39-dimensional feature vector. The feature vectors of all valid speech frames are stored in chronological order as acoustic feature maps.

[0032] For example, when a user says "Xiao Ai, Xiao Ai, it's too hot," VAD detects that the speech starts at frame 20 and ends at frame 85, for a total of 66 valid speech frames, generating a 66×39 acoustic feature map.

[0033] S2. The acoustic feature map is input to the first-level speech processing unit. The first-level speech processing unit runs a low-power wake word detection model and a short command recognition model in the field of moxibustion. When a wake word is detected, a wake signal is output and the command recognition is enabled. When a short command is recognized, the corresponding control command is directly output to the main control module for execution. It should be noted that the short command recognition model in the moxibustion field in step S2 is a deep separable convolutional neural network, and its output layer classification categories include at least: increase temperature, decrease temperature, turn on essential oil, turn off essential oil, query remaining time, and emergency stop; the training data of the model includes noisy speech samples collected in the moxibustion scenario to improve the recognition robustness in real environment.

[0034] As one embodiment, the short command recognition model for moxibustion in step S2 is a deep separable convolutional neural network (DS-CNN), with the following specific structure: Input layer: 40×39 (40 frames of MFCC features) Layer 1: Conv2D, 32 3×3 convolutional kernels, stride 1, padding=same, activation function ReLU. Layer 2: Depthwise separable convolution, 64 3×3 depthwise convolution kernels, stride 1, followed by pointwise convolution of 1×1×32×64, with ReLU activation function. Layer 3: Depthwise separable convolution, 64 3×3 depthwise convolution kernels, stride 2, pointwise convolution 1×1×64×64, ReLU activation function. Layer 4: Depthwise separable convolution, 128 3×3 depthwise convolution kernels, stride 1, pointwise convolution 1×1×64×128, ReLU activation function. Layer 5: Depthwise separable convolution, 128 3×3 depthwise convolution kernels, stride 2, pointwise convolution 1×1×128×128, ReLU activation function. Layer 6: Global Average Pooling Layer 7: Fully connected layer, 128 nodes, dropout=0.3 Output layer: Fully connected layer, 7 nodes (6 types of instructions + 1 type of non-instruction), softmax activation. The model has a total of 85K parameters, and a single inference time of about 15ms on the CI1122 with a power consumption of 8mW.

[0035] The training data includes 5000 hours of audio samples from moxibustion scenarios, of which 3000 hours were collected from real-world environments (including moxibustion burning sounds, fan sounds, and ambient human voices), and 2000 hours were synthetic data (enhanced by adding noise with different signal-to-noise ratios). The sample size is balanced across each instruction category, approximately 5000 samples per category. Training uses the cross-entropy loss function and the Adam optimizer, with an initial learning rate of 0.001. After 50 training epochs, the accuracy reached 96.3%.

[0036] For example, in a 60dB moxibustion burning noise environment, when a user says "turn on the essential oil", the model outputs a probability of 0.91 for "turn on the essential oil", a probability of 0.03 for "turn off the essential oil", a probability of 0.02 for "non-command", and all others are <0.01, which is a correct recognition.

[0037] As one embodiment, step S2 of the present invention is specifically implemented as follows: The first-level speech processing unit adopts the CI1122 ultra-low power speech coprocessor, which internally embeds two types of neural network models—wake-up word detection model and short instruction recognition model, both of which are Deep Separable Convolutional Neural Network (DS-CNN) architecture.

[0038] The wake-up word detection model takes a single frame of 39-dimensional MFCC features (contextual data from 10 consecutive frames) as input and outputs the confidence score (0-1) of the wake-up word "Xiao Ai Xiao Ai". The model structure includes: an input layer (10×39), two depthwise separable convolutional layers (3×3 kernels, 1 stride, 64 channels), a global average pooling layer, a fully connected layer (128 nodes), and a softmax output layer (two classes: wake-up word / non-wake-up word). The model has only 28K parameters and consumes approximately 3mW of power.

[0039] The short command recognition model takes 40 consecutive frames of acoustic feature maps (40×39) as input and outputs a probability distribution of preset command categories. The command set includes 6 categories: "Increase Temperature", "Decrease Temperature", "Turn On Essential Oil", "Turn Off Essential Oil", "Remaining Time", and "Emergency Stop", plus a "Non-Command" category, for a total of 7 output categories. The model structure is similar, with 85K parameters and an operating power consumption of approximately 8mW.

[0040] The system defaults to a low-power monitoring state, running only the wake-word detection model. When the wake-word confidence exceeds the first threshold of 0.85, a wake-up signal is output, the main control module illuminates the "voice wake-up" indicator, and the short command recognition model is enabled. Within the next 3 seconds, the user's voice is fed into the short command recognition model. If the probability of a certain command category exceeds the second threshold of 0.7, the corresponding control command is directly output to the main control module for execution.

[0041] For example, when Ms. Zhang says, "Xiao Ai, Xiao Ai, turn the temperature up a bit," the wake-up word detection model outputs a confidence level of 0.92, triggering wake-up. Subsequently, the "turn the temperature up a bit" speech is processed by the short command recognition model, which assigns a probability of 0.88 to the "turn the temperature up" category, directly outputting the "turn the temperature up" command to the main control module. The main control module executes the PID temperature setpoint increase of 2°C and announces "temperature has been turned up" through the speech synthesis module.

[0042] S3. When the first-level speech processing unit fails to recognize the short command, it calculates the complex intent tendency score of the current speech. When the score exceeds the preset wake-up threshold, it is determined to be a complex interaction intent and a wake-up request is sent to the second-level speech processing unit. It should be noted that the specific steps for calculating the complex intent tendency score of the current speech in step S3 include: S31. Extract speech duration, number of pauses within a sentence, average speech rate, and speech emotion feature parameters from the effective speech frame sequence. The emotion feature parameters include the fundamental frequency mean, energy jitter, and speech rate change rate. S32. Input the acoustic feature map into a lightweight intention tendency prediction network, and at the same time use the emotion feature parameters as auxiliary input. The network outputs the probability value that the current voice belongs to the consultation type interaction. S33. The speech duration, number of pauses within a sentence, average speech rate and probability value are weighted and fused together to obtain a complex intent tendency score; S34. When the score exceeds the first threshold, it is determined to be a complex interaction intention; when the score exceeds the higher second threshold, it is marked as a complex interaction during the voice acquisition stage, and the effective voice frame sequence and its acoustic feature map are directly transmitted to the secondary processing unit to shorten the response time.

[0043] As one embodiment, the specific implementation of calculating the complex intent tendency score of the current speech in step S3 is as follows: S31. Feature Extraction: Extracting speech duration from effective speech frame sequences. T (seconds), calculated from the start and end times of the speech segment detected by VAD; number of pauses within a sentence. P By detecting that the energy within a speech segment is lower than E L Counting intervals (>150ms); average speech rate S The emotional feature parameters are estimated by dividing the number of words in the speech-recognition text by the duration; the emotional feature parameters include the fundamental frequency mean. F 0 mean (The fundamental frequency is extracted every 10ms using the autocorrelation method and then averaged), energy fluctuation. Jitter (mean squared error of the logarithm of energy in adjacent frames), speech rate change rate (the speech segment is divided into 5 equal segments, and the variance of the speech rate of each segment is calculated).

[0044] S32, Intent Tendency Prediction Network: Input is acoustic feature map ( T ×39), the output is the consultation probability. P intent Network structure: Conv2D(32,3×3) → MaxPool(2×2) → Conv2D(64,3×3) → MaxPool(2×2) → Flatten → Dense(64) → Dense(1, sigmoid). The network has approximately 120,000 parameters, and inference time on edge devices is <5ms. The training data includes 1000 hours of audio recordings of moxibustion scenarios, labeled to indicate whether the interaction is a consultation.

[0045] S33, Weighted Fusion: The scoring formula is as follows Score = 0.1· T + 0.15· P + 0.1·1 / (| S -3|+0.1) +0.4· P intent + 0.15·( F 0 mean / 200) + 0.1· JitterThe weights were optimized using a genetic algorithm, achieving an accuracy of 91% on the validation set.

[0046] S34, Threshold Judgment: When Score A second-level wake-up is triggered when the value is >0.6; when Score When the confidence level is greater than 0.85, the system marks it as a "high-confidence complex interaction" during the voice acquisition phase and directly transmits the data to the secondary unit. For example, if a user says, "I haven't been sleeping well lately, will moxibustion help?" T =2.8, P =1, S =3.2, P intent =0.95, F 0 mean =210Hz, Jitter =0.12, calculated as follows: Score =0.28+0.15+0.11+0.38+0.158+0.012=1.09>0.85 Trigger high-confidence pass-through.

[0047] S4. In response to the wake-up request, start the secondary speech processing unit and transmit the effective speech frame sequence and its corresponding acoustic feature map to the secondary speech processing unit. As one embodiment, step S4 of the present invention is specifically implemented as follows: After receiving the secondary wake-up request, the main control module starts the power supply and clock of the secondary voice processing unit (Rockchip RK3588S) through the power management unit. The processor is in deep sleep mode by default (power consumption <10mW), and it takes about 50ms to start up to the working state (power consumption 1.5W).

[0048] Simultaneously, the primary speech processing unit transmits the cached current valid speech frame sequence (raw PCM data) and its corresponding acoustic feature map to the shared memory of the secondary speech processing unit via the SPI interface using DMA. After the transmission is complete, the primary speech processing unit returns to low-power monitoring mode and continues to monitor for wake words.

[0049] To reduce redundant calculations, the acoustic feature maps transmitted by the first-level unit can be directly reused by the speech recognition model of the second-level unit, avoiding secondary feature extraction and saving approximately 30ms of processing time.

[0050] For example, in the "shoulders are particularly sore" example above, the first-level unit transmits 3.2 seconds of PCM data (51,200 bytes) and a 120×39 acoustic feature map (18,720 floating-point numbers) to the second-level unit, with the transmission taking approximately 5ms. The second-level unit immediately obtains this data upon startup and enters the speech understanding process.

[0051] S5. The secondary speech processing unit runs a locally deployed lightweight large language model, takes the acoustic feature map and effective speech frame sequence as input, performs integrated recognition and understanding through an end-to-end speech-semantic joint model, and generates interactive instructions containing intent categories and slot parameters by combining the knowledge base of moxibustion domain. It should be noted that the sub-step in step S5, where the secondary speech processing unit runs a locally deployed lightweight large language model to generate interactive instructions containing intent categories and slot parameters, includes: S51. Input the received valid speech frame sequence into the end-to-end speech recognition model, which is a convolutional enhanced Transformer architecture based on connection temporal classification, and directly outputs the text sequence. S52. The text sequence is concatenated with the preceding information read from the dialogue state buffer to form input text with context. S53. Input the contextualized input text into the lightweight large language model and load the predefined moxibustion domain prompt template; S54. The lightweight large language model generates structured intent labels and slot parameters through reasoning. If the intent is acupoint consultation or symptom consultation, the corresponding physiotherapy plan is retrieved from the local moxibustion knowledge base according to the slot parameters and assembled into interactive instructions. S55. Output the generated interaction command and its confidence score together, and update the intent and slot parameters of this interaction to the dialogue state buffer.

[0052] As one embodiment, the sub-step in step S5 where the secondary speech processing unit runs a locally deployed lightweight large language model to generate interactive instructions containing intent categories and slot parameters is specifically implemented as follows: S51. End-to-end speech recognition: The Conformer-CTC model trained using the WeNet open-source framework has a structure of 2 layers of Conv2D downsampling (3×3 convolutional kernels, stride 2), 12 layers of Conformer encoder (8 attention heads, 256 hidden layers), and a CTC output layer (5000 Chinese characters). The model has 38 million parameters and an inference real-time factor of 0.8 on an RK3588S NPU (i.e., processing 1 second of speech takes 0.8 seconds). Input speech: "My shoulders feel particularly sore, which acupoint should I moxibust?" Output text sequence: "My shoulders feel particularly sore, which acupoint should I moxibust?"

[0053] S52, Context Concatenation: The dialog state cache stores the current dialog state in JSON format. { "session_id": "20250327101234", "history": [ {"role": "user", "content": "Xiaoi Xiaoi", "intent": "wakeup"}, {"role": "system", "content": "Woken up."} , "last_intent": "wakeup", "slots": {} } Concatenate the last system response above with the current user input to form the input text: "Above: Woken up.; Current: I feel my shoulders are extremely sore. Which acupoints should be moxibusted?"

[0054] S53. Inference of the large language model: The lightweight large language model is Qwen2.5 - 0.5B, which is obtained after knowledge distillation (the teacher model is Qwen2.5 - 7B), 4-bit GPTQ quantization, and 30% structured pruning. The number of parameters is approximately 80 million, deployed on the NPU, and the INT8 inference latency is approximately 250 ms. The loaded prompt template is: "Current conversation context: [Above] User's current input: [Current text] Please analyze the user's intention based on knowledge in the moxibustion field. Possible intentions include: setting temperature, setting time, acupoint consultation, symptom consultation, status query, emergency stop.

[0055] Extract slot parameters: acupoint name, temperature value, time value, symptom description.

[0056] Output format: Intention: [Intention]; Slots: [Slot name: Slot value,...]" Model output: "Intention: Symptom consultation; Slots: Symptom description: Shoulder soreness" S54. Knowledge base retrieval: Query the moxibustion knowledge base based on "Symptom description: Shoulder soreness".

[0057] The knowledge base is stored in SQLite, with table structures including: acupoint_id, name, pinyin, location, indication, symptom_keywords, moxa_method, and contraindication. Full-text search is used to match the symptom_keywords field, finding results for "Jianjing acupoint" (keywords containing "shoulder soreness" and "shoulder pain") and "Jianyu acupoint" (keywords containing "shoulder and arm pain"). Search results are sorted by matching degree, with the top three results selected. Additionally, the knowledge base supports vector retrieval as a backup: symptom text is converted to a 128-dimensional vector using Sentence-BERT, and cosine similarity is calculated between this vector and all pre-calculated acupoint vectors as a supplement.

[0058] S55, Interactive Command Assembly: Assemble into JSON commands: { "intent": "symptom_consult", “slots”: {“symptom”: “shoulder pain”}, "response": { The text states: "For shoulder pain, moxibustion can be applied to the Jianjing acupoint, located at the midpoint of the line connecting the Dazhui and acromion. It is recommended to apply moxibustion for 15-20 minutes each time, with the temperature controlled at 50-55℃. Combining this with moxibustion at the Jianyu and Tianzong acupoints will enhance the effect." "acupoints": ["Jianjing point", "Jianpi point", "Tianzong point"], “details”: [ {"name": "Jianjing acupoint", "location": "midpoint of the line connecting Dazhui and the acromion", "method": "gentle moxibustion for 15-20 minutes"} {"name": "Jianyu acupoint", "location": "Anterior and inferior to the acromion, between the acromion and the greater tubercle of the humerus", "method": "Gentle moxibustion for 10-15 minutes"} {"name": "Tianzong acupoint", "location": "Central depression of the infraspinous fossa of the scapula", "method": "Gentle moxibustion for 10-15 minutes"} ] }, "confidence": 0.89 } Update this interaction to the dialogue state cache, add two new records to history, update last_intent to "symptom_consult" and slots to {"symptom": "shoulder pain"}.

[0059] S6. The interaction command is sent to the main control module for execution, and the execution result is broadcast to the user through the speech synthesis module. At the same time, the context information of this interaction is stored in the dialogue state cache for semantic disambiguation in subsequent multi-round dialogues.

[0060] It should be noted that the lightweight large language model is a Transformer model that has undergone knowledge distillation, weight quantization, and structured pruning and compression, and is deployed on the embedded processor of the moxibustion device; the local moxibustion knowledge base pre-stores acupoint location information, acupoint therapeutic effects, acupoints corresponding to common symptoms, and structured data on moxibustion contraindications, and supports two query methods: vector retrieval and keyword matching.

[0061] As one embodiment, the lightweight large language model is a Transformer model compressed through knowledge distillation, weight quantization, and structured pruning. The specific compression process is as follows: (1) Knowledge distillation: Using Qwen2.5-7B as the teacher model and 5 million questions and answers in the field of moxibustion as the training set, the student model Qwen2.5-0.5B is trained. The loss function is KL divergence + cross-entropy, so that the output distribution of the student model approximates the teacher model.

[0062] (2) Structured pruning: The attention heads of Qwen2.5-0.5B are sorted by L1 norm and the 30% of heads with the lowest contribution are pruned (from 8 heads to 5 heads); the amplitude of the neurons in the FFN layer is pruned and the 20% with the smallest amplitude is pruned.

[0063] (3) Weight quantization: The GPTQ algorithm is used to quantize the model weights from FP16 to INT4, while keeping the activation at FP16 and dynamically dequantizing during inference. After quantization, the model size is reduced from 900MB to about 60MB.

[0064] The final model has approximately 80 million parameters (approximately 500 million in the original 0.5B, compressed by 84%), deployed on the RK3588S NPU, with an INT4 inference latency of approximately 250ms and a power consumption of 1.2W.

[0065] The local moxibustion knowledge base adopts a hybrid architecture of SQLite + FAISS: structured data (acupoint information) is stored in the SQLite table `acupoints`, containing fields such as `id`, `name`, `pinyin`, `location`, `indication`, `symptom_keywords`, `moxa_method`, `contraindication`, and `image_path`; unstructured data (symptom description text) is converted into 128-dimensional vectors using Sentence-BERT and stored in the FAISS index, supporting vector similarity retrieval. During queries, a full-text SQL search is first performed using `symptom_keywords`. If the number of results is less than 3, a vector search is initiated to supplement the results. The results from both searches are then merged, deduplicated, and sorted. The knowledge base includes 409 national standard acupoints and 2000 common symptom-acupoint correspondences.

[0066] It should be noted that the method also includes multi-level security redundancy and conflict arbitration steps: S7. The main control module monitors the working status parameters of the moxibustion device in real time, including temperature, working time, and sensor status. When the temperature exceeds the safety threshold or the sensor malfunctions, the voice synthesis module is automatically triggered to broadcast a warning message and force entry into the safety protection mode. At this time, the voice control commands only respond to emergency stop and confirmed stop. S8. When the primary voice processing unit and the secondary voice processing unit output commands simultaneously, the arbitrator of the main control module executes the commands according to the predefined priority rules. The priorities from high to low are: emergency stop command > safety-related command > real-time control command > consultation-type interactive command. S9. When the voice interaction system fails to function properly due to a malfunction, the backup touch control or mobile APP control channel will be automatically activated, and the user will be prompted to use the alternative interaction method through indicator lights or display screen.

[0067] As one embodiment, the present invention further includes multi-level security redundancy and conflict arbitration steps: S7. The main control module monitors the operating status parameters of the moxibustion device in real time: it monitors the heating element temperature via an NTC sensor (sampled every 100ms), records the cumulative operating time via a timer, and monitors the sensor communication status via a self-test program. When the detected temperature exceeds the safety threshold of 60℃ or the sensor communication fails 5 times consecutively, the voice synthesis module is automatically triggered to broadcast the warning message "Temperature too high, please check the device" or "Sensor malfunction, please stop using," and forcibly enters the safety protection mode. In the safety protection mode, voice control commands only respond to "Emergency Stop" and "Confirm Stop," and other commands are ignored.

[0068] S8. Command Conflict Arbitration: When the primary voice processing unit and the secondary voice processing unit output commands simultaneously, the arbitrator of the main control module executes the priority rules. In this embodiment, the priorities from high to low are as follows: First priority (highest): Emergency stop commands (such as "emergency stop", "too hot", "stop immediately") are directly identified and executed by the first-level unit, regardless of whether the second-level unit is working; Second priority: Safety-related instructions (such as "lower temperature" or "turn off essential oils") should be responded to first; Third priority: Real-time control commands (such as "increase temperature" or "turn on essential oil"); Fourth priority: Consultation-type interactive commands.

[0069] For example, after a user identifies "temperature increase" in the first-level unit, they immediately say "too hot." The second-level unit may be processing the inquiry, but "too hot" is identified as an emergency stop by the first-level unit (through keyword matching). The arbitrator immediately terminates the execution of the second-level instruction and executes the emergency stop.

[0070] S9. Backup Interaction Channel: When the voice interaction system malfunctions (such as microphone damage or voice chip failure), the main control module detects no response from the primary unit via a watchdog timer and automatically activates the backup touch control (touch buttons on the display screen) and mobile APP control channels. An indicator light (flashing red LED) prompts the user to use the alternative interaction method. The touchscreen displays the message "Voice malfunction, please touch operation".

[0071] For example, if the microphone is contaminated by essential oils during use, causing signal distortion, the VAD will not be able to detect voice. After 30 consecutive seconds of no valid voice input, the system will determine that the voice module is abnormal, automatically light up the red LED and enable the touch screen, and the user can adjust the temperature by touching the buttons.

[0072] It should be noted that the method also includes environmental adaptive optimization and personalized learning steps: S10. Record the acoustic features and signal-to-noise ratio of each user's wake word pronunciation. Use an online Bayesian learning algorithm to update the weight parameters of the background noise adaptive layer of the wake word model, so that the wake-up rate of the model is stabilized above the preset ratio in non-stationary noise environments such as moxibustion burning sound and ambient human voice. The update and the front-end beamforming in step S11 form a two-level noise suppression. S11. Statistically analyze the frequency distribution of various commands in daily user use, dynamically adjust the confidence threshold of the short command recognition model, use a lower threshold for high-frequency commands to improve response speed, and use a higher threshold for low-frequency commands to reduce false triggering. S12. Record the names of the acupoints commonly used by the user and their temperature preferences, build a user profile, and add personalized preference fields to the prompt template to make the recommended physiotherapy plan more in line with the individual needs of the user.

[0073] As an embodiment, the present invention further includes steps of environmental adaptive optimization and personalized learning: S10. Wake-up word model adaptation: Record the acoustic features (40-dimensional MFCC mean) of the user's pronunciation of the wake-up word each time and the current signal-to-noise ratio (calculated by the ratio of the signal energy during VAD to the noise energy in the silent segment). Use the online Bayesian learning algorithm to update the mean and variance parameters of the background noise adaptation layer (BN layer) of the wake-up word model. Specifically, after every 50 awakenings, the features of the most recent 50 awakenings are weighted and fused with the features of the initial training data to update the running_mean and running_var of the BN layer, making the model more robust to the timbre of the current user and the noise characteristics of the current environment. It is measured that in the environment of the sound of moxibustion burning, the wake-up rate increases from 92% to 98.5% after adaptation.

[0074] S11. Dynamic adjustment of confidence threshold: Statistically analyze the frequency distribution of various instructions in the user's daily use, and update it once every 24 hours. For high-frequency instructions (such as "increase temperature", usage frequency > 5 times / day), lower the recognition threshold from 0.7 to 0.6 to improve the response speed; for low-frequency instructions (such as "remaining time", usage frequency < 1 time / day), keep the threshold at 0.7 to reduce false triggers; for instructions that have never been used, set the threshold to 0.8. The adjustment rules are stored in Flash through the user profile JSON file.

[0075] S12. Personalized prompt template: Record the names of the acupoints commonly used by the user (such as the user often moxibusts "Zusanli") and temperature preferences (such as preferring 52°C), and build a user profile: { "user_id": "U10086", "freq_acupoints": ["Zusanli", "Guanyuan", "Mingmen"], "pref_temp": 52, "last_used": "2025-03-27" } Add personalized preference fields to the prompt template, in the format of: "Current conversation context: [Previous text] User's current input: [Current text] User's preferred acupoints: [Zusanli, Guanyuan, Mingmen] Preferred temperature: 52°C Please analyze user intent based on knowledge of moxibustion and user preferences... When a user asks, "I want to do moxibustion on my stomach today," the model combines the user's preferred acupoint "Guanyuan" and recommends "Guanyuan" instead of a general recommendation, making the treatment plan more in line with the user's individual needs.

[0076] Example 2: Please see Figure 2 , Figure 2 This is a schematic diagram of the hardware device operation according to an embodiment of the present invention. The hardware device specifically includes: a cascaded voice interaction device 401 for moxibustion devices, a processor 402, and a storage medium 403.

[0077] A cascaded voice interaction device 401 for moxibustion devices: The cascaded voice interaction device 401 for moxibustion devices implements the cascaded voice interaction method for moxibustion devices.

[0078] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the cascaded voice interaction method for moxibustion devices.

[0079] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the cascaded voice interaction method for moxibustion devices.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cascaded voice interaction method for moxibustion devices, characterized in that: Includes the following steps: S1. Acquire the user's voice signal in real time through a microphone array, preprocess the voice signal, detect voice activity and extract acoustic features to obtain an effective voice frame sequence and its corresponding acoustic feature map; S2. The acoustic feature map is input to the first-level speech processing unit. The first-level speech processing unit runs a low-power wake word detection model and a short command recognition model in the field of moxibustion. When a wake word is detected, a wake signal is output and the command recognition is enabled. When a short command is recognized, the corresponding control command is directly output to the main control module for execution. S3. When the first-level speech processing unit fails to recognize the short command, it calculates the complex intent tendency score of the current speech. When the score exceeds the preset wake-up threshold, it is determined to be a complex interaction intent and a wake-up request is sent to the second-level speech processing unit. S4. In response to the wake-up request, start the secondary speech processing unit and transmit the effective speech frame sequence and its corresponding acoustic feature map to the secondary speech processing unit. S5. The secondary speech processing unit runs a locally deployed lightweight large language model, takes the acoustic feature map and effective speech frame sequence as input, performs integrated recognition and understanding through an end-to-end speech-semantic joint model, and generates interactive instructions containing intent categories and slot parameters by combining the knowledge base of moxibustion domain. S6. The interaction command is sent to the main control module for execution, and the execution result is broadcast to the user through the speech synthesis module. At the same time, the context information of this interaction is stored in the dialogue state cache for semantic disambiguation in subsequent multi-round dialogues.

2. The cascaded voice interaction method for moxibustion devices as described in claim 1, characterized in that: The sub-steps of preprocessing the speech signal, detecting speech activity, and extracting acoustic features in step S1 include: S11. A dual-microphone array is used to collect voice signals. Adaptive beamforming is performed through a generalized sidelobe canceller algorithm to enhance the user's sound source direction signal and suppress the noise of burning moxa sticks and environmental human voice interference in the moxibustion environment. S12. Pre-emphasize the beamformed signal to compensate for the high-frequency attenuation of the voice signal; S13. Divide the pre-emphasized signal into frames according to the preset frame length and frame shift, and apply a Hamming window for windowing processing. S14. Calculate the short-time energy and zero-crossing rate for each frame of signal, use the dual-threshold method to detect speech activity, remove silent frames and pure noise frames, and obtain an effective speech frame sequence. S15. Extract Mel frequency cepstral coefficient features from the effective speech frame sequence to generate an acoustic feature map.

3. The cascaded voice interaction method for moxibustion devices as described in claim 1, characterized in that, The specific steps in step S3 for calculating the complex intent tendency score of the current speech include: S31. Extract speech duration, number of pauses within a sentence, average speech rate, and speech emotion feature parameters from the effective speech frame sequence. The emotion feature parameters include the fundamental frequency mean, energy jitter, and speech rate change rate. S32. Input the acoustic feature map into a lightweight intention tendency prediction network, and at the same time use the emotion feature parameters as auxiliary input. The network outputs the probability value that the current voice belongs to the consultation type interaction. S33. The speech duration, number of pauses within a sentence, average speech rate and probability value are weighted and fused together to obtain a complex intent tendency score; S34. When the score exceeds the first threshold, it is determined to be a complex interaction intention; when the score exceeds the higher second threshold, it is marked as a complex interaction during the voice acquisition stage, and the effective voice frame sequence and its acoustic feature map are directly transmitted to the secondary processing unit to shorten the response time.

4. The cascaded voice interaction method for moxibustion devices as described in claim 1, characterized in that: The short command recognition model in the moxibustion field in step S2 is a deep separable convolutional neural network. Its output layer classification categories include at least: increase temperature, decrease temperature, turn on essential oil, turn off essential oil, query remaining time, and emergency stop. The training data of the model includes noisy speech samples collected in the moxibustion scenario to improve the recognition robustness in real environment.

5. The cascaded voice interaction method for moxibustion devices as described in claim 1, characterized in that: The sub-step in step S5 where the secondary speech processing unit runs a locally deployed lightweight large language model to generate interactive instructions containing intent categories and slot parameters includes: S51. Input the received valid speech frame sequence into the end-to-end speech recognition model, which is a convolutional enhanced Transformer architecture based on connection temporal classification, and directly outputs the text sequence. S52. The text sequence is concatenated with the preceding information read from the dialogue state buffer to form input text with context. S53. Input the contextualized input text into the lightweight large language model and load the predefined moxibustion domain prompt template; S54. The lightweight large language model generates structured intent labels and slot parameters through reasoning. If the intent is acupoint consultation or symptom consultation, the corresponding physiotherapy plan is retrieved from the local moxibustion knowledge base according to the slot parameters and assembled into interactive instructions. S55. Output the generated interaction command and its confidence score together, and update the intent and slot parameters of this interaction to the dialogue state buffer.

6. The cascaded voice interaction method for moxibustion devices as described in claim 5, characterized in that: The lightweight large language model is a Transformer model that has undergone knowledge distillation, weight quantization, and structured pruning and compression, and is deployed on the embedded processor of the moxibustion device; the local moxibustion knowledge base pre-stores acupoint location information, acupoint therapeutic effects, acupoints corresponding to common symptoms, and structured data on moxibustion contraindications, and supports two query methods: vector retrieval and keyword matching.

7. The cascaded voice interaction method for moxibustion devices as described in claim 1, characterized in that, The method also includes multi-level security redundancy and conflict arbitration steps: S7. The main control module monitors the working status parameters of the moxibustion device in real time, including temperature, working time, and sensor status. When the temperature exceeds the safety threshold or the sensor malfunctions, the voice synthesis module is automatically triggered to broadcast a warning message and force entry into the safety protection mode. At this time, the voice control commands only respond to emergency stop and confirmed stop. S8. When the primary voice processing unit and the secondary voice processing unit output commands simultaneously, the arbitrator of the main control module executes the commands according to the predefined priority rules. The priorities from high to low are: emergency stop command > safety-related command > real-time control command > consultation-type interactive command. S9. When the voice interaction system fails to function properly due to a malfunction, the backup touch control or mobile APP control channel will be automatically activated, and the user will be prompted to use the alternative interaction method through indicator lights or display screen.

8. The cascaded voice interaction method for moxibustion devices as described in claim 1, characterized in that: The method also includes environmental adaptive optimization and personalized learning steps: S10. Record the acoustic features and signal-to-noise ratio of each user's wake word pronunciation. Use an online Bayesian learning algorithm to update the weight parameters of the background noise adaptive layer of the wake word model, so that the wake-up rate of the model is stabilized above the preset ratio in non-stationary noise environments such as moxibustion burning sound and ambient human voice. The update and the front-end beamforming in step S11 form a two-level noise suppression. S11. Statistically analyze the frequency distribution of various commands in daily user use, dynamically adjust the confidence threshold of the short command recognition model, use a lower threshold for high-frequency commands to improve response speed, and use a higher threshold for low-frequency commands to reduce false triggering. S12. Record the names of commonly used acupoints and temperature preferences of users to build user profiles. Add personalized preference fields to the prompt template to make the physiotherapy plan recommendations more in line with the individual needs of users.

9. A storage medium, characterized in that: The storage medium stores instructions and data to implement the cascaded voice interaction method for moxibustion devices as described in any one of claims 1 to 8.

10. A cascaded voice interaction device for moxibustion equipment, characterized in that: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the cascaded voice interaction method for moxibustion devices as described in any one of claims 1 to 8.

Images