An online voice air conditioning system and method

Abstract

The application discloses an online voice air conditioner system and method, and the method comprises the following steps: configuring two wake-up modes of a long wake-up word and a short wake-up word on a voice module; when the short wake-up word is recognized, uploading the recording containing the wake-up word to an online voice platform, performing intention judgment through a cloud large model, and not performing reply broadcast and not entering a wake-up state if the judgment is invalid intention; storing an offline model and an online model on the voice module, independently separating the running codes of the two sets of models, and switching between the offline model and the online model through a flag bit distributor switching protocol; when switching to the offline mode, only enabling the long wake-up word, and enabling the offline model to run full-quantity command word recognition in an offline memory pool; when switching to the online mode, enabling the long wake-up word and the short wake-up word, and enabling the online model to run recording collection and uploading and cloud large model intention judgment, improving the capacity of offline voice command words, and enabling the offline mode to have full-quantity command word recognition capability.

Description

Technical Field

[0001] This invention relates to the field of intelligent air conditioning control technology, specifically to an online voice-controlled air conditioning system and method. Background Technology

[0002] Voice control has become an important interaction method for smart air conditioners; existing voice-activated air conditioners typically require users to say a wake-up word, wait for the device to respond, and then input control commands, resulting in a lengthy interaction process. Furthermore, common wake-up words are usually around four characters long, making it difficult to say them consecutively with the desired commands, leading to a less than smooth user experience.

[0003] To improve the user experience, existing technologies include shortening wake words and eliminating the need for wake-up commands. However, while shortening the wake word to two characters improves the continuous speaking experience, it significantly increases the probability of false wake-ups. Test data shows that a two-character wake word can generate up to 356 false wake-ups within 72 hours, severely disturbing users.

[0004] Several existing patents have attempted to address the above problems: CN107388487B discloses a scheme with multiple preset wake words corresponding to different control modes, and supports switching between online and offline modes. However, both of its wake words follow the same "wake-up-wait-command" interaction mode, and do not achieve continuous speaking interaction. Its online and offline switching adopts the same space model switching method, which results in a large switching delay.

[0005] CN107591151B discloses a mechanism in which an online engine performs secondary confirmation after offline wake-up to reduce false wake-ups. However, this solution requires the online engine to be constantly running, which occupies the limited memory resources of the embedded chip, resulting in a severe compression of the offline command word recognition capability.

[0006] CN109326289A and CN118248138A disclose semantic intent judgment in wake-free voice interaction, while CN122201268A discloses a scheme for intent recognition using a cloud-based large language model.

[0007] In summary, existing technologies suffer from the following technical problems: On resource-constrained embedded voice chips, when offline and online models run in combination, the online model requires code space and runtime memory for functions such as recording, audio uploading, and network communication, resulting in a severe compression of the recognition capability of offline voice command words. In this case, only about 50 command words can be supported during mixed operation. How to improve the command word recognition capability in offline mode to nearly 20,000 words through architectural innovation, while ensuring low latency and high reliability during mode switching, remains a technical challenge that existing technologies have not yet solved. Furthermore, how to shorten the wake word to improve the continuous speaking experience while reducing the false wake-up rate to zero without affecting the user experience in offline mode requires the coordinated design of offline architecture, cloud-based large-model filtering, and a long / short wake word mechanism. Summary of the Invention

[0008] This invention proposes an online voice-activated air conditioning system and method to address the problems described in the background section.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: An online voice-activated air conditioning method of the present invention includes the following steps: S1. Configure two wake-up modes, long wake-up word and short wake-up word, on the voice module; S2. When a short wake word is detected, the recording containing the wake word is uploaded to the online voice platform. The cloud-based big data model is used to determine the intent. If the intent is determined to be invalid, no response will be made and the wake-up state will not be entered. S3. Store offline and online models on the voice module. The running code of the two models runs independently and separately. Switching between the offline and online models is achieved through the flag dispatcher switching protocol. S4. When switching to offline mode, only the long wake word is enabled, and the offline model exclusively uses the offline memory pool to run full command word recognition; when switching to online mode, both the long and short wake words are enabled, and the online model runs recording acquisition and uploading and cloud-based big model intent judgment.

[0010] Preferably, the RAM of the voice module is dynamically allocated and shared by an offline memory pool and an online memory pool. The offline memory pool is dynamically allocated and fixed during compilation, and is used to store offline acoustic model parameters, command word network, wake word model, and DSP inference buffer. The online memory pool is dynamically allocated and requested at runtime, and is used to store recording acquisition buffer, audio encoding buffer, network transmission queue, and simplified wake word model. The two memory pools have independent memory manager instances and do not interfere with each other.

[0011] Preferably, the flag bit distributor switching protocol includes: after the WiFi module detects a network status change or receives a mode switching command, it writes a 1-byte mode flag bit to the voice module via a serial port; the flag bit distributor on the voice module listens to the flag bit, and upon receiving a switching command, sequentially executes the following steps: suspending the DSP task context of the current model and saving it to the Flash context storage area, releasing the RAM space occupied by the current model, reading the target model firmware from the Flash and loading it into the target memory pool, switching the memory manager instance pointer, activating the entry function of the target model, and restoring the DSP task context of the target model from the Flash context storage area.

[0012] Preferably, the DSP and NPU within the voice module run different execution contexts in offline and online modes, respectively. In offline mode, the DSP runs complete offline acoustic model inference, and the NPU runs an offline command word recognition neural network. In online mode, the DSP runs a simplified wake word detection, recording, and audio encoding, and the NPU runs a simplified wake word neural network. When switching from offline to online mode, the DSP and NPU register states and intermediate calculation results of the offline model are saved to the context storage area in Flash, and the offline memory pool RAM occupied by the offline model is released. When switching back to offline mode, the offline model is reloaded from Flash to the offline memory pool, and the execution context of the DSP and NPU is restored from the Flash context storage area.

[0013] Preferably, the offline model and the online model employ completely independent training processes and are not derived from each other. The training objective of the offline model is to maximize the command word recognition coverage and wake word accuracy. The training data includes voice recordings of air conditioning function command words and long wake words from no fewer than 120 people, and the command word network contains approximately 20,000 command words. The training objective of the online model is to maximize the wake word accuracy and recording quality. The training data consists of dedicated recordings of both long and short wake words and does not include the command word network. The two models employ different model compression strategies. The offline model uses INT8 quantization and pruning to adapt to the capacity budget of the offline memory pool, while the online model uses a simplified wake word model to adapt to the capacity budget of the online memory pool.

[0014] Preferably, it also includes a handover reliability assurance mechanism: a watchdog timer is started during the handover process, and if the handover times out, it automatically reverts to offline mode; the Flash checksum of the target model is verified before the handover to ensure firmware integrity; the audio being played during the handover process is faded out to avoid popping sounds; each handover records the timestamp, source mode, target mode, and handover delay in the Flash log area; if an abnormality occurs during the handover process, it automatically reverts to the mode before the handover and reports the abnormal event.

[0015] Preferably, the long and short wake words are accurately identified through a multi-level process: the first level is to enhance voice training by collecting recordings for both long and short wake words and importing them into the model for training; the second level is to perform secondary screening based on voice duration, distinguishing between short and long wake words; the third level is to confirm the wake word by uploading the wake-up recording to the cloud for ASR and NLP recognition, with the large model ultimately confirming the wake word type.

[0016] Preferably, the specific process of intent judgment by the cloud-based big model is as follows: after the voice module recognizes the short wake word, it starts recording and uploads it to the online voice platform. After ASR speech-to-text and NLP natural language processing, the lightweight control inference big model determines the wake word content and decides the response method. If no service that the air conditioner can perform is matched in the cloud, the voice broadcast will not be sent, ensuring that the air conditioner will not generate broadcast interference when it is mistakenly awakened by surrounding sounds without recognizing the user's effective operation.

[0017] Preferably, in the offline mode, short wake words are disabled and only long wake words are retained to avoid false wake-ups when there is no large model filtering offline; the offline model supports a custom wake word learning function, and users can input custom wake words; the online mode supports traditional interaction with long wake words and continuous interaction with short wake words, and also supports weather query, air conditioner operation status query, playback content and intelligent correction function.

[0018] An online voice-activated air conditioning system includes an air conditioning motherboard MCU, a WiFi module, a voice module, an IoT platform, and an online voice platform, wherein: The voice module is used to recognize wake-up voices and stores two independently running voice models: an offline model and an online model. It is configured with a flag dispatcher, which is used to switch between online and offline models by suspending the DSP task context of the current model, switching the memory manager instance pointer, activating the target model entry function, and restoring the DSP task context of the target model. The WiFi module is used to connect to the online platform, upload voice recordings, receive control commands from the IoT platform and audio replies from the online voice platform, and is responsible for issuing commands to switch between online and offline models. The IoT platform is used to process mobile application messages, maintain online connection with the WiFi module, and forward control commands. The online voice platform is used for parsing and processing voice commands, and includes ASR, NLP and a lightweight control reasoning model to determine intent and decide on the response method; The air conditioner's mainboard MCU is used to receive control commands forwarded by the WiFi module and execute air conditioner function control.

[0019] As can be seen from the above technical solution, the present invention provides an online voice-activated air conditioning method. Compared with the prior art, the present invention has the following advantages: 1. This invention supports both a wake-up-free user experience and traditional wake-up voice interaction through voice interaction. The two modes are seamlessly compatible and require no learning cost for users.

[0020] 2. This invention reduces the number of false wake-ups caused by short wake words by judging intent through a large cloud-based model, thus completely eliminating the disturbance of false wake-ups while ensuring a continuous speaking experience.

[0021] 3. This invention improves the capacity of offline voice command words by independently separating the online and offline model code, switching the flag dispatcher protocol, and using the DSP time-division execution context, thus enabling the offline mode to recognize all command words.

[0022] 4. This invention avoids the problem of accidental wake-up due to short wake words when offline, and switching to independent operation can also increase offline voice capabilities and avoid resource waste.

[0023] 5. This invention achieves a continuous speaking experience, zero false wake-ups, and offline capability with multiple command words through the synergy of long and short wake-up words, cloud-based large model filtering, and an independent and separate operating architecture. Attached Figure Description

[0024] Figure 1 This is a flowchart of an online voice-activated air conditioning method according to the present invention.

[0025] Figure 2 This is a system block diagram of an online voice-activated air conditioning system according to the present invention.

[0026] Figure 3 This is a diagram of the independent and separate operation architecture of the online and offline models of this invention.

[0027] Figure 4 This is a flowchart of the interaction process of the long and short wake-up words of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0029] like Figure 1 As shown, an online voice-activated air conditioning method according to this embodiment includes the following steps: S1. Configure two wake-up modes, long wake-up word and short wake-up word, on the voice module. The short wake-up word is used for continuous voice input with control commands, and the long wake-up word is used to wake up the device and wait for a response before inputting control commands. S2. When a short wake word is detected, the recording containing the wake word is uploaded to the online voice platform. The cloud-based big data model is used to determine the intent. If the intent is determined to be invalid, no response will be made and the wake-up state will not be entered. S3. Store offline and online models on the voice module. The running code of the two models runs independently and separately. Before running, the offline and online models are stored in Flash in the form of firmware. The limited RAM of the voice module is divided into dynamically allocated offline memory pool and online memory pool. Both memory pools adopt dynamic allocation method, and memory pools can also be dynamically allocated. The switching between offline and online models is realized through the flag bit dispatcher switching protocol. During the switching, the RAM occupied by the current model is released and the target model is loaded from Flash to the corresponding memory pool. S4. When switching to offline mode, only the long wake word is enabled, and the offline model exclusively uses the offline memory pool to run full command word recognition; when switching to online mode, both the long and short wake words are enabled, and the online model runs recording acquisition and uploading and cloud-based big model intent judgment.

[0030] like Figure 2 As shown, an online voice-activated air conditioning system according to this embodiment includes an air conditioning motherboard MCU, a WiFi module, a voice module, an IoT platform, and an online voice platform; the functional division of each module is as follows: Voice module: Responsible for recognizing wake-up voice, it stores two sets of independently running voice models, one offline and one online. Both models are stored in Flash as firmware before running, and support dynamic allocation of dual memory pools and Flash loading switching. The voice module usually uses an embedded voice chip with integrated DSP and NPU, with RAM capacity typically ranging from 512KB to 1MB and Flash capacity typically ranging from 2MB to 4MB.

[0031] WiFi module: Responsible for connecting to the online platform, uploading voice recordings, receiving control commands from the IoT platform and audio responses from the online voice platform, and issuing commands for switching between online and offline models; the WiFi module and the voice module communicate via serial port.

[0032] IoT platform: processes mobile application messages, maintains online connection with WiFi modules, and forwards control commands.

[0033] Online voice platform: responsible for parsing and processing voice commands, including ASR automatic speech recognition, NLP natural language processing and a lightweight control reasoning model, used to determine intent and decide on the response method.

[0034] Air conditioner mainboard MCU: Receives control commands forwarded by the WiFi module and executes air conditioner function control, such as power on / off, mode switching, temperature adjustment, and fan speed adjustment.

[0035] Independent and separate operating architecture for online and offline models: like Figure 3 As shown, the present invention is that the offline model and the online model on the voice module do not run in the same set of code, but the running code of the two are completely separated. The invention achieves maximum utilization of computing power and reliable switching through the following five technical mechanisms.

[0036] (1) Dual memory pool mechanism; The limited RAM of the voice chip is divided into two memory pools. Both models are stored in Flash as firmware before running. When switching, only the target model is loaded from Flash to the corresponding RAM memory pool, and the RAM occupied by the current model is released. The two models do not reside in RAM at the same time. Pool A, Offline Memory Pool: Belongs to the offline model and is dynamically allocated. When switching to offline mode, the offline model is loaded from Flash and dynamically allocated during runtime. The typical capacity is about 400KB, which is used to store acoustic model parameters, command word network, wake word model and DSP inference buffer.

[0037] Pool B, Online Memory Pool: Belongs to the online model and also uses dynamic allocation. When switching to online mode, it loads the online model from Flash and dynamically allocates memory during runtime. Its typical capacity is about 150KB, used to store the recording acquisition buffer, audio encoding buffer, network transmission queue, and a simplified wake word model.

[0038] Flash firmware storage area: Both offline and online models are stored in Flash as firmware before running, with a typical Flash size of about 2MB to 4MB. The offline model firmware is about 400KB, containing acoustic model parameters, command word network, and wake word model, while the online model firmware is about 150KB, containing a simplified wake word model, recording acquisition, and audio encoding modules.

[0039] In addition, a context storage area is provided in Flash to save and restore the task context state of the DSP / NPU during switching.

[0040] The two memory pools have independent memory manager instances and do not interfere with each other. In traditional hybrid operation schemes, the runtime data of offline and online models share the same memory space. The recording and acquisition, audio encoding, network communication and other modules of online functions reside in memory. Even when not used in offline mode, they occupy about 150KB to 200KB of RAM, which results in the available command word network space of the offline model being compressed to only 50 command words.

[0041] This invention utilizes a dual memory pool and Flash storage-dynamic loading mechanism. When switching to offline mode, it loads the offline model from Flash and exclusively uses approximately 400KB of RAM in Pool A, which can accommodate an acoustic model and language model network with approximately 20,000 command words, achieving a 400-fold increase in command word capacity.

[0042] (2) Flag distributor switching protocol; like Figure 3 As shown, the switching between online and offline models is achieved through a lightweight flag dispatcher, rather than a traditional full firmware reboot process: Step a, Switching trigger: The WiFi module detects a change in network status, or the mobile APP sends a mode switching command.

[0043] Step b, Flag writing: The WiFi module writes a 1-byte mode flag to the voice module via the serial port. 0x00 indicates offline mode and 0x01 indicates online mode.

[0044] Step c, Distributor Response: The flag distributor on the voice module listens for the serial port flags and performs the following operations upon receiving the switching command: (c1) Suspend the DSP task context of the current model, save the register state and task stack pointer to the Flash context storage area, which takes less than 10ms; (c2) Release the RAM space currently occupied by the model and return it to the corresponding memory manager instance, which takes less than 5ms; (c3) Read the target model firmware data from Flash and load it into the target memory pool. The dynamically allocated RAM area takes about 80 to 120 ms to load the offline model (about 400 KB) and about 30 to 50 ms to load the online model (about 150 KB). (c4) Switch the memory manager instance pointer, from the MM of Pool A to the MM of Pool B, or vice versa, in less than 1ms; (c5) Activate the entry function of the target model and initialize the model runtime environment, which takes less than 5ms; (c6) Read and restore the DSP task context of the target model from the Flash context storage area, restore the register state and task stack pointer, which takes less than 10ms.

[0045] The total switching latency is completed within 200ms, which is virtually imperceptible to the user. Compared with the traditional complete firmware reboot scheme, this invention achieves targeted loading through a flag dispatcher, loading only the target model from Flash to the allocated RAM memory pool, skipping redundant firmware reboot and global initialization processes, and significantly shortening the switching latency. Since the code of both models is stored in Flash, the switching process includes suspending the current context, releasing the current RAM, loading the target model from Flash to the target RAM, and restoring the target context, achieving full release and efficient reloading of RAM resources.

[0046] (3) DSP / NPU dual time-division execution context; The DSP and NPU within the voice chip operate in different execution contexts in offline and online modes: DSP: In offline mode, it runs the complete offline acoustic model inference, with MFCC feature matching of approximately 20,000 command words plus wake word detection. In online mode, it runs a simplified version of wake word detection plus recording acquisition and audio encoding.

[0047] NPU: In offline mode, it runs an offline command word recognition neural network, i.e., large-scale command word network inference. In online mode, it runs a simplified wake word neural network, i.e., only short and long wake words are detected.

[0048] CPU: In offline mode, it runs offline command post-processing logic and air conditioning control command output; in online mode, it runs network communication protocol stack, audio stream transmission, and model switching distributor.

[0049] The key design feature is that when the execution context of the offline model switches to online mode, its DSP / NPU register state and intermediate computation results are saved to the context storage area in Flash, and the RAM occupied by the offline model is released. When switching back to offline mode, the offline model is reloaded from Flash to the offline memory pool, and the execution context is restored from the Flash context storage area. This save-release-reload-restore mechanism ensures that RAM resources are fully released during mode switching, while maintaining the continuity of the model state through the Flash context storage area. Compared with traditional cold-start schemes after model unloading, this scheme avoids the overhead of re-initializing model parameters by saving and restoring the execution state through the Flash context storage area.

[0050] (4) Differentiated independent training strategy; Offline and online models use completely independent training processes and are not related by origin. Offline model: The training objective is to maximize command word recognition coverage and wake word accuracy; the training data includes voice recordings of air conditioning function command words and long wake words from no fewer than 120 people; the command word network contains approximately 20,000 command words, including synonyms and dialect variations; the model compression uses INT8 quantization and pruning for embedded chips, adapted to a 400KB RAM budget; only long wake words are supported; additional features include custom wake word learning.

[0051] Online model: The training objective is to maximize wake word accuracy and recording quality; the training data consists of dedicated recordings of both long and short wake words; it does not include a command word network, meaning command word recognition is performed by cloud-based ASR; the model compression uses a simplified wake word model, adapted to a 150KB RAM budget; it supports both long and short wake words, meaning the large cloud-based model filters out false wake words; additional functions include recording acquisition and transmission, online audio playback, and network communication.

[0052] The offline and online models of this invention are two independently trained models. The command word network of the offline model is far superior to that of the online model. The training data, training objectives, and model compression strategies of the two models are all different, and there is no derivation relationship between them. This independence allows the offline model to be deeply optimized for the computing power and memory constraints of embedded chips, without being constrained by the architecture of the online model. Both models are stored in Flash as firmware before running, and are loaded from Flash to RAM as needed during switching, without interfering with each other.

[0053] (5) Switching to a reliability assurance mechanism; To ensure the reliability of online / offline mode switching, the present invention designs the following safeguard mechanism: Watchdog monitoring: The watchdog timer is started during the switching process and expires after 1000ms. If the switching times out, it will automatically fall back to offline mode.

[0054] Pre-switch status verification: Before switching, verify the Flash checksum and CRC32 of the target model to ensure firmware integrity and prevent Flash firmware corruption from causing loading failure.

[0055] Seamless audio transition: During the switching process, the audio that is playing is faded out to avoid sudden interruptions and popping sounds.

[0056] Switching log recording: Each switch records the timestamp, source mode, target mode, and switch delay, which are stored in the Flash log area for fault diagnosis.

[0057] Anomaly Recovery: If an anomaly occurs during the switching process, such as a Flash read error or CRC check failure, the system will automatically revert to the mode before the switch and report the anomaly to the IoT platform.

[0058] like Figure 4 As shown, the interaction process is described in detail below as an embodiment of the present invention: (1) Traditional interaction method, i.e., long wake word; The voice system is configured with two wake words of varying lengths. When using the traditional interaction method, the user speaks a long wake word, such as "Xiaobai Xiaobai," and waits for the device to respond. At this point, the local recognition model on the voice module recognizes the wake word, enters the wake-up state, and uploads the wake word recording to the online voice platform.

[0059] The online voice platform uses natural language processing to identify long wake words, then sends out random response words and simultaneously notifies the voice module via the WiFi module to check the recognition status and receive the next round of voice recordings. After hearing the response words, the user inputs commands.

[0060] After the voice recording is uploaded to the online voice platform and recognized, if it is a control command, the online voice platform notifies the IoT platform to send it to the WiFi module, which then sends it to the air conditioner's mainboard MCU for air conditioner function control. Simultaneously, the online voice platform sends a corresponding voice response to the WiFi module, which then sends it to the voice module for playback. If it is not a control command, the online voice platform determines the content, retrieves the response from the corresponding resource, and then sends it to the module for playback.

[0061] (2) Continuous interaction method, i.e., short wake words; When using a short wake-up word interaction method, the user says a short wake-up word, such as "Xiaobai," followed by a command, such as "Xiaobai, turn on the air conditioner." The voice module first recognizes the short wake-up word and activates the system, then proceeds to the voice recording upload stage. The system waits for the voice recording to be uploaded to the online voice platform, where it judges the command content and responds in the same manner as described above.

[0062] (3) Long and short wake word recognition mechanism; Long and short wake words are accurately identified primarily through the following multi-level methods: Level 1 - Enhanced Voice Training: Collect more relevant recordings and import them into the model, then train separately for long and short wake words.

[0063] Level 2 – Secondary Filtering Based on Voice Duration: The software performs secondary filtering based on voice duration, with short wake words having significantly shorter durations than long wake words.

[0064] Level 3 - Cloud-based recognition and confirmation: The wake-up recording is uploaded to the cloud for ASR and NLP recognition, and the large model ultimately confirms the wake-up word type.

[0065] (4) False wake-up filtering mechanism; When a user uses a short wake word but does not respond to any subsequent commands, the online voice platform's large model determines that if there are no further commands or the corresponding valid intent cannot be recognized, no broadcast content will be sent, and the module will not enter the wake-up state.

[0066] When a user uses a long wake word followed by a command and speaks continuously, the voice module will still upload the voice to the online voice platform after being woken up by the long wake word. At this time, the online voice platform will first issue a response to the long wake word, and then respond according to the command content.

[0067] (5) Run in offline mode; When the device loses network access or switches to the offline model via a mobile app, the WiFi module writes a flag bit 0x00. The flag bit distributor saves the DSP task context of the online model to the Flash context storage area, releases the RAM occupied by the online model, loads the offline model from Flash to the offline memory pool, and restores the execution context of the offline model. The voice module only retains the offline model with the long wake word. Since the offline model does not need to run functions such as online recording, it can have more resources to support more local command words and custom wake words. At the same time, since there is no large model to avoid false wake-ups when offline, the problem of high false wake-up probability with short wake words is avoided.

[0068] Speed ​​mode, also known as offline model; The mobile app has two modes: Speed ​​Mode and Smart Mode. When Speed ​​Mode is selected, the WiFi module writes a flag bit 0x00 to the voice module. The flag bit distributor suspends the online model context and saves it to Flash, releases the online model RAM, loads the offline model from Flash to Pool A memory pool, and restores the offline model execution context. The switching is completed within 200ms.

[0069] Users can only wake the air conditioner using the wake word "Xiaobai Xiaobai," and after the air conditioner responds, users can control it by saying "Turn on the air conditioner." "Xiaobai Xiaobai" is a long wake word. In addition, the offline model can support more commands, such as "Air conditioner on" or "Turn on the air conditioner." It can also learn wake words and input custom wake words; in the offline model, speech recognition can only recognize trained words, and other content will not be responded to.

[0070] Intelligent mode, i.e., online model; When Smart Mode is selected, the WiFi module writes flag 0x01 to the voice module. The flag distributor suspends the offline model context and saves it to Flash, releases the offline model RAM, loads the online model from Flash into Pool B memory pool (150KB), and restores the online model execution context. The switching is completed within 200ms.

[0071] In addition to using the wake word "Xiaobai Xiaobai" and waiting for a response before controlling the air conditioner, users can also use the phrase "Xiaobai, turn on the air conditioner" to control it without interruption. "Xiaobai" is the short wake word mentioned in the text. Furthermore, online functionality supports features such as weather inquiries, air conditioner status checks, playing children's songs, and intelligent error correction. For example, when using "Xiaobai, turn on heating mode," if the current indoor temperature has already reached over 30 degrees Celsius, the voice command will not execute immediately but will instead provide a prompt.

[0072] Because "Xiaobai" is only two characters long, it's more easily woken up by surrounding sounds, making traditional wake-up and response interactions unsuitable. When the online model recognizes "Xiaobai" as the wake-up command, it begins recording and uploading the recording to the cloud for recognition. If the cloud doesn't find a service the air conditioner can perform, it won't issue a voice announcement. This ensures that if the air conditioner is mistakenly woken up by surrounding sounds before recognizing user interaction, it won't suddenly announce a message and disturb the user.

[0073] As an embodiment of the present invention, the experimental data of voice-controlled air conditioning are as follows: Experiment 1: Comparison of False Wake-Up Data with Short Wake Words Test environment: 72 hours, the air conditioner was placed next to the TV, the TV was playing programs at normal volume, and the TV was about 1 meter away from the air conditioner.

[0074] Short wake word plus no online large model filtering: 356 false wake-ups in 72 hours. The short wake word directly triggered wake-up and broadcast, generating a lot of disturbance.

[0075] Short wake word plus online big model intent judgment: 0 false wake-ups within 72 hours. The cloud big model recognizes no valid intent and does not issue a broadcast, so the user is unaware of it.

[0076] Conclusion: The cloud-based big data model's intent judgment reduced the number of false wake-ups from short wake words from 356 times per 72 hours to 0 times per 72 hours, a reduction of 100%.

[0077] Experiment 2: Comparison of Offline Command Word Capacity Data Hybrid operation, that is, offline and online sharing memory space: approximately 200KB of RAM is available offline, and approximately 50 command words are available offline.

[0078] Independent and separate operation, i.e., dynamic loading with dual memory pools and Flash storage: approximately 400KB of RAM available offline (exclusively in Pool A), approximately 20,000 offline command terms, a 400-fold improvement.

[0079] Conclusion: Running the online and offline models independently increases the offline command word capacity from approximately 50 to approximately 20,000, a 400-fold increase.

[0080] Experiment 3: Comparison of Mode Switching Delay Data The traditional solution, which involves a complete firmware reboot and reload, has a 2-3 second switching delay and requires a complete firmware reboot, model re-initialization, and parameter reconfiguration, resulting in noticeable lag for users.

[0081] The present invention, namely the flag dispatcher-directed loading, has a switching latency of less than 200ms. Directed loading is triggered by the flag, and the target model is loaded only from Flash to the allocated RAM memory pool, skipping firmware restart and redundant initialization processes, so that the user is basically unaware of it.

[0082] Conclusion: This invention achieves targeted loading through a flag dispatcher, reducing the switching latency from 2 to 3 seconds to less than 200ms compared to the traditional full firmware reboot scheme, a reduction of more than 90%.

[0083] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk), etc.

[0084] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0085] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An online voice-activated air conditioning method, characterized in that, Includes the following steps: S1. Configure two wake-up modes, long wake-up word and short wake-up word, on the voice module; S2. When a short wake word is detected, the recording containing the wake word is uploaded to the online voice platform. The cloud-based big data model is used to determine the intent. If the intent is determined to be invalid, no response will be made and the wake-up state will not be entered. S3. Store offline and online models on the voice module. The running code of the two models runs independently and separately. Switching between the offline and online models is achieved through the flag dispatcher switching protocol. S4. When switching to offline mode, only the long wake word is enabled, and the offline model exclusively uses the offline memory pool to run full command word recognition; when switching to online mode, both the long and short wake words are enabled, and the online model runs recording acquisition and uploading and cloud-based big model intent judgment.

2. The online voice-activated air conditioning method according to claim 1, characterized in that: The RAM of the voice module is dynamically allocated and shared between an offline memory pool and an online memory pool. The offline memory pool is dynamically allocated and fixed during compilation, and is used to store offline acoustic model parameters, command word network, wake word model, and DSP inference buffer. The online memory pool is dynamically allocated and requested at runtime, and is used to store recording acquisition buffer, audio encoding buffer, network transmission queue, and simplified wake word model. The two memory pools have independent memory manager instances and do not interfere with each other.

3. The online voice-activated air conditioning method according to claim 2, characterized in that: The flag bit distributor switching protocol includes: after the WiFi module detects a change in network status or receives a mode switching command, it writes a 1-byte mode flag bit to the voice module via a serial port; the flag bit distributor on the voice module listens to the flag bit, and upon receiving a switching command, sequentially executes the following steps: suspending the DSP task context of the current model and saving it to the Flash context storage area, releasing the RAM space occupied by the current model, reading the target model firmware from Flash and loading it into the target memory pool, switching the memory manager instance pointer, activating the entry function of the target model, and restoring the DSP task context of the target model from the Flash context storage area.

4. The online voice-activated air conditioning method according to claim 3, characterized in that: The DSP and NPU within the voice module operate in different execution contexts in offline and online modes. In offline mode, the DSP runs complete offline acoustic model inference, and the NPU runs an offline command word recognition neural network. In online mode, the DSP runs a simplified wake word detection, recording, and audio encoding, and the NPU runs a simplified wake word neural network. When switching from offline to online mode, the DSP and NPU register states and intermediate calculation results of the offline model are saved to the context storage area in Flash, and the offline memory pool RAM occupied by the offline model is released. When switching back to offline mode, the offline model is reloaded from Flash to the offline memory pool, and the execution context of the DSP and NPU is restored from the Flash context storage area.

5. The online voice-activated air conditioning method according to claim 4, characterized in that: The offline and online models employ completely independent training processes and are not derived from each other. The training objective of the offline model is to maximize command word recognition coverage and wake word accuracy. The training data includes voice recordings of air conditioning function command words and long wake words from at least 120 people, and the command word network contains approximately 20,000 command words. The training objective of the online model is to maximize wake word accuracy and recording quality. The training data consists of dedicated recordings of both long and short wake words and does not include the command word network. The two models employ different model compression strategies. The offline model uses INT8 quantization and pruning to adapt to the capacity budget of the offline memory pool, while the online model uses a simplified wake word model to adapt to the capacity budget of the online memory pool.

6. The online voice-activated air conditioning method according to claim 5, characterized in that: It also includes a handover reliability assurance mechanism: a watchdog timer is started during the handover process, and if the handover times out, it automatically falls back to offline mode; the Flash checksum of the target model is verified before the handover to ensure firmware integrity; During the switching process, the audio being played is faded out to avoid popping sounds; each switch records the timestamp, source mode, target mode, and switch delay in the Flash log area; if an error occurs during the switching process, it automatically reverts to the mode before the switch and reports the error event.

7. The online voice-activated air conditioning method according to claim 6, characterized in that: The long and short wake words are accurately identified through a multi-level process: the first level is to enhance voice training by collecting recordings for both long and short wake words and importing them into the model for training; the second level is to filter the voice duration by distinguishing between short and long wake words; the third level is to confirm the wake word by uploading the wake-up recording to the cloud for ASR and NLP recognition, and finally confirming the wake word type by the large model.

8. The online voice-activated air conditioning method according to claim 7, characterized in that: The specific process of intent judgment by the cloud-based big model is as follows: after the voice module recognizes the short wake word, it starts recording and uploads it to the online voice platform. After ASR speech-to-text and NLP natural language processing, the lightweight control inference big model determines the wake word content and decides the response method. If no service that the air conditioner can perform is matched in the cloud, the voice broadcast will not be sent to ensure that the air conditioner will not generate broadcast interference when it is mistakenly woken up by surrounding sounds without recognizing the user's effective operation.

9. The online voice-activated air conditioning method according to claim 8, characterized in that: In the offline mode, short wake words are disabled, and only long wake words are retained to avoid false wake-ups when there is no large model to filter them offline. The offline model supports a custom wake word learning function, and users can enter custom wake words. The online mode supports traditional interaction with long wake words and continuous interaction with short wake words. It also supports weather queries, air conditioner operation status queries, playback content, and intelligent correction functions.

10. An online voice-activated air conditioning system, employing the online voice-activated air conditioning method according to any one of claims 1-9, characterized in that, This includes the air conditioner mainboard MCU, WiFi module, voice module, IoT platform, and online voice platform, among which: The voice module is used to recognize wake-up voices and stores two independently running voice models: an offline model and an online model. It is configured with a flag dispatcher, which is used to switch between online and offline models by suspending the DSP task context of the current model, switching the memory manager instance pointer, activating the target model entry function, and restoring the DSP task context of the target model. The WiFi module is used to connect to the online platform, upload voice recordings, receive control commands from the IoT platform and audio replies from the online voice platform, and is responsible for issuing commands to switch between online and offline models. The IoT platform is used to process mobile application messages, maintain online connection with the WiFi module, and forward control commands. The online voice platform is used for parsing and processing voice commands, and includes ASR, NLP and a lightweight control reasoning model to determine intent and decide on the response method; The air conditioner's mainboard MCU is used to receive control commands forwarded by the WiFi module and execute air conditioner function control.

Images