Voice interaction method and device of smart home equipment and electronic equipment

Abstract

The invention relates to the technical field of smart home, and provides a voice interaction method and device of smart home equipment and electronic equipment. The voice interaction method of the smart home equipment comprises the following steps: acquiring a voice instruction, environment sensing data and a current state of the smart home equipment; analyzing the voice instruction by using an intention understanding engine according to the environment perception data to obtain a regulation and control intention for the smart home equipment; wherein the intention understanding engine is realized based on a first lightweight large language model; according to the regulation and control intention, in combination with the context information, generating a control instruction for the smart home equipment; wherein the context information comprises at least one of the current state of the smart home equipment, environment perception data and user preference; and executing a regulation and control task for the smart home equipment according to the control instruction. According to the invention, the voice interaction operation of the smart home equipment can be efficiently and accurately executed on the premise of ensuring the data security.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a voice interaction method, device, and electronic device for smart home devices. Background Technology

[0002] Currently, voice control of smart home devices mostly adopts an "edge-cloud collaboration" model. The local device in the home is responsible for voice signal acquisition, initial noise reduction and compression, and then the data is transmitted to a cloud server via the internet. The cloud, with its powerful computing power and massive data storage, hosts core intelligent modules such as automatic speech recognition, natural language understanding, decision-making and command generation, and device linkage logic, ultimately issuing control commands to specific devices on the home network for execution. While this architecture utilizes cloud resources to achieve complex semantic processing, it has significant drawbacks: control loop latency depends on network conditions, resulting in a poor user experience when the network is weak; home voice monitoring and data transmission pose privacy risks; and the cloud's understanding of ambiguous and personalized commands is limited, making it difficult to achieve personalized contextual understanding and dynamic decision-making.

[0003] To address the aforementioned issues, an existing technical solution has emerged: an "offline voice control system based on edge computing." This solution offloads core processing tasks locally, with the system consisting of edge computing nodes, a local device network, and a cloud synchronization module. The workflow involves first voice acquisition and front-end processing, then local voice recognition, followed by local command parsing and matching, and finally local command execution. However, this solution operates on resource-constrained edge devices, cannot handle ambiguous or contextualized commands, heavily relies on preset keywords, and suffers from rigid control logic. While it offers advantages in reducing latency and protecting privacy, it sacrifices the "intelligent" characteristics of smart home device systems, failing to meet users' needs for natural, smooth, and personalized interaction.

[0004] Therefore, how to efficiently and accurately perform voice interaction operations on smart home devices while ensuring data security is an urgent technical problem to be solved. Summary of the Invention

[0005] This invention provides a voice interaction method, device, and electronic device for smart home devices to address the aforementioned deficiencies in the prior art, thereby enabling efficient and accurate execution of voice interaction operations of smart home devices while ensuring data security.

[0006] This invention provides a voice interaction method for smart home devices, comprising: The system acquires voice commands, environmental perception data, and the current state of smart home devices. Using an intent understanding engine, it parses the voice commands based on the environmental perception data to obtain the control intent for the smart home devices. The intent understanding engine is implemented based on a first lightweight large-scale language model. Based on the control intent and contextual information, it generates control instructions for the smart home devices. The contextual information includes at least one of the current state of the smart home devices, the environmental perception data, and stored user preferences. Based on the control instructions, it executes the control task for the smart home devices.

[0007] According to the present invention, a voice interaction method for a smart home device is provided, the method further includes: obtaining a first lightweight large language model from a cloud server; and deploying the first lightweight large language model locally in a home edge intelligent hub to obtain the intent understanding engine.

[0008] According to the present invention, a voice interaction method for smart home devices is provided, wherein the intent understanding engine is trained by: acquiring a training dataset; wherein the training dataset includes labeled sample data, the sample data including voice commands acquired locally from a home edge intelligent hub, and the labels being accurate control commands for the smart home devices corresponding to the voice commands; performing the following operations in each training round: inputting the sample data into a first lightweight large language model to obtain predicted values ​​of the control commands output by the first lightweight large language model; obtaining gradient parameters of the first lightweight large language model based on the predicted values ​​of the control commands and the labels corresponding to the sample data; sending the gradient parameters to a cloud server so that the cloud server processes the locally generated gradient parameters sent by multiple home edge intelligent hubs to obtain gradient update amounts; obtaining the gradient update amounts from the cloud server and updating the parameters of the first lightweight large language model.

[0009] According to a voice interaction method for a smart home device provided by the present invention, the step of obtaining the gradient parameters of a first lightweight large-scale language model based on the predicted value of the control command and the label corresponding to the sample data includes: using a loss function to determine the difference between the predicted value of the control command and the label corresponding to the sample data; performing a derivative operation on the loss function to obtain the initial gradient parameters of the first lightweight large-scale language model; determining the command sensitivity of the sample data based on the voiceprint information and emotion information carried by the sample data; and adding noise to the initial gradient parameters based on the command sensitivity of the sample data to obtain the gradient parameters of the first lightweight large-scale language model.

[0010] According to the present invention, a voice interaction method for a smart home device, after using an intent understanding engine to parse the voice command based on the environmental perception data to obtain the control intent for the smart home device, the method further includes: determining that the confidence level of the control intent is less than a preset threshold; uploading the voice command, after desensitization, together with the environmental perception data to a cloud server, so that the cloud server uses a large language model to parse the desensitized voice command based on the environmental perception data to re-obtain the control intent for the smart home device; and receiving the control intent from the cloud server.

[0011] According to the present invention, a voice interaction method for a smart home device, after receiving the control intention from the cloud server, the method further includes: receiving a second lightweight large language model from the cloud server and deploying it locally in a home edge intelligent hub; wherein the second lightweight large language model is obtained by the cloud server optimizing the global model corresponding to the first lightweight large language model using desensitized voice commands received from each home edge intelligent hub.

[0012] The present invention also provides a voice interaction device for smart home devices, installed at the edge of a home smart hub, the device comprising: The system includes: an acquisition module for acquiring voice commands, environmental perception data, and the current state of smart home devices; a parsing module for using an intent understanding engine to parse the voice commands based on the environmental perception data to obtain the control intent for the smart home devices; wherein the intent understanding engine is implemented based on a first lightweight large-scale language model; a generation module for generating control instructions for the smart home devices based on the control intent and context information; wherein the context information includes at least one of the current state of the smart home devices, the environmental perception data, and stored user preferences; and an execution module for executing the control task for the smart home devices according to the control instructions.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the voice interaction method of any of the smart home devices described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the voice interaction method of any of the smart home devices described above.

[0015] This invention also provides a voice interaction system for smart home devices, including multiple home edge intelligent hubs and a cloud server. The cloud server performs multiple training rounds to obtain a trained first lightweight large-scale language model: acquiring gradient parameters of the initial lightweight large-scale language model trained locally, sent by each home edge intelligent hub; obtaining gradient update amounts based on a global model and the gradient parameters sent by all home edge intelligent hubs; sending the gradient update amounts to the corresponding home edge intelligent hubs so that the home edge intelligent hubs update the gradient parameters of their locally trained lightweight large-scale language models according to the gradient update amounts; the smart home device acquires voice commands, environmental perception data, and the current state of the smart home device; using an intent understanding engine, it parses the voice commands based on the environmental perception data to obtain control intentions for the smart home device; wherein the intent understanding engine is implemented based on the first lightweight large-scale language model; based on the control intentions, combined with the state of the smart home device and the environmental perception data, it generates control commands for the smart home device; and executes control tasks for the smart home device according to the control commands.

[0016] The voice interaction method, device, and electronic device for smart home devices provided by this invention acquire voice commands, environmental perception data, and the current state of the smart home devices. Utilizing an intent understanding engine based on a first lightweight large-scale language model, and combining this with deep analysis of the environmental perception data, it can accurately capture the user's true control intent for the smart home devices. This overcomes the shortcomings of traditional solutions in understanding ambiguous and contextualized commands, achieving deep semantic analysis of abstract commands such as "I'm cold." By combining contextual information such as device status, environmental data, and user preferences, optimal control strategies can be dynamically generated, such as automatically adjusting the air conditioner's operating mode based on indoor temperature and user habits. Ultimately, by executing control tasks, while ensuring millisecond-level response speed, it completely eliminates the privacy leakage risks associated with cloud transmission, providing users with a home control experience that combines intelligence, real-time performance, and security. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is a schematic diagram of the hardware environment for an interaction method of a smart device according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the voice interaction method for smart home devices provided by the present invention; Figure 3 This is a flowchart illustrating the training method for a lightweight large-scale language model provided by the present invention. Figure 4 This is a schematic diagram of the structure of the voice interaction system for smart home devices provided by the present invention; Figure 5 This is a schematic diagram of the processing module of the home edge intelligent hub and cloud server provided by the present invention.

[0020] Figure 6 This is a schematic diagram of the voice interaction process of the smart home device provided by the present invention; Figure 7 This is a schematic diagram of the structure of the voice interaction device for smart home devices provided by the present invention; Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] According to one aspect of the present invention, an interaction method for smart home devices is provided. This interaction method for smart home devices is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned interaction method for smart home devices can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0024] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0025] The following is combined Figures 2 to 3 The voice interaction method of the present invention is described.

[0026] Figure 2 This is a flowchart illustrating the voice interaction method for smart home devices provided by the present invention.

[0027] This method is executed by a home edge intelligence hub. The home edge intelligence hub is a hardware device deployed locally on the user's device. As the real-time control core of the smart home system, it is responsible for smart home interaction tasks such as voice command acquisition, local semantic parsing, and device control command generation. The home edge intelligence hub can include, but is not limited to: a high-performance smart gateway, which can integrate microphone arrays, environmental sensors (temperature / humidity / light sensors), and local computing modules (such as NPU chips); a smart speaker equipped with a dedicated AI chip, which can have a built-in edge computing unit; and a home central control unit, which acts as an edge computing node and can connect to various smart home devices (such as air conditioners, underfloor heating, and fresh air systems).

[0028] like Figure 2 As shown, the method includes the following steps.

[0029] Step 201: Obtain voice commands, environmental perception data, and the current status of smart home devices.

[0030] Voice commands are vague commands issued by users in the form of voice to control smart home devices. For example, a user might say a vague command like "It feels a bit stuffy."

[0031] In practical implementation, user voice can be collected through the microphone array built into the home edge smart hub; wake word detection can be performed to confirm the user's intent; and the collected voice signal can be processed in the front end, including echo cancellation, beamforming, and noise reduction, etc. Figure 5 As shown, a speech recognition module can be used to process speech signals, convert them into text information, and finally obtain speech instructions described in text.

[0032] In the embodiments provided by the present invention, the acquisition and processing of voice commands are completed entirely locally without uploading to the cloud, which fundamentally avoids the risk of leakage of voice data during transmission and significantly reduces the latency of voice acquisition, providing a foundation for subsequent real-time semantic analysis.

[0033] Step 202: Using the intent understanding engine, based on environmental perception data, the voice commands are parsed to obtain the control intent for smart home devices; wherein, the intent understanding engine is implemented based on the first lightweight large language model.

[0034] The intent understanding engine is a module used to parse user voice commands and extract their precise intent. Environmental perception data refers to real-time environmental status information (such as temperature, humidity, and light intensity) collected in the smart home system.

[0035] In some embodiments, a first lightweight large language model can be obtained from a cloud server; the first lightweight large language model is deployed locally at the home edge intelligence hub to obtain an intent understanding engine.

[0036] In the specific implementation process, the home edge intelligence hub obtains the first lightweight large language model from the cloud server; deploys the model on local devices to form an intent understanding engine; collects real-time environmental data (such as indoor temperature 25℃) through environmental sensors; inputs voice commands (such as "I am cold") and environmental perception data into the intent understanding engine; the engine calls the model to perform semantic parsing and extract the user's intent (such as "raise the ambient temperature").

[0037] In the embodiments provided by this invention, the intent understanding engine is implemented based on a first lightweight large-scale language model. It combines environmental perception data (such as temperature and humidity) to perform deep semantic analysis of voice commands, extracting abstract control intentions (such as "increase the ambient temperature"). This overcomes the limitation of traditional edge devices that can only process preset keywords, achieving accurate understanding of ambiguous commands (such as "I'm cold") and contextual commands (such as "create a movie-watching atmosphere"). Through localized semantic analysis, the system response speed is improved to the millisecond level, while completely avoiding the privacy leakage risks associated with cloud processing.

[0038] Step 203: Based on the control intention and combined with the context information, generate control commands for smart home devices; wherein, the context information includes at least one of the current state of the smart home devices, environmental perception data, and stored user preferences.

[0039] Contextual information refers to dynamic data related to device status, environmental conditions, and user habits in a smart home system. Control commands are the instructions that drive smart home devices to perform specific operations.

[0040] In the specific implementation process, the home edge intelligence hub obtains the current status of smart home devices (such as the air conditioner is on and the temperature is 22°C) from the device communication module; combines environmental perception data (indoor temperature 25°C) and user preferences (user preferred temperature 26°C); maps the adjustment intention (increasing the ambient temperature) into specific control commands (adjust the air conditioner temperature to 26°C) through the device control command generator; and generates primary and backup control strategies (such as prioritizing the adjustment of the air conditioner, and starting the electric heater if the air conditioner is unavailable).

[0041] The device control command generator is a core module in a smart home system responsible for transforming abstract intentions into specific device operation sequences. By integrating environmental context, device status, and user preferences, it generates executable control commands, ensuring the accuracy and scenario-based adaptation of multi-device collaboration.

[0042] In practical implementation, the device control command generator receives abstract control intentions (such as "increase ambient temperature") from the intent understanding engine; simultaneously, it obtains real-time data (such as indoor temperature 22℃, humidity 60%) from the environmental state perception module; it obtains the current state of smart home devices (such as air conditioner is on, heating mode is enabled, target temperature is 20℃) from the device communication module; and it combines this with historical settings in the user preference database (such as the user's preference for a winter room temperature of 24℃) to construct complete contextual information. Afterward, the device control command generator can use a rule engine and / or machine learning model to generate primary and backup control strategies based on the contextual information. The primary strategy is to generate the instruction "adjust the air conditioner's target temperature to 24℃" if the air conditioner is temperature-adjustable and the current temperature is lower than the user's preference. The backup strategy is to generate the instruction "start the electric heater and set the temperature to 24℃" if the air conditioner is faulty or has reached its maximum power.

[0043] In the embodiments provided by this invention, the device control command generator dynamically generates primary / backup control strategies based on the control intent and contextual information (device status, environmental data, user preferences). This effectively solves the problem of rigid control logic in traditional systems and enables scenario-based adaptation for multi-device collaboration. For example, for the intent to "raise the ambient temperature," if the air conditioner is adjustable, a "set to 24℃" command is generated; if the air conditioner malfunctions, the command switches to "start the electric heater." By combining a rule engine with a machine learning model, the accuracy and robustness of the control strategy are ensured, while reducing reliance on preset rules and improving the system's adaptability.

[0044] Step 204: Execute control tasks for smart home devices according to control instructions.

[0045] In practice, the home edge smart hub can send control commands to smart home devices through local wireless communication protocols (such as Wi-Fi and Zigbee); after receiving the commands, the smart home devices will perform corresponding operations (such as setting the air conditioner to 26°C); the operation results will be confirmed through the device status feedback mechanism; if the execution fails, a backup control strategy will be triggered.

[0046] In some embodiments, if the confidence level of the control intention is determined to be less than a preset threshold, the voice command, after being desensitized, is uploaded to the cloud server along with the environmental perception data. This allows the cloud server to use a large language model to parse the desensitized voice command based on the environmental perception data and re-obtain the control intention for the smart home device; the control intention is then received from the cloud server.

[0047] Confidence level is an assessment of the reliability of the parsed results by the intent understanding engine. Desensitization is the process of removing sensitive information (such as user identity information, location information, etc.) from voice commands.

[0048] In the specific implementation process, such as Figure 6 As shown, the home edge intelligent hub calculates the confidence level of the control intention (e.g., by judging through the probability output of the model); if the confidence level is lower than the threshold (e.g., <80%), the voice command is desensitized (the user's voiceprint features are deleted and the timestamp is replaced with a general value); the desensitized command is uploaded to the cloud server; the cloud uses a large language model with a large number of parameters and strong cognitive capabilities to re-parse the intention in combination with environmental perception data; the home edge intelligent hub receives the optimized control intention from the cloud.

[0049] In some embodiments, a second lightweight large language model can be received from a cloud server and deployed locally at a home edge intelligence hub; wherein, the second lightweight large language model is obtained by optimizing the global model corresponding to the first lightweight large language model using desensitized voice commands received from each home edge intelligence hub.

[0050] In the specific implementation process, the cloud server collects the de-identified voice commands and / or model update data uploaded by each home edge intelligence hub through the federated learning center; it optimizes the global model using a secure aggregation algorithm to generate a second lightweight large language model; the optimized model is sent to the home edge intelligence hub through incremental updates; after receiving the model, the home edge intelligence hub deploys it locally to replace the original model.

[0051] Figure 3 This is a flowchart illustrating the training method for a lightweight large-scale language model provided by this invention. Figure 3 As shown, the method includes the following steps.

[0052] Step 301: Obtain the training dataset; wherein, the training dataset includes labeled sample data, the sample data includes voice commands obtained locally from the home edge intelligent hub, and the labels are the accurate control commands for smart home devices corresponding to the voice commands.

[0053] The training dataset is a set of labeled voice commands and corresponding control commands used for model training. The sample data includes user voice commands collected locally from the home edge intelligence hub (such as "The living room lights are a bit glaring"), and the labels are the corresponding accurate device control commands (such as "Turn the brightness of the living room main light to 30%).

[0054] In the specific implementation process, the microphone array of the home edge smart hub can be used to collect users' daily voice commands; after front-end processing (noise reduction, echo cancellation) of the voice, it is converted into text to obtain the voice commands described in text; professional annotators or automated tools generate corresponding device control command labels for each voice command; the voice commands and labels are stored as sample data to build a training dataset containing multiple scenarios and multiple device types.

[0055] Perform the following steps in each training round: Step 302: Input the sample data into the first lightweight large language model to obtain the predicted value of the control command output by the first lightweight large language model.

[0056] In the specific implementation process, sample data can be input into the first lightweight large language model, and the model outputs the predicted value of the control command for smart home devices (such as the predicted command "living room light brightness 50%).

[0057] Step 303: Obtain the gradient parameters of the first lightweight large language model based on the predicted values ​​of the control commands and the labels corresponding to the sample data.

[0058] In the specific implementation process, a loss function can be used to determine the difference between the predicted value of the control command and the label corresponding to the sample data; the derivative of the loss function can be calculated to obtain the initial gradient parameters of the first lightweight large language model; the command sensitivity of the sample data can be determined based on the voiceprint information and emotion information carried by the sample data; and noise can be added to the initial gradient parameters based on the command sensitivity of the sample data to obtain the gradient parameters of the first lightweight large language model.

[0059] In some embodiments, voiceprint recognition and voice emotion analysis can be combined to determine the voiceprint information and emotion information carried by the sample data; and machine learning models can be used to determine the instruction sensitivity of the sample data based on the voiceprint information and emotion information carried by the sample data.

[0060] In some embodiments, voiceprint and emotion information carried by the sample data can also be used as multimodal sensitivity features of the sample data. Simultaneously, environmental sensors are used to capture scene information, such as daytime multi-person gatherings or nighttime scenes with no user movement. Based on the multimodal sensitivity features and scene information, a hybrid model combining rule engines and machine learning is used to determine the instruction sensitivity of the sample data. For example, a rule engine can be used to filter explicit sensitive information (e.g., information involving user identity) from the sample data. The multimodal sensitivity features and scene information are input into the machine learning model to predict implicit sensitivity (e.g., whispered conversations in multi-person scenes may contain sensitive information), and the sensitivity index is determined in real-time by combining the network status of the home edge intelligent hub. For example, if a network anomaly is detected, a first value is assigned to the sensitivity index; if the network is normal, a second value is assigned to the sensitivity index; the first value is greater than the second value. The final instruction sensitivity is the product of the model-predicted implicit sensitivity and the sensitivity index.

[0061] When adding noise to the initial gradient parameters, nonlinear algorithms (e.g., nonlinear algorithms based on exponential functions, nonlinear algorithms based on piecewise functions, etc.) can be used to determine the noise intensity based on the instruction sensitivity; based on the noise intensity, noise is sampled from the Laplace distribution; and the noise is added to the initial gradient parameters to generate the desensitized gradient parameters.

[0062] Step 304: Send the gradient parameters to the cloud server so that the cloud server can process the locally generated gradient parameters sent by multiple home edge smart hubs and obtain the gradient update amount.

[0063] In the specific implementation process, the home edge intelligence hub uploads gradient parameters to the cloud federated learning platform; the cloud server receives gradient data from multiple devices; and performs secure aggregation of gradients (such as weighted average based on secret sharing) to generate global gradient update.

[0064] Step 305: Obtain the gradient update amount from the cloud server and update the parameters of the first lightweight large language model.

[0065] In the specific implementation process, the home edge intelligent hub can receive global gradient updates from the cloud server; update the model parameters using a preset optimizer combined with the local learning rate; replace the original model layers through incremental updates; verify the accuracy of the updated model on the local validation set (e.g., an improvement of 2%), and complete the training rounds.

[0066] In the embodiments provided by this invention, during the federated learning process, gradient data is desensitized by instruction sensitivity grading and differential privacy noise addition to prevent the cloud from back-inferring the user's original instructions; the cloud server aggregates gradients from multiple devices to achieve global knowledge sharing, which can effectively improve the model's generalization ability to fuzzy instructions (such as "I'm cold"); the local lightweight model collaborates with the cloud, which can effectively reduce the computational load of a single device while maintaining the continuous evolution of model performance.

[0067] In some embodiments, to improve the performance and training efficiency of the first lightweight large-scale language model, methods such as knowledge distillation, heterogeneous computation scheduling, model pruning, and quantization compression techniques can be introduced simultaneously during the federated training process. For example, knowledge distillation can be used to allow a higher-performance cloud-based large-scale language model (teacher model) to guide the student model (first lightweight large-scale language model) in learning, enabling the student model to maintain high prediction accuracy despite a reduction in the number of parameters. Heterogeneous computation scheduling can rationally allocate tasks from different computing resources (such as CPU, GPU, etc.) to accelerate the model training process. Ultimately, a lightweight large-scale language model with a significantly reduced number of parameters while maintaining high prediction accuracy is generated.

[0068] The following is combined Figure 4 The present invention describes the voice interaction system of the smart home device.

[0069] Figure 4 This is a structural schematic diagram of the voice interaction system for smart home devices provided by the present invention.

[0070] like Figure 4 As shown, the voice interaction system 400 of the smart home device includes a cloud server 410 and multiple home edge intelligent hubs: home edge intelligent hub 420, home edge intelligent hub 421, home edge intelligent hub 422, etc.

[0071] In the actual implementation process, the cloud server and multiple home edge smart hubs can perform the following interactive operations.

[0072] The cloud server performs the following multiple rounds of training to obtain the first lightweight large-scale language model after training: Obtain the gradient parameters of the initial lightweight large language model trained locally, sent by each family edge intelligence hub; based on the global model, obtain the gradient update amount according to the gradient parameters sent by all family edge intelligence hubs; send the gradient update amount to the corresponding family edge intelligence hub so that the family edge intelligence hub updates the gradient parameters of the locally trained lightweight large language model according to the gradient update amount.

[0073] The smart home device acquires voice commands, environmental perception data, and the current state of the smart home device; using an intent understanding engine, it parses the voice commands based on the environmental perception data to obtain the control intent for the smart home device; wherein, the intent understanding engine is implemented based on a first lightweight large language model; based on the control intent, combined with the state of the smart home device and the environmental perception data, it generates control commands for the smart home device; and based on the control commands, it executes the control tasks for the smart home device.

[0074] For details on the implementation methods and technical effects of the above interactive operations, please refer to [link / reference]. Figure 2 and Figure 3 The relevant content will not be repeated here.

[0075] The voice interaction device for smart home devices provided by the present invention is described below. The voice interaction device for smart home devices described below can be referred to in correspondence with the voice interaction method for smart home devices described above.

[0076] Figure 7 This is a structural schematic diagram of the voice interaction device for smart home devices provided by the present invention. Figure 7 As shown, the voice interaction device 700 of the smart home device includes the following modules.

[0077] The acquisition module 710 is used to acquire voice commands, environmental perception data, and the current status of smart home devices. The parsing module 720 is used to use an intent understanding engine to parse the voice command based on environmental perception data to obtain the control intent for smart home devices; wherein, the intent understanding engine is implemented based on a first lightweight large language model; The generation module 730 is used to generate control instructions for the smart home device based on the control intention and in combination with context information; wherein, the context information includes at least one of the current state of the smart home device, the environmental perception data, and stored user preferences; The execution module 740 is used to execute control tasks for the smart home device according to the control instructions.

[0078] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions stored in the memory 830 to execute a voice interaction method for the smart home device. This method includes: acquiring voice commands, environmental perception data, and the current state of the smart home device; using an intent understanding engine, parsing the voice commands based on the environmental perception data to obtain a control intent for the smart home device; wherein the intent understanding engine is implemented based on a first lightweight large-scale language model; generating control instructions for the smart home device based on the control intent and context information; wherein the context information includes at least one of the current state of the smart home device, the environmental perception data, and stored user preferences; and executing a control task for the smart home device according to the control instructions.

[0079] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the voice interaction method for smart home devices provided by the above methods, the method comprising: acquiring a voice command, environmental perception data, and the current state of the smart home device; using an intent understanding engine, parsing the voice command based on the environmental perception data to obtain a control intent for the smart home device; wherein the intent understanding engine is implemented based on a first lightweight large language model; generating a control instruction for the smart home device based on the control intent and in combination with context information; wherein the context information includes at least one of the current state of the smart home device, the environmental perception data, and stored user preferences; and executing a control task for the smart home device according to the control instruction.

[0081] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the voice interaction methods for the smart home devices provided above. The method includes: acquiring voice commands, environmental perception data, and the current state of the smart home device; using an intent understanding engine, parsing the voice commands based on the environmental perception data to obtain a control intent for the smart home device; wherein the intent understanding engine is implemented based on a first lightweight large language model; generating control instructions for the smart home device based on the control intent and context information; wherein the context information includes at least one of the current state of the smart home device, the environmental perception data, and stored user preferences; and executing a control task for the smart home device according to the control instructions.

[0082] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice interaction method for a smart home device, characterized in that, The method, applied to a home edge intelligence hub, includes: Acquire voice commands, environmental perception data, and the current status of smart home devices; Using an intent understanding engine, the voice command is parsed based on the environmental perception data to obtain the control intent for smart home devices; wherein, the intent understanding engine is implemented based on a first lightweight large language model; Based on the control intention and combined with context information, control instructions for the smart home device are generated; wherein, the context information includes at least one of the current state of the smart home device, the environmental perception data, and stored user preferences; According to the control instructions, the control tasks for the smart home devices are executed.

2. The voice interaction method for smart home devices according to claim 1, characterized in that, The method further includes: Retrieve the first lightweight large language model from the cloud server; The first lightweight large language model is deployed locally in the home edge intelligence hub to obtain the intent understanding engine.

3. The voice interaction method for smart home devices according to claim 1 or 2, characterized in that, The intent understanding engine was trained in the following way: Obtain a training dataset; wherein the training dataset includes labeled sample data, the sample data includes voice commands obtained locally from the home edge intelligent hub, and the labels are accurate control commands for smart home devices corresponding to the voice commands; Perform the following operations in each training round: The sample data is input into the first lightweight large language model to obtain the predicted value of the control command output by the first lightweight large language model. The gradient parameters of the first lightweight large language model are obtained based on the predicted value of the control command and the label corresponding to the sample data. The gradient parameters are sent to the cloud server so that the cloud server can process the locally generated gradient parameters sent by multiple home edge smart hubs to obtain the gradient update amount. The gradient update amount is obtained from the cloud server, and the parameters of the first lightweight large language model are updated.

4. The voice interaction method for smart home devices according to claim 3, characterized in that, The step of obtaining the gradient parameters of the first lightweight large language model based on the predicted value of the control command and the label corresponding to the sample data includes: The difference between the predicted value of the control command and the label corresponding to the sample data is determined using a loss function. The initial gradient parameters of the first lightweight large language model are obtained by taking the derivative of the loss function. The command sensitivity of the sample data is determined based on the voiceprint and emotion information carried by the sample data. Based on the instruction sensitivity of the sample data, noise is added to the initial gradient parameters to obtain the gradient parameters of the first lightweight large language model.

5. The voice interaction method for smart home devices according to claim 2, characterized in that, After using an intent understanding engine to parse the voice command based on the environmental perception data and obtain the control intent for smart home devices, the method further includes: If the confidence level of the control intention is determined to be less than a preset threshold, the voice command is desensitized and uploaded to the cloud server along with the environmental perception data. The cloud server then uses a large language model to parse the desensitized voice command based on the environmental perception data and re-obtain the control intention for the smart home device. Receive the control intention from the cloud server.

6. The voice interaction method for smart home devices according to claim 5, characterized in that, After receiving the control intention from the cloud server, the method further includes: Receive a second lightweight large language model from the cloud server and deploy it locally at the home edge intelligence hub; The second lightweight large language model is obtained by optimizing the global model corresponding to the first lightweight large language model using desensitized voice commands received from the edge intelligence hubs of each home by the cloud server.

7. A voice interaction device for a smart home device, characterized in that, The device, located at the edge of the home smart hub, includes: The acquisition module is used to acquire voice commands, environmental perception data, and the current status of smart home devices; The parsing module is used to use the intent understanding engine to parse the voice command based on the environmental perception data to obtain the control intent for the smart home device; wherein, the intent understanding engine is implemented based on a first lightweight large language model; The generation module is used to generate control commands for the smart home device based on the control intention and in combination with context information; wherein, the context information includes at least one of the current state of the smart home device, the environmental perception data, and stored user preferences; The execution module is used to perform control tasks for the smart home device according to the control instructions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the voice interaction method for the smart home device as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the voice interaction method for the smart home device as described in any one of claims 1 to 6.

10. A voice interaction system for a smart home device, characterized in that, Includes multiple home edge intelligence hubs and cloud servers; The cloud server performs the following multiple rounds of training to obtain the first lightweight large-scale language model after training: Obtain the gradient parameters of the initial lightweight large language model trained locally, sent by each family edge intelligence hub; Based on the global model, the gradient update amount is obtained according to the gradient parameters sent by all home edge intelligent hubs; The gradient update amount is sent to the corresponding home edge intelligence center so that the home edge intelligence center updates the gradient parameters of the locally trained lightweight large language model according to the gradient update amount. The smart home device acquires voice commands, environmental perception data, and the current status of the smart home device. Using an intent understanding engine, the voice command is parsed based on the environmental perception data to obtain the control intent for smart home devices; wherein, the intent understanding engine is implemented based on a first lightweight large language model; Based on the stated control intention, and combined with the status and environmental perception data of the smart home devices, control commands are generated for the smart home devices. According to the control instructions, the control tasks for the smart home devices are executed.

Images