Control method, device, equipment and system of Internet of Things equipment

Abstract

The invention discloses a control method and device of Internet of Things equipment, control equipment and a control system.The method is applied to the control equipment and comprises the steps that intention analysis is conducted on a natural language instruction through a large language model, intention information is generated, and the large language model is deployed on a local server or the control equipment; target Internet of Things equipment in a target area is matched according to the intention information, an equipment control instruction corresponding to the target Internet of Things equipment is generated, the target area comprises multiple pieces of Internet of Things equipment, and the multiple pieces of Internet of Things equipment comprise the target Internet of Things equipment; and sending the equipment control instruction to the target Internet of Things equipment so as to control the target Internet of Things equipment according to the natural language instruction.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular, to a control method and device for an Internet of Things device, a control device, and a control system. Background Art

[0002] In the early stage of the development of large models, the parameter scale was regarded as a key indicator to measure the model performance. The competition of large-scale models mainly focused on the comparison of the number of parameters, such as the competition between hundreds of billions of parameters and trillions of parameters. The industry generally believes that the model ability is positively correlated with the parameter scale, and the larger the parameters, the stronger the performance. However, with the in-depth research and technological evolution, many small-scale parameter model architectures have gradually shown excellent performance. The definition of "small models" has become more blurred accordingly. Currently, model design is developing towards the direction of lightweight, high efficiency, and customization, marking that the field of artificial intelligence is entering a new stage centered on resource optimization and performance balance.

[0003] In this context, applying AI to small or low-computing-power devices (such as smartphones, tablets, embedded sensors, cameras, smartwatches, etc.) faces huge technical challenges. The main difficulty lies in the high demand of AI algorithms for computing power and the limited knowledge storage and reasoning capabilities of devices, which often leads to insufficient performance in terms of professionalism, real-time performance, and user experience.

[0004] Current smart home control systems usually rely on cloud computing power for parsing and reasoning of user instructions. Although this architecture can utilize cloud computing resources to provide powerful computing capabilities, it inevitably brings problems such as privacy leakage, response delay, and network dependence. With the increasing demand of users for privacy protection and efficient real-time interaction, the limitations of this centralized processing method become more obvious.

[0005] In view of the problems in the related technology that relying on cloud computing power for parsing and reasoning of user instructions brings privacy leakage, response delay, and network dependence, etc., no effective solution has been proposed yet. Summary of the Invention

[0006] Embodiments of the present application provide a control method and device for an Internet of Things device, a control device, and a control system, so as to at least solve the problems in the related technology that relying on cloud computing power for parsing and reasoning of user instructions brings privacy leakage, response delay, and network dependence, etc.

[0007] According to an embodiment of the present application, a control method for an Internet of Things device is provided, which is applied to a control device and includes: parsing the intent of a natural language instruction through a large language model to generate intent information, wherein the large language model is deployed on a local server or the control device; matching a target Internet of Things device in a target area according to the intent information and generating a device control instruction corresponding to the target Internet of Things device, wherein the target area includes multiple Internet of Things devices, and the multiple Internet of Things devices include the target Internet of Things device; sending the device control instruction to the target Internet of Things device through a local area network protocol to control the target Internet of Things device according to the natural language instruction.

[0008] In an exemplary embodiment, the control device includes at least one of the following: a voice collection module, a text collection module, a visual collection module; before parsing the intent of the natural language instruction through the large language model to generate intent information, the method further includes: receiving voice information input by a target object through the voice collection module, or receiving text information input by the target object through the text collection module, or receiving image information input by the target object through the visual collection module; performing noise cancellation on the voice information through audio processing technology to obtain clarified voice information, or performing standardization processing on the text information through text processing technology to obtain standardized text information, or performing image recognition processing on the image information through visual processing technology to obtain semantic information corresponding to the image information; processing the clarified voice information or the standardized text information or the semantic information through a natural language processing module to obtain the natural language instruction.

[0009] In an exemplary embodiment, parsing the intent of the natural language instruction through the large language model to generate intent information includes: performing semantic analysis on the natural language instruction through the large language model to extract key information in the natural language instruction, wherein the key information includes at least one of the following: device type, device name, control action; generating multiple structured intent information according to the key information, wherein the intent information includes the multiple structured intent information.

[0010] In an exemplary embodiment, before matching the target Internet of Things device in the target area according to the intent information and generating a device control instruction corresponding to the target Internet of Things device, the method further includes: obtaining operation habit data of the target object for operating the multiple Internet of Things devices, wherein the operation habit data includes: operation behavior, preference setting; training a user preference model corresponding to the target object according to the operation habit data.

[0011] In an exemplary embodiment, matching a target Internet of Things device in a target area according to the intent information and generating a device control instruction corresponding to the target Internet of Things device includes: obtaining status data of the target object within a target time period corresponding to the natural language instruction; determining scene information corresponding to the natural language instruction according to the status data and the user preference model, and determining target intent information from the multiple pieces of structured intent information according to the scene information; determining the target Internet of Things device according to the target intent information, and generating the device control instruction.

[0012] In an exemplary embodiment, matching a target Internet of Things device in a target area according to the intent information and generating a device control instruction corresponding to the target Internet of Things device includes: performing precise matching in the device information of the multiple Internet of Things devices according to the intent information; generating the device control instruction when the target Internet of Things device is matched; and when the target Internet of Things device is not matched, performing fuzzy matching in the multiple pieces of device information according to the intent information to match to the target Internet of Things device, and generating the device control instruction.

[0013] In an exemplary embodiment, performing precise matching in the device information of the multiple Internet of Things devices according to the intent information includes: calculating the similarity between the intent information and the multiple pieces of device information in sequence to obtain multiple similarity scores; sorting the multiple Internet of Things devices according to the multiple similarity scores, and determining whether a target similarity score corresponding to the Internet of Things device ranked first is greater than a preset threshold; when the target similarity score is greater than or equal to the preset threshold, determining the Internet of Things device corresponding to the target similarity score as the target Internet of Things device; and when the target similarity score is less than the preset threshold, determining that the target Internet of Things device is not matched.

[0014] In an exemplary embodiment, matching a target Internet of Things device in a target area according to the intent information and generating a device control instruction corresponding to the target Internet of Things device includes: extracting factual information from the natural language instruction, where the factual information is structured data of the intent information; updating a fact library according to the status information of the multiple Internet of Things devices at the current moment to obtain an updated fact library; performing rule inference according to the factual information and the updated fact library through a rule engine, and determining the target Internet of Things device according to the inference result, and generating the device control instruction.

[0015] In an exemplary embodiment, the method further includes: obtaining cloud configuration data from a cloud server at a preset period, where the cloud configuration data includes: device configuration information, rule data; updating local configuration data according to the cloud configuration data, and instructing a rule engine to regenerate a plurality of rules according to the updated local configuration data, where the local configuration data is stored on the local server or the control device.

[0016] According to another embodiment of the embodiments of the present application, there is also provided a control device for an Internet of Things device, including: a parsing module, configured to perform intent parsing on a natural language instruction through a large language model to generate intent information, where the large language model is deployed on the local server or the control device; a generating module, configured to match a target Internet of Things device in a target area according to the intent information and generate a device control instruction corresponding to the target Internet of Things device, where the target area includes a plurality of Internet of Things devices, and the plurality of Internet of Things devices includes the target Internet of Things device; a sending module, configured to send the device control instruction to the target Internet of Things device through a local area network protocol to control the target Internet of Things device according to the natural language instruction.

[0017] According to yet another embodiment of the embodiments of the present application, there is also provided a control device, including: the above-mentioned control device for an Internet of Things device; a human-computer interaction unit connected to the control device, configured to receive a natural language instruction input by a target object; an AI acceleration unit connected to the control device, configured to improve the processing efficiency of the parsing module of the control device; a communication unit connected to the control device, configured to perform information interaction with the plurality of Internet of Things devices.

[0018] According to yet another embodiment of the embodiments of the present application, there is also provided a control system for an Internet of Things device, including: a control device, configured to perform intent parsing on a natural language instruction through a large language model to generate intent information, where the large language model is deployed on the local server or the control device; matching a target Internet of Things device in a target area according to the intent information and generating a device control instruction corresponding to the target Internet of Things device, where the target area includes a plurality of Internet of Things devices, and the plurality of Internet of Things devices includes the target Internet of Things device; sending the device control instruction to the target Internet of Things device through a local area network protocol to control the target Internet of Things device according to the natural language instruction; the target Internet of Things device, configured to receive the device control instruction through the local area network protocol and execute the device control instruction.

[0019] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the above method when running.

[0020] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the above processor executes the above method through the computer program.

[0021] In the embodiments of the present application, the large language model parses the received natural language instruction to generate intent information, wherein the large language model is deployed on a local server or the control device; then, according to the intent information, the target Internet of Things device in the target area is matched, and a device control instruction corresponding to the target Internet of Things device is generated, wherein there are multiple Internet of Things devices in the target area, and the multiple Internet of Things devices include the target Internet of Things device; finally, the device control instruction is sent to the target Internet of Things device, so as to control the target Internet of Things device according to the natural language instruction of the target object; adopting the above solution, the edge LLM large model is deployed using local hardware resources (local server or control device), and the user instruction parsing and reasoning work are completed locally. While quickly inferring and generating control instructions, the user's data privacy is ensured; thus, the problems of privacy leakage, response delay, and network dependence caused by relying on cloud computing power for user instruction parsing and reasoning in the related art are solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0023] Figure 1 is a hardware structure block diagram of a control device for a control method of an Internet of Things device according to an embodiment of the present application;

[0024] Figure 2 is a flowchart of a control method of an Internet of Things device according to an embodiment of the present application;

[0025] Figure 3 is a schematic structural diagram of an optional intelligent Internet of Things device control device based on an edge LLM according to an embodiment of the present application;

[0026] Figure 4 is a schematic flow diagram of an optional intelligent Internet of Things device control method based on an edge LLM according to an embodiment of the present application;

[0027] Figure 5It is a structural block diagram of a control device for an Internet of Things device according to an embodiment of the present application;

[0028] Figure 6 It is a structural block diagram of a control device according to an embodiment of the present application;

[0029] Figure 7 It is a structural block diagram of a control system for an Internet of Things device according to an embodiment of the present application;

[0030] Figure 8 It is a network topology diagram (one) of a control system for an Internet of Things device according to an embodiment of the present application;

[0031] Figure 9 It is a network topology diagram (two) of a control system for an Internet of Things device according to an embodiment of the present application. Detailed implementation manners

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a control device or a similar computing device. Taking running on a control device as an example, Figure 1 It is a hardware structural block diagram of a control device for a control method of an Internet of Things device according to an embodiment of the present application. As Figure 1 shown, the control device may include one or more ( Figure 1Only one processor 102 is shown (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a field-programmable gate array FPGA), and a memory 104 for storing data. In one exemplary embodiment, the control device may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only illustrative and does not limit the structure of the above control device. For example, the control device may further include more or fewer components than those Figure 1 shown in, or have an equivalent function to those Figure 1 shown in or different configurations with more functions than those Figure 1 shown in.

[0035] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the control device through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by the communication provider of the control device. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] In this embodiment, a control method for an Internet of Things device is provided, which is applied to a control device. Figure 2 is a flowchart of the control method for an Internet of Things device according to an embodiment of the present application. The process includes the following steps:

[0038] Step S202, perform intent parsing on a natural language instruction through a large language model to generate intent information, where the large language model is deployed on a local server or the control device;

[0039] It should be noted that the above control device can be a central control screen, or a device such as a computer or a mobile phone. This application does not limit this.

[0040] Step S204: Match the target Internet of Things device in the target area according to the intention information, and generate a device control instruction corresponding to the target Internet of Things device, where the target area includes multiple Internet of Things devices, and the multiple Internet of Things devices include the target Internet of Things device;

[0041] Step S206: Send the device control instruction to the target Internet of Things device through the local area network protocol to control the target Internet of Things device according to the natural language instruction.

[0042] Through the above steps, the large language model is used to perform intention parsing on the received natural language instruction to generate intention information, where the large language model is deployed in the local server or the control device; then, the target Internet of Things device in the target area is matched according to the intention information, and a device control instruction corresponding to the target Internet of Things device is generated, where there are multiple Internet of Things devices in the target area, and the multiple Internet of Things devices include the target Internet of Things device; finally, the device control instruction is sent to the target Internet of Things device, so as to control the target Internet of Things device according to the natural language instruction of the target object; by adopting the above solution, the edge LLM large model is deployed using local hardware resources (local server or control device), and the user instruction parsing and reasoning work are completed locally. While quickly inferring and generating control instructions, the user's data privacy is guaranteed; thus, the problems of privacy leakage, response delay, and network dependence caused by relying on cloud computing power for user instruction parsing and reasoning in the related technology are solved.

[0043] In an optional embodiment, the control device includes at least one of the following: a voice acquisition module, a text acquisition module, and a visual acquisition module; before the large language model performs intention parsing on the natural language instruction to generate intention information, the method further includes: receiving the voice information input by the target object through the voice acquisition module, or receiving the text information input by the target object through the text acquisition module, or receiving the image information input by the target object through the visual acquisition module; performing noise elimination on the voice information through audio processing technology to obtain the clarified voice information, or performing standardization processing on the text information through text processing technology to obtain the standardized text information, or performing image recognition processing on the image information through visual processing technology to obtain the semantic information corresponding to the image information; processing the clarified voice information or the standardized text information or the semantic information through a natural language processing module to obtain the natural language instruction.

[0044] This embodiment details how a smart home control device receives user instructions through various input methods and uses advanced processing technologies to convert these inputs into machine - understandable natural language instructions for further intent parsing by a large - language model and controlling the device. The following is the specific implementation process of this embodiment:

[0045] 1. Multi - modal input reception:

[0046] The control device is equipped with multiple input acquisition modules to support different user interaction methods:

[0047] 1) Voice acquisition module: Such as a built - in microphone, used to capture the user's voice instructions.

[0048] The control device is equipped with a high - sensitivity microphone (voice acquisition module) to capture the voice signals of the user (target object). The design and positioning of the microphone consider the room acoustic environment to ensure that voice instructions issued from different directions and distances can be accurately captured.

[0049] 2) Text acquisition module: Such as touch - screen input or an externally connected keyboard, used to receive the text instructions input by the user.

[0050] 3) Visual acquisition module: Such as a camera, used to capture the user's gestures, facial expressions, or image inputs, which is particularly important in contactless or intuitive control scenarios.

[0051] 2. Pre - processing of input data:

[0052] The received raw data needs to be pre - processed to eliminate noise, standardize the format, or extract key information. This step is the basis for ensuring that the large - language model can accurately understand the user's intent.

[0053] 1) Audio processing: Use audio processing technologies, such as noise suppression algorithms, to remove background noise from the captured voice information and obtain clear and high - quality voice input.

[0054] The received voice information is first pre - processed through audio processing technologies, with a focus on noise elimination for the voice information. Noise elimination technologies use signal processing algorithms, such as spectral masking, adaptive filtering, etc., to remove background noise, noise, and other interfering sounds, ensuring the clarity and intelligibility of voice instructions. This process is an important step in ensuring the accuracy of subsequent speech recognition and natural language understanding.

[0055] 2) Text processing: Standardize the text information input by the user, including spelling correction, grammar correction, and format adjustment, to ensure the normativity and consistency of the text.

[0056] 3) Visual processing: Through image recognition technology, semantic information is extracted from the image information collected visually, such as the meaning of gestures, the interpretation of facial expressions, or the recognition of specific objects.

[0057] 3. Natural language instruction generation:

[0058] The processed voice information, text information, or image semantic information is then passed to the Natural Language Processing (NLP) module. The NLP module uses techniques such as semantic analysis, entity recognition, and syntactic analysis to deeply understand the meaning of the user's instructions, identifying entities (such as device names) and actions (such as "turn on", "turn off") in them. This step is crucial for converting the user's spoken instructions into operations executable by the system.

[0059] 4. Structured output and control instruction generation:

[0060] The natural language instructions parsed by the NLP module are converted into a structured data format, such as JSON or XML, which contains information such as device type, device name, and specific operations. This structured output not only facilitates the understanding of subsequent decision-making modules but also facilitates the generation of precise smart home control instructions. For example, when the user says "turn off the lights in the living room", the structured output may be {"deviceType":"light","deviceName":"livingRoomLight","action":"off"}, such a format clearly indicates the device to be operated and the action to be performed.

[0061] Through the above steps, the smart home control device can effectively receive and parse natural language instructions, ensuring the real-time response and high accuracy of the system. At the same time, the local processing method reduces the dependence on the network and enhances the privacy protection of user data, providing a more intelligent and secure control solution for smart home applications. This embodiment demonstrates the application potential of edge computing in the smart home scenario, pushing the computing and data processing capabilities closer to the edge of users and devices, thereby optimizing the overall performance and user experience of the system.

[0062] In an exemplary embodiment, the natural language instructions are parsed for intent by a large language model to generate intent information, including: performing semantic analysis on the natural language instructions by the large language model to extract key information in the natural language instructions, where the key information includes at least one of the following: device type, device name, control action; generating multiple structured intent information based on the key information, where the intent information includes the multiple structured intent information.

[0063] This embodiment describes how to use a large language model (LLM) deployed at the edge to deeply understand the natural language instructions received by the smart home system and convert them into specific and structured intent information to drive the precise control of Internet of Things devices. The following is a detailed description of this embodiment:

[0064] 1. Semantic analysis through the large language model:

[0065] When the user issues a control instruction to the smart home control device in the form of voice or text, such as "Turn on the lights in the living room", the large language model (LLM) plays a key role in understanding and parsing this instruction. Through in-depth semantic analysis, the LLM can understand the meaning of the user's instruction, including the implicit intent and specific details in the instruction.

[0066] 2. Extraction of key information:

[0067] After semantic analysis, the LLM extracts the key information in the instruction, which is an important basis for the smart home system to perform control operations. The key information specifically includes, but is not limited to:

[0068] 1) Device type: Identify the types of Internet of Things devices mentioned by the user, such as "lights", "air conditioners", "curtains", etc.

[0069] 2) Device name: Determine the unique identifier of the specific device, such as "lights in the living room", "master bedroom air conditioner", etc., which helps the system accurately locate the target device.

[0070] 3) Control action: Identify the specific operation that the user wants to perform on the device, such as "turn on", "turn off", "increase brightness", etc.

[0071] 3. Generation of structured intent information:

[0072] Based on the key information extracted from the user's instruction, the LLM generates structured intent information, which is a clear and standardized form of instruction expression, consisting of multiple fields and facilitating further processing by the smart home system. Each piece of structured intent information usually contains fields such as device type, device name, and control action, and is encoded in JSON, XML, or other standardized data formats.

[0073] For example, for the instruction "Dim the lights in the bedroom", the LLM may generate the following structured intent information:

[0074] json{"deviceType":"light","deviceName":"bedroomLight","action":"dim"}

[0075] For more complex instructions such as "Get ready to watch a movie", the LLM may parse multiple control actions, with each action corresponding to a structured intent message:

[0076] json[{"deviceType":"curtain","deviceName":"livingRoomCurtain","action":"close"},{"deviceType":"light","deviceName":"livingRoomLight","action":"dim"},{"deviceType":"TV","deviceName":"livingRoomTV","action":"playMovie"}]

[0077] These structured intent messages are then passed to the decision-making layer module, which, based on locally stored rules, algorithms, and device metadata, performs rapid reasoning and decision-making to generate control instructions for specific devices and sends them via the communication module to the corresponding intelligent devices for execution.

[0078] This embodiment realizes the local processing of users' natural language instructions by deploying the LLM on edge devices, which not only improves the response speed and control accuracy of the smart home system, but also enhances the privacy protection of user data, reduces the dependence on cloud resources, indicating that the smart home control technology is developing towards a more intelligent, efficient, and secure direction.

[0079] It should be noted that the above key information may also include parameters such as room, scenario, automation, etc., and may also include other parameters, which are not limited in this application.

[0080] Optionally, before matching the target IoT device in the target area according to the intent information and generating the device control instruction corresponding to the target IoT device, the method further includes: obtaining the operation habit data of the target object for operating the multiple IoT devices, where the operation habit data includes: operation behavior, preference settings; training a user preference model corresponding to the target object according to the operation habit data.

[0081] This embodiment highlights how the smart home system generates more personalized and intelligent device control instructions by learning the user's behavior patterns and preferences to improve the user's quality of life and usage experience. The following is a detailed description of this embodiment:

[0082] 1. Obtain user operation habit data:

[0083] The smart home system continuously collects and records the user's operation behaviors and preference settings in daily life. These data are sourced from the user's frequent interactions with smart Internet of Things devices. Operation behavior data may include the user's operations on devices at different times and in different scenarios, such as turning lights on and off, adjusting the temperature, setting the alarm, etc. Preference setting data covers the user's personal preferences in device usage, such as the preference for light brightness and color, the set range of air conditioner temperature, or the selection of music playlists, etc. The system comprehensively collects the user's operation habit data through built-in sensors, interaction records of the user interface, and the usage status information of the devices.

[0084] 2. Train the personalized user preference model:

[0085] The collected operation habit data is used to train the personalized user preference model. This model is built based on machine learning technology and can learn and understand the user's behavior patterns and preference trends. The training process may include the following steps:

[0086] 1) Data preprocessing: Clean and organize the operation habit data to ensure the quality and consistency of the data.

[0087] 2) Feature extraction: Extract key features from operation behaviors and preference settings, such as operation time, device type, operation type, user feedback, etc., for model training.

[0088] 3) Model selection and training: Adopt appropriate machine learning algorithms (such as deep learning, clustering analysis, time series analysis, etc.) to train the extracted features and generate a model that can predict the user's behavior and preferences. The training goal is to enable the model to predict the user's device usage preferences in a specific scenario based on the user's historical operation habits.

[0089] 3. Match the target area and generate personalized control instructions:

[0090] When the system receives the user's natural language instruction, it first parses the user's intention through a large language model (LLM) to generate intention information. Subsequently, the system will call the user's personalized preference model, combine the current environmental information (such as time, weather, device status, etc.), intelligently match the Internet of Things devices in the target area, and generate control instructions that meet the user's expectations.

[0091] For example, assume that the user is used to dimming the lights in the bedroom, closing the curtains, and adjusting the air conditioner to a lower temperature before going to bed. When the user simply says "getting ready to sleep", the smart home system can not only understand the basic intention (turning off the lights, curtains, and air conditioner in the bedroom), but also automatically apply the specific preference settings stored in the user preference model, such as adjusting the lights to the user-preferred brightness level and setting the air conditioner temperature to the interval that the user usually sets, so as to create an optimal pre-sleep environment.

[0092] Through this embodiment, the smart home system can not only accurately understand the user's immediate needs, but also provide more intelligent, personalized and considerate services according to the user's historical behaviors and preference settings, creating a more comfortable and convenient home environment for the user. This strategy not only improves user satisfaction, but also demonstrates the great potential of smart home technology in personalized services and intelligent control.

[0093] In an exemplary embodiment, matching the target Internet of Things device in the target area according to the intent information and generating a device control instruction corresponding to the target Internet of Things device includes: obtaining status data of the target object within a target time period corresponding to the natural language instruction; determining scenario information corresponding to the natural language instruction according to the status data and the user preference model, and determining target intent information from the multiple pieces of structured intent information according to the scenario information; determining the target Internet of Things device according to the target intent information and generating the device control instruction.

[0094] This embodiment details how the smart home system intelligently matches Internet of Things devices in the target area and generates accurate device control instructions according to the user's natural language instructions, combined with the current device status and user preferences, to achieve personalized control of the home environment. The following is a step-by-step analysis of the embodiment:

[0095] 1. Obtain device status data:

[0096] When the user issues a natural language instruction, the smart home system first needs to obtain the current status data of all relevant Internet of Things devices in the target area (such as the bedroom, living room). These status data include but are not limited to: whether the device is turned on, the working mode of the device, the set parameters of the device (such as temperature, brightness, volume, etc.). The system obtains comprehensive device status information by communicating with each device in real time or reading from the locally stored status information.

[0097] 2. Determine scenario information and target intent:

[0098] Subsequently, the system will determine the scenario information corresponding to the user instruction according to the collected device status data and in combination with the user preference model. The user preference model contains the user's preference settings for the home environment at different time periods. For example, the user may prefer dimmer light at night and bright environment in the morning. Combining this information, the system can identify the context and potential needs of the user instruction, and judge the possible scenarios the user may be in, such as "going to bed", "getting up in the morning", "watching a movie", etc. This process may involve the analysis of multi-dimensional data such as device status, time, and user historical behaviors to accurately infer the user's intent.

[0099] For example, when a user issues a "get ready for bed" command at night, the system not only understands that the user wants to turn off the lights, but also considers that the user may want to adjust the air conditioner to a comfortable temperature, close the curtains, etc. These are all scenario requirements inferred based on the user preference model and the current device status.

[0100] 3. Match the target IoT device and generate control instructions:

[0101] Once the system determines the scenario information of the user's command, the next step is to match the scenario information with multiple structured intent information to find the target intent information. Since LLM has previously generated multiple structured intent information, the system now needs to filter out the intent information that is most relevant to the current scenario. This screening process relies on strategies such as rule engines, algorithm recommendations, and similarity matching, combining scenario information and user preferences to intelligently select the most appropriate control solution.

[0102] For example, the command "get ready for bed" may correspond to multiple intent information, including adjusting lights, setting temperature, operating curtains, etc. The system will determine the target intent information that best meets the user's expectations based on the settings for bedtime scenes in the user preference model, such as turning off the bedroom lights, adjusting the air conditioner to 25 degrees, closing the curtains, etc.

[0103] Finally, the system determines the target IoT device based on the selected target intent information and generates corresponding device control instructions. These instructions are sent to the target device through the communication module, triggering the device to respond accordingly, such as turning off the lights, adjusting the temperature, operating the curtains, etc., thereby realizing personalized control of the smart home environment.

[0104] This embodiment combines device status data, user preference models and scenario information to enable the smart home system to understand and respond to user needs more intelligently and accurately, providing a highly personalized and intelligent home control experience, and reflecting the advancement and superiority of smart home technology in understanding and responding to user needs.

[0105] Based on the above steps, the target IoT device in the target area is matched according to the intent information, and a device control instruction corresponding to the target IoT device is generated, including: performing precise matching in the device information of the multiple IoT devices according to the intent information; generating the device control instruction when the target IoT device is matched; and performing fuzzy matching in the multiple device information according to the intent information to match the target IoT device and generate the device control instruction when the target IoT device is not matched.

[0106] This embodiment details how a smart home control system determines target Internet of Things (IoT) devices based on natural language instructions issued by users, through precise matching and fuzzy matching mechanisms, and generates corresponding control instructions to achieve intelligent execution of user intentions. The following is the process analysis of this embodiment:

[0107] I. Precise matching mechanism:

[0108] 1. Intent information parsing: After the edge LLM module receives a user's natural language instruction, it performs semantic parsing on it to generate structured intent information. This information usually contains key elements such as specific operation targets (e.g., "the lights in the living room") and operation types (e.g., "turn off").

[0109] 2. Device information retrieval: Based on the parsed intent information, the system retrieves in the local database to find IoT devices that exactly match the intent information. Here, the matching includes device name, type, and current status, etc.

[0110] 3. Generate device control instructions: Once a device that exactly matches the intent information is found in the database, the system generates specific device control instructions, such as "turn off the lights in the living room", and immediately sends them to the target device through the communication module to execute the user's natural language instruction.

[0111] II. Fuzzy matching mechanism:

[0112] When the precise matching mechanism fails to find the target device, it may be because the user's instruction description is not precise enough, or there is no entry in the device information library that exactly matches the instruction. At this time, the system will start the fuzzy matching mechanism to determine the device referred to in the user's intention.

[0113] 1. Intent analysis and device attribute comparison: The system analyzes the keywords and context in the intent information, and then compares them with the device attributes in the local database. For example, the system may identify the action of "dimming" and then search for all lighting devices with dimming functions.

[0114] 2. Fuzzy matching of device names: Based on the device attribute comparison, the system attempts to lock the target device through fuzzy matching of device names. This may include prefix, suffix matching of device names, or matching based on device categories (e.g., "all devices in the bedroom").

[0115] 3. Consideration of context information and user preferences: The system will also comprehensively consider the current environmental context (such as time, user's location) and user operation habit data to infer the most likely target device. For example, if the user is used to dimming the lights in the bedroom at night, the system will give priority to matching the lighting devices in the bedroom.

[0116] 4. Generate control instructions: After determining the target IoT device through fuzzy matching, the system generates specific control instructions according to the device control protocol and sends them to the target device for execution. For example, if the user issues an instruction "Dim the lights", the system may send a control instruction "Lower the brightness of the bedroom lights to 50%".

[0117] Comprehensive benefits of this embodiment: By combining the precise matching and fuzzy matching mechanisms, the understanding and execution ability of the smart home control system for user natural language instructions are significantly improved. It can handle both clear and specific instructions and fuzzy and generalized descriptions, thus enhancing the intelligence and user-friendliness of the system. This mechanism avoids over-reliance on cloud services, improves the system's response speed and data security, and is a key step in achieving highly personalized control and enhancing the user experience in smart home technology.

[0118] Through this embodiment, the smart home system can more accurately identify and respond to user needs, reduce the user's input burden, and at the same time provide a more considerate and intelligent home environment control service by dynamically learning the user's preferences and habits.

[0119] Optionally, the precise matching in the device information of the multiple IoT devices according to the intention information includes: calculating the similarity between the intention information and the device information of the multiple devices in sequence to obtain multiple similarity scores; sorting the multiple IoT devices according to the multiple similarity scores, and determining whether the target similarity score corresponding to the IoT device ranked first is greater than a preset threshold; in the case where the target similarity score is greater than or equal to the preset threshold, determining the IoT device corresponding to the target similarity score as the target IoT device; in the case where the target similarity score is less than the preset threshold, determining that the target IoT device is not matched.

[0120] This embodiment relates to the intelligent device matching mechanism of the smart home control system. By quantifying the similarity between user intentions and device information, it intelligently determines the specific device that the user wants to control, so as to achieve accurate and efficient execution of natural language instructions. The following is an analysis of the detailed steps:

[0121] 1. Intention information parsing and similarity calculation:

[0122] 1) Intention information extraction: When the user inputs a natural language instruction, such as "Turn off the lights in the study", through voice or text, the system first uses the edge LLM module to parse the instruction and extract key information, such as device type (lights), device location (study), and operation action (turn off).

[0123] 2) Device Information Retrieval: The system retrieves information on all IoT devices from the local device information library, including device name, type, location, current status, etc. This information is used for subsequent matching and similarity calculation.

[0124] 3) Similarity Calculation: For each IoT device, the system calculates the similarity between its information and the intent information. This can be achieved through various algorithms. For example, the cosine similarity between the device name and the intent description is calculated using a word vector model, or natural language processing techniques are used to evaluate the consistency between the device type and the operation action and the user's intent. Through these calculations, the system generates a similarity score for each device.

[0125] 2. Sorting of IoT Devices and Confirmation of Target Devices (i.e., the above-mentioned target IoT devices):

[0126] 1) Device Sorting: The system sorts all IoT devices according to the calculated similarity scores, usually in descending order of similarity.

[0127] 2) Threshold Judgment: A similarity threshold is set as the standard for device matching. This threshold can be automatically adjusted by the system based on previous matching experiences or can be a pre-configured fixed value to determine whether a device sufficiently matches the user's intent.

[0128] 3) Determination of Target Device: The system checks whether the IoT device with the highest similarity score in the sorted list is greater than or equal to the preset similarity threshold. If the similarity score meets the condition, then this device is regarded as the target IoT device in the user's intent, and the system can generate and execute control instructions accordingly. For example, for the instruction "Turn off the lights in the study", if the similarity score of the "Study Lights" device exceeds the threshold, then the system will generate the control instruction "Turn off the study lights".

[0129] 4) Handling of Unmatched Cases: If the similarity of the IoT device with the highest similarity score is lower than the preset threshold, the system determines that no matching target device has been found. In this case, the system may need to request the user to further clarify the instruction or perform a fuzzy match based on a more extensive device information library to try to find the device that the user may want to control.

[0130] Through this embodiment, the smart home control system can accurately recognize the user's intent after the user inputs a natural language instruction, intelligently match the target device, and generate corresponding control instructions, thereby providing a smooth, efficient, and personalized home control experience. At the same time, by setting the similarity threshold, the system can avoid misoperations and improve the accuracy and security of control. This mechanism reflects the intelligence and flexibility of smart home technology in handling complex user requirements and is one of the key technologies for improving the level of home intelligence.

[0131] Optionally, match the target Internet of Things device in the target area according to the intention information, and generate a device control instruction corresponding to the target Internet of Things device, including: extracting factual information from the natural language instruction, where the factual information is the structured data of the intention information; updating the fact library according to the status information of the multiple Internet of Things devices at the current moment to obtain an updated fact library; performing rule reasoning according to the factual information and the updated fact library through a rule engine, and determining the target Internet of Things device according to the reasoning result, and generating the device control instruction.

[0132] This embodiment describes the process of how a smart home control system matches intention information, identifies devices, and generates control instructions through a local rule engine based on a user's natural language instruction. This process covers the reception, parsing, reasoning of user instructions to the generation of final control instructions, ensuring that the system can accurately and efficiently respond to user needs. The following are the detailed steps of this embodiment:

[0133] 1. Reception and parsing of user instructions: The user sends a natural language instruction to the system through voice, text, or other input methods, such as: "Please turn on the lights in the dining room". This instruction is first received through a voice collection module, a text collection module, or a visual collection module, and then pre-processed through audio processing, text processing, or visual processing technologies to eliminate noise, standardize the format, or extract semantic information from images. The pre-processed information is sent to the natural language processing module for further parsing into structured factual information, which specifically describes the key elements of the user's intention, such as device type (lights), device location (dining room), and operation action (turn on).

[0134] 2. Update of the fact library: The fact library is the basis for the rule engine to make decisions, and it contains the status information of all Internet of Things devices in the current system. To ensure the accuracy of decisions, the system needs to update the fact library in real time or regularly. After the user instruction is parsed into factual information, the system checks the status of the corresponding device, such as whether it is already on, the current brightness, etc., and combines other device states (such as whether there are other activities in the room) to update the relevant entries in the fact library to reflect the latest device and environmental states.

[0135] 3. Rule Engine Inference and Device Matching: The updated fact library and the parsed user intent (factual information) are sent to the local rule engine. The rule engine infers these information based on a preset rule set, aiming to determine the device that the user specifically wants to control (i.e., the target IoT device) and the corresponding operation instructions. The inference process includes: Condition Matching: Search for rules in the rule library that match the user intent and device status, such as "If the user intent is to turn on the lights and the target area is the dining room, then control the lighting device in the dining room." Device Identification: Based on the successfully matched rules, identify the target IoT device in the target area, such as "the lighting device in the dining room." Instruction Generation: Based on the device identification result, generate specific device control instructions, such as "Turn on the lighting device in the dining room and set the brightness to 50%."

[0136] 4. Generation and Execution of Control Instructions: The control instructions inferred by the rule engine are sent to the smart home device communication module, which is responsible for converting the instructions into a specific communication protocol that the device can understand, such as Zigbee, BLE Mesh, etc. The control instructions are then sent to the target IoT device through the local network to perform the corresponding operations. The execution result will be fed back to the system to update the fact library, forming a closed-loop control process.

[0137] Through this mechanism, the smart home control system can independently complete the understanding of user natural language instructions, monitoring of device status, identification and control of devices locally, reducing the dependence on cloud services, improving data security and privacy protection, and at the same time enhancing the system's response speed and efficiency. This decision-making scheme based on the local rule engine is an important part of realizing natural language control and intelligent device management in smart home technology, and can provide a more personalized and intelligent home control experience.

[0138] Optionally, the method further includes: obtaining cloud configuration data from the cloud server at a preset period, where the cloud configuration data includes: device configuration information, rule data; updating the local configuration data according to the cloud configuration data, and instructing the rule engine to regenerate multiple rules according to the updated local configuration data, where the local configuration data is stored on the local server or the control device.

[0139] This embodiment describes how the smart home control system regularly obtains configuration data from the cloud server, including device configuration information and rule data, and synchronizes these data to the local area to update the decision-making basis of the local rule engine, ensuring that the system can adapt to environmental changes and user demand updates. The following are the detailed steps of this embodiment:

[0140] 1. Maintenance of Cloud Configuration Data: A complete set of configuration data is maintained on the cloud server, including device configuration information and rule data. The device configuration information contains detailed attributes of all IoT devices in the system, such as device type, device ID, control protocol, device status, etc. The rule data is a series of rules designed based on user preferences, historical operations, and intelligent scenarios, which are used to guide the rule engine on how to parse user instructions and infer specific device control actions.

[0141] 2. Data Pulling at Preset Intervals: The control device (such as a central control screen) in the smart home control system is set with a predefined interval, such as daily, weekly, or a custom time interval. At these time points, the control device will actively establish a communication connection with the cloud server to request the latest cloud configuration data. This process is usually achieved through a secure network protocol (such as HTTPS) to ensure the security of data transmission.

[0142] 3. Update of Local Configuration Data: After receiving the configuration data from the cloud, the control device will compare the data with the local configuration data to identify new device information, modified device attributes, or updated rule data. Based on these differences, the control device will update the local configuration data to reflect the latest system status and user rules. The local configuration data can be stored in a local server or the internal storage of the control device to ensure that the system can still operate without a network connection.

[0143] 4. Rule Update of the Rule Engine: With the update of the local configuration data, the rule engine needs to reload and parse the updated rule data so as to apply the latest rules in subsequent decision-making processes. This process may include the addition, modification, or deletion of rules to ensure that the rule engine can adapt to changes in the system and user preferences and provide more accurate and personalized device control instructions.

[0144] 5. Rule Application and Device Control: The updated rule engine performs more accurate intent parsing and inference on the user's natural language instructions based on the latest local configuration data. For example, if the user says "Get ready to watch a movie", the rule engine will, according to the updated rules, identify the "home theater mode" and generate a series of control instructions, such as turning on the TV, the stereo, dimming the lights, closing the curtains, etc., to create a suitable viewing environment.

[0145] Through this embodiment, the smart home system can maintain synchronization with the cloud, and even when running locally, it can obtain the latest device configuration and user rule information, enhancing the adaptability and intelligent control ability of the system. At the same time, it also reduces the risk of control failure caused by network latency or interruption. This regular cloud data synchronization and local rule update mechanism is an important step in realizing continuous optimization and personalized services in smart home technology.

[0146] Optionally, the above control device may be a central control screen. An embodiment of the present application provides a smart home control device based on an edge LLM, such as Figure 3 shown. The device includes a central control screen hardware, an edge LLM module, a decision-making layer module, and a smart Internet of Things device communication module.

[0147] Among them, the central control screen hardware includes a low-power processor, a memory, an edge acceleration unit (such as an NPU dedicated hardware processing unit, also known as a neural network processor, and a DSP digital signal processing unit), a display screen, and a local storage device.

[0148] The edge LLM module runs on the central control screen and is used to perform intent recognition and parsing on the natural language control instructions input by the user.

[0149] The decision-making layer module runs on the central control screen. Using technologies such as rule engines, algorithm recommendations, and similarity matching, it matches the user intent output by the model with home devices and scenario information, and infers specific control instructions.

[0150] The smart Internet of Things device communication module is used to perform local communication with other controlled devices and supports multiple communication protocols (such as Zigbee, Ble Mesh, OpenThread, Wi-Fi, etc.).

[0151] Based on the above smart home control device, the present application also provides a smart Internet of Things device control method based on an edge LLM, such as Figure 4 shown. The smart Internet of Things device control method is applied to the Figure 3 smart home control device shown. The method includes the following steps:

[0152] 1. Receive the natural language instruction input by the user;

[0153] 2. Use the large language model deployed at the edge to parse the user intent and generate structured intent information;

[0154] 3. Based on the local rule engine or similarity matching or recommendation algorithm, match the user's home devices and scenario information, and infer specific device control instructions;

[0155] 4. Send the control instruction to the target device for execution through the communication module.

[0156] The home metadata points to the local edge LLM large model, which means providing data for the LLM large model to train. After training, pruning and other operations are also required to complete a sufficiently small LLM large language model that can run on the local edge device.

[0157] The smart home control instruction points to the large language model (LLM) through the Natural Language Understanding (NLU), which means parsing the natural language instruction input by the user using the natural language understanding solution and inputting the parsed instruction into the LLM. The LLM parses the natural language intention. Among them, the NLU module can be integrated into the LLM.

[0158] Optionally, the central control screen hardware may include a voice collection module such as a microphone, which is used to collect the voice information input by the user after the device is awakened, and convert the voice information into a natural language instruction through methods such as NLU natural language processing, and then input it into the edge LLM module.

[0159] Optionally, the step of "using the large language model deployed at the edge to parse the user's intention and generate structured intention information" specifically includes: using the LLM to perform semantic parsing on the user input, and extracting the key instruction and the target device. For example, when the natural language instruction input by the user is "turn on the living room light", the edge LLM parses the user's intention and generates the structured intention information as "device type: light; device name: living room light; device action: turn on". The output format of the structured data can be in the form of "device type; device name; device action", or in the form of "room; device; action", etc. The structured intention information generated each time may be more than one. For example, when the user says "I want to watch a movie", after the edge LLM module parses it, the output structured data format may be "room: living room; device: curtain; action: close", "room: living room; device: living room light; action: dim the brightness", and "room: living room; device: TV; action: turn on", etc.

[0160] Optionally, by learning the user's usage habits and preferences, the LLM deployed at the edge in this embodiment can recommend personalized smart home scenarios or device settings for the user. The specific implementation method is as follows:

[0161] Data collection: Record the user's daily operations and preference settings;

[0162] Model training: Use a small-scale LLM to train the collected data to generate a user preference model;

[0163] Scenario recommendation: Recommend corresponding smart scenarios or device settings according to the user's current status or historical behavior.

[0164] For example, in the case where the user habitually turns off the living room lights at 10 pm every day, the system actively recommends or automatically executes this operation when approaching this time.

[0165] Optionally, the LLM deployed at the edge in this embodiment can enhance the multi-modal interaction ability of the central control screen and improve the intelligence level of information presentation. The specific implementation method is as follows:

[0166] Voice interaction: Process voice input through the LLM to achieve natural language conversations;

[0167] Text generation: Generate detailed text information according to user queries, such as device status reports, etc.;

[0168] Visual presentation: Combine the text generated by the LLM to optimize the information display on the UI interface.

[0169] For example, when the user asks, "What's the weather like today?" The system generates a detailed weather report through the LLM and presents it in a graphically rich form on the screen.

[0170] Optionally, the edge-deployed LLM in this embodiment runs on a local device, without relying on cloud computing power, has a fast response speed, and can effectively protect user privacy. The specific implementation method is as follows:

[0171] Local deployment: Deploy the LLM model on the central control screen device for localization processing;

[0172] Data isolation: User data is saved locally to avoid uploading to the cloud and protect privacy.

[0173] For example, the user's voice commands are parsed and processed locally to avoid uploading voice data and ensure privacy security.

[0174] Optionally, this embodiment describes the steps of "matching the user's home devices and scenario information based on a local rule engine or similarity matching or recommendation algorithm, and inferring specific device control instructions", which specifically include: According to the structured inference results output by the LLM, the decision-making layer completes the following tasks:

[0175] 1. Match the inference results with information such as the device metadata of the actual user's home to accurately obtain the information of the controlled subject;

[0176] 2. For cases where accurate matching is not possible, perform fuzzy matching and extended matching;

[0177] 3. Finally, translate the matching results into specific control instructions, which are sent by the communication module to specific intelligent devices.

[0178] Among them, the specific matching logic includes:

[0179] 1. Exact name matching, such as room name, device name, light group name, scenario exact matching;

[0180] 2. Name prefix matching, such as spotlights and spotlight 1;

[0181] 3. Fuzzy name matching before and after, such as ceiling lights and living room ceiling light 1;

[0182] 4. Control devices under the room, such as all downlights in the living room;

[0183] 5. Control device types, such as turning on all ceiling lights (limited by the device ID differentiation);

[0184] 6. Control devices or scenarios, such as turning on the movie-watching scenario. When the scenario cannot be matched, query whether there is a device name match.

[0185] Specific implementation methods include:

[0186] 1. Rule engine:

[0187] Based on preset rules and conditions, quickly match user intentions with device scenario configurations. Specifically:

[0188] 1) The cloud manages the rules (rules in the rule engine) and specific config information (device configuration information, which defines the functions, behaviors, and interactions with the cloud of the device). The rules information is saved as structured json;

[0189] 2) The screen dynamically pulls the cloud config data regularly to update the local rules engine, avoiding hard-coding issues (referring to directly writing fixed values in the program code instead of providing these values through configuration files, databases, user input, or other dynamic methods);

[0190] 3) When the user performs voice control, the LLM model calls the rule engine to run all the rules.

[0191] 2. Algorithm recommendation:

[0192] Adopt a machine learning-based model to analyze the user's historical operation habits and preferentially recommend matching instructions. Specifically:

[0193] 1) Historical operation record learning: Record the user's device control history and build an operation preference database.

[0194] 2) Recommendation algorithms:

[0195] a. Collaborative filtering: Recommend control instructions according to the operation habits of similar users.

[0196] b. Content recommendation: Recommend based on the matching of device attributes and user historical behaviors.

[0197] c. Time series model: Use a time series model (such as LSTM) to predict the user's possible next operation.

[0198] d. Dynamic adjustment: The algorithm learns the user's new behaviors in real time and updates the recommendation model.

[0199] 3. Similarity matching:

[0200] Select the most suitable device or operation through the similarity calculation of intent semantics and device attributes. Specifically,

[0201] 1) Semantic analysis: Parse the natural language instructions of the user through a large language model (LLM) to extract the semantic features of the intent.

[0202] 2) Similarity calculation: Calculate the matching degree between the user intent and the device attributes based on cosine similarity, Euclidean distance, or deep learning embedding model.

[0203] 3) Priority ranking: Sort according to the similarity score and select the device or operation that best matches the intent.

[0204] Fuzzy matching support:

[0205] 1. Fuzzy matching of device names: Support prefix, suffix, or regular expression matching of device names (e.g., "table lamp" matches "bedroom table lamp").

[0206] 2. Scene semantic extension: When the matching fails, try to extend to a wider range of device or scene control.

[0207] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present application.

[0208] Figure 5 is a structural block diagram of a control device 50 of an Internet of Things device according to an embodiment of the present application; as Figure 5 shown, it includes:

[0209] A parsing module 52, configured to perform intent parsing on natural language instructions through a large language model to generate intent information, where the large language model is deployed on a local server or a control device;

[0210] It should be noted that the above control device can be a central control screen, or can also be a device such as a computer or a mobile phone. The present application does not limit this.

[0211] A generation module 54, configured to match a target Internet of Things device in a target area according to the intent information and generate a device control instruction corresponding to the target Internet of Things device, where the target area includes multiple Internet of Things devices, and the multiple Internet of Things devices include the target Internet of Things device;

[0212] A sending module 56, configured to send the device control instruction to the target Internet of Things device through a local area network protocol to control the target Internet of Things device according to the natural language instruction.

[0213] It should be noted that the sending module can directly send the device control instruction to the target Internet of Things device, or send the device control instruction to the gateway, and then forward it to the target Internet of Things device through the gateway. This application does not limit this.

[0214] Through the above device, the natural language instruction received is subjected to intent parsing by a large language model to generate intent information, where the large language model is deployed in a local server or the control device; then, according to the intent information, a target Internet of Things device in the target area is matched, and a device control instruction corresponding to the target Internet of Things device is generated, where there are multiple Internet of Things devices in the target area, and the multiple Internet of Things devices include the target Internet of Things device; finally, the device control instruction is sent to the target Internet of Things device, so as to control the target Internet of Things device according to the natural language instruction of the target object; adopting the above solution, the edge LLM large model is deployed using local hardware resources (local server or control device), and the user instruction parsing and reasoning work are completed locally. While quickly inferring and generating control instructions, the user's data privacy is ensured; thus, the problems of privacy leakage, response delay, and network dependence caused by relying on cloud computing power for user instruction parsing and reasoning in the related technology are solved.

[0215] Optionally, the control device includes at least one of the following: a voice acquisition module, a text acquisition module, a visual acquisition module. The parsing module 52 is further configured to receive voice information input by the target object through the voice acquisition module, or receive text information input by the target object through the text acquisition module, or receive image information input by the target object through the visual acquisition module; perform noise cancellation on the voice information through audio processing technology to obtain clear voice information, or perform standardization processing on the text information through text processing technology to obtain standardized text information, or perform image recognition processing on the image information through visual processing technology to obtain semantic information corresponding to the image information; process the clear voice information or the standardized text information or the semantic information through a natural language processing module to obtain the natural language instruction.

[0216] Optionally, the parsing module 52 is further configured to perform semantic analysis on the natural language instruction through the large language model, extract key information in the natural language instruction, where the key information includes at least one of the following: device type, device name, control action; generate multiple pieces of structured intent information according to the key information, where the intent information includes the multiple pieces of structured intent information.

[0217] Optionally, the generating module 54 is further configured to obtain operation habit data of the target object for operating the multiple Internet of Things devices, where the operation habit data includes: operation behavior, preference settings; train a user preference model corresponding to the target object according to the operation habit data.

[0218] Optionally, the generating module 54 is further configured to obtain status data of the target object within the target time period corresponding to the natural language instruction; determine scenario information corresponding to the natural language instruction according to the status data and the user preference model, and determine target intent information from the multiple pieces of structured intent information according to the scenario information; determine the target Internet of Things device according to the target intent information, and generate the device control instruction.

[0219] Optionally, the generating module 54 is further configured to perform precise matching in the device information of the multiple Internet of Things devices according to the target intent information; generate the device control instruction when the target Internet of Things device is matched; when the target Internet of Things device is not matched, perform fuzzy matching in the multiple pieces of device information according to the target intent information to match to the target Internet of Things device, and generate the device control instruction.

[0220] Optionally, the generating module 54 is further configured to calculate the similarity between the target intent information and the multiple pieces of device information in sequence to obtain multiple similarity scores; sort the multiple Internet of Things devices according to the multiple similarity scores, and determine whether the target similarity score corresponding to the Internet of Things device ranked first is greater than a preset threshold; when the target similarity score is greater than or equal to the preset threshold, determine the Internet of Things device corresponding to the target similarity score as the target Internet of Things device; when the target similarity score is less than the preset threshold, determine that the target Internet of Things device is not matched.

[0221] Optionally, the generating module 54 is further configured to extract factual information from the natural language instruction, where the factual information is structured data of the intent information; update the fact library according to the status information of the multiple Internet of Things devices at the current moment to obtain an updated fact library; perform rule reasoning according to the factual information and the updated fact library through a rule engine, determine the target Internet of Things device according to the reasoning result, and generate the device control instruction.

[0222] Optionally, the generating module 54 is further configured to obtain cloud configuration data from a cloud server at a preset period, where the cloud configuration data includes: device configuration information, rule data; update the local configuration data according to the cloud configuration data, and instruct the rule engine to regenerate multiple rules according to the updated local configuration data, where the local configuration data is stored on the local server or the control device.

[0223] Figure 6 is a structural block diagram of a control device according to an embodiment of the present application, as Figure 6 shown, including: a control device 50 of an Internet of Things device as Figure 5 shown; a human-computer interaction unit 62 connected to the control device 50, configured to receive a natural language instruction input by a target object; an AI acceleration unit 64 connected to the control device 50, configured to improve the processing efficiency of the parsing module 52 of the control device 50; and a communication unit 66 connected to the control device 50, configured to perform information interaction with the multiple Internet of Things devices.

[0224] Through the human-computer interaction unit, the user can interact with the smart home system through natural language (such as voice or text), providing a more natural and intuitive control method. This design enables the system to better understand the user's intent, reduces the complexity of operations, and improves the user experience and the user-friendliness of the system.

[0225] The AI acceleration unit 64 is connected to the parsing module 52 in the control device 50. Its main function is to accelerate natural language processing tasks (such as large language model reasoning processes) through dedicated hardware (such as NPU, GPU, DSP, etc.). The use of the AI acceleration unit significantly reduces the calculation time of instruction parsing, improves the response speed of the system, and better meets the real-time requirements of voice control.

[0226] The function of the communication unit is to interact with multiple Internet of Things (IoT) devices in the system. It supports multiple communication protocols, such as Zigbee, Wi-Fi, etc., to adapt to the communication requirements of different devices. The communication unit efficiently sends control instructions to the target device and can also receive the status feedback of the device, forming a closed-loop control to ensure the accurate execution of control actions and enabling real-time adjustment and optimization of strategies.

[0227] By closely integrating the human-computer interaction unit, the AI acceleration unit, and the communication unit with the control device, the control device can achieve rapid user intent recognition, efficient instruction processing, and precise device control. This integrated design improves the intelligence of the system, enabling it to better adapt to user needs and environmental changes and providing personalized and timely home services.

[0228] Figure 7 is a structural block diagram of a control system for an Internet of Things device according to an embodiment of the present application, as Figure 7 shown. The control system includes:

[0229] A control device 72, configured to parse the intent of a natural language instruction through a large language model to generate intent information, where the large language model is deployed on a local server or the control device; match a target Internet of Things device within a target area according to the intent information and generate a device control instruction corresponding to the target Internet of Things device, where the target area includes multiple Internet of Things devices, and the multiple Internet of Things devices include the target Internet of Things device; send the device control instruction to the target Internet of Things device through a local area network protocol to control the target Internet of Things device according to the natural language instruction;

[0230] In an optional embodiment, the structural composition of the control device 72 may be an architecture as Figure 6 shown, Figure 6 for illustrative purposes only. In actual applications, the control device may also include more or fewer components. For example, the control device may also include a processor and a memory, and the present application does not limit this. Figure 6 The target Internet of Things device 74, configured to receive the device control instruction through the local area network protocol and execute the device control instruction.

[0231]

[0232] ​Through the above system, the local understanding and processing of natural language instructions, as well as the local transmission and execution of device control instructions, are realized, greatly improving the response speed, security, privacy protection ability and control accuracy of the system. This design not only provides a user-friendly and intuitive interaction method, but also reduces the dependence on the network. Even in the case of network disconnection or instability, the system can still operate normally, ensuring the stability and reliability of the smart home environment. In addition, due to the localization of the large language model and device control logic, the occupancy of cloud resources is also reduced, which helps to reduce the system operation cost and improve the resource utilization efficiency.

[0233] In an alternative embodiment, Figure 8 The network topology of the control system of the above-mentioned Internet of Things device is shown, as Figure 8 For a simple system architecture, the above control device 72 (such as a smart phone, smart speaker, central control screen, etc.) and the target Internet of Things device 74 (such as a smart bulb, smart socket, smart curtain, etc.) communicate directly through a local area network (Local Area Network, LAN).

[0234] The characteristics and advantages of this form are as follows:

[0235] 1. Direct LAN connection: There is no intermediate device between the control device and the controlled device, and the local area network protocol (such as WIFI, Zigbee or Z-Wave) is directly used for data exchange, reducing communication latency and improving the instant response ability of the system.

[0236] 2. Privacy protection: Since all data and instruction processing are completed within the local area network without being uploaded to the cloud through the Internet, the user's voice, text instructions and device status data are more secure, avoiding potential privacy leakage risks.

[0237] 3. Cost and resource efficiency: Such a system architecture reduces the demand for cloud server resources, reduces network communication costs, and also means lower power consumption and operation costs.

[0238] In another alternative embodiment, Figure 9 A complex network topology is shown, as Figure 9 As shown, this network topology includes multiple components such as a cloud server, a router, a control device, a gateway and a target Internet of Things device. This architecture can provide more powerful functions and a wider scope of application, and its structure and advantages are as follows:

[0239] 1. Cloud server: Supports remote users to access and control smart devices at home through the Internet;

[0240] 2. Routers and Gateways: The router is responsible for connecting the local area network to the Internet, while the gateway performs protocol conversion and adaptation between different communication protocols to ensure that information between different devices can be correctly understood and transmitted. This enables the system to integrate various communication technologies such as WIFI, Zigbee, BLE Mesh, OpenThread, etc., improving device compatibility and communication efficiency.

[0241] 3. Control Devices and Target IoT Devices: The control device communicates with the gateway through a local area network protocol (such as WIFI), and the gateway further sends the control instructions to the target IoT device through an appropriate protocol. This hierarchical communication structure provides better device management capabilities and broader device support.

[0242] An embodiment of the present application also provides a storage medium, which includes a stored program. When the above program runs, it executes the method of any one of the above.

[0243] Optionally, in this embodiment, the above storage medium may be set to store program code for performing the following steps:

[0244] S11, perform intent parsing on the natural language instruction through a large language model to generate intent information, where the large language model is deployed on a local server or the control device;

[0245] S12, match the target IoT device in the target area according to the intent information and generate a device control instruction corresponding to the target IoT device, where the target area includes multiple IoT devices, and the multiple IoT devices include the target IoT device;

[0246] S13, send the device control instruction to the target IoT device through the local area network protocol to control the target IoT device according to the natural language instruction.

[0247] An embodiment of the present application also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0248] Optionally, the above electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0249] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0250] S11. Parse the intent of the natural language instruction through a large language model to generate intent information, where the large language model is deployed on a local server or the control device;

[0251] S12. Match the target Internet of Things device in the target area according to the intent information, and generate a device control instruction corresponding to the target Internet of Things device, where the target area includes multiple Internet of Things devices, and the multiple Internet of Things devices include the target Internet of Things device;

[0252] S13. Send the device control instruction to the target Internet of Things device through the local area network protocol to control the target Internet of Things device according to the natural language instruction.

[0253] Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk or optical disc, etc., various media that can store program codes.

[0254] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated here.

[0255] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the present application is not limited to any specific combination of hardware and software.

[0256] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A control method for an Internet of Things device, characterized in that: Applied to control equipment, including: Performing intent parsing on the natural language instruction through a large language model to generate intent information, wherein the large language model is deployed on a local server or the control device; Matching a target IoT device in a target area according to the intention information, and generating a device control instruction corresponding to the target IoT device, wherein the target area includes a plurality of IoT devices, and the plurality of IoT devices include the target IoT device; The device control instruction is sent to the target IoT device via a local area network protocol, so as to control the target IoT device according to the natural language instruction.

2. The control method of the Internet of Things device according to claim 1, characterized in that: The control device includes at least one of the following: a voice acquisition module, a text acquisition module, and a visual acquisition module; Before performing intent parsing of the natural language instruction by using the large language model to generate intent information, the method further includes: Receiving voice information input by the target object through the voice acquisition module, or receiving text information input by the target object through the text acquisition module, or receiving image information input by the target object through the visual acquisition module; Performing noise elimination on the voice information through audio processing technology to obtain clarified voice information, or performing standardization processing on the text information through text processing technology to obtain standardized text information, or performing image recognition processing on the image information through visual processing technology to obtain semantic information corresponding to the image information; The clarified voice information or the standardized text information or the semantic information is processed by a natural language processing module to obtain the natural language instruction.

3. The control method of the Internet of Things device according to claim 1, characterized in that: The natural language instruction is parsed for intent by a large language model to generate intent information, including: Performing semantic analysis on the natural language instruction by using the large language model to extract key information in the natural language instruction, wherein the key information includes at least one of the following: device type, device name, and control action; A plurality of structured intent information are generated according to the key information, wherein the intent information includes the plurality of structured intent information.

4. The control method of the Internet of Things device according to claim 3, characterized in that: Before matching the target IoT device in the target area according to the intention information and generating a device control instruction corresponding to the target IoT device, the method further includes: Acquire the target object's operation habit data for operating the multiple IoT devices, wherein the operation habit data includes: operation behavior and preference settings; A user preference model corresponding to the target object is trained according to the operation habit data.

5. The control method of the Internet of Things device according to claim 4, characterized in that: Matching a target IoT device in a target area according to the intent information and generating a device control instruction corresponding to the target IoT device includes: Acquire state data of the target object within a target time period corresponding to the natural language instruction; Determining the scenario information corresponding to the natural language instruction according to the state data and the user preference model, and determining the target intent information from the multiple structured intent information according to the scenario information; The target IoT device is determined according to the target intent information, and the device control instruction is generated.

6. The control method of the Internet of Things device according to claim 1 or 3, characterized in that: Matching a target IoT device in a target area according to the intent information and generating a device control instruction corresponding to the target IoT device includes: Performing precise matching on the device information of the multiple IoT devices according to the intent information; If the target IoT device is matched, the device control instruction is generated; if the target IoT device is not matched, fuzzy matching is performed among the multiple device information according to the intent information to match the target IoT device and generate the device control instruction.

7. The control method of the Internet of Things device according to claim 6, characterized in that: Accurately matching the device information of the multiple IoT devices according to the intent information includes: Calculating the similarity between the intention information and the plurality of device information in sequence to obtain a plurality of similarity scores; Sorting the plurality of IoT devices according to the plurality of similarity scores, and determining whether the target similarity score corresponding to the first-ranked IoT device is greater than a preset threshold; When the target similarity score is greater than or equal to the preset threshold, determining the IoT device corresponding to the target similarity score as the target IoT device; When the target similarity score is less than the preset threshold, it is determined that the target IoT device is not matched.

8. The control method of the Internet of Things device according to claim 3, characterized in that: Matching a target IoT device in a target area according to the intent information and generating a device control instruction corresponding to the target IoT device includes: Extracting factual information from the natural language instruction, wherein the factual information is structured data of the intent information; The fact base is updated according to the status information of the plurality of IoT devices at the current moment to obtain an updated fact base; The rule engine performs rule inference according to the fact information and the updated fact base, determines the target IoT device according to the inference result, and generates the device control instruction.

9. The control method of the Internet of Things device according to claim 8, characterized in that: The method further comprises: Obtaining cloud configuration data from a cloud server according to a preset period, wherein the cloud configuration data includes: device configuration information and rule data; The local configuration data is updated according to the cloud configuration data, and the rule engine is instructed to regenerate a plurality of rules according to the updated local configuration data, wherein the local configuration data is stored on the local server or the control device.

10. A control device for an Internet of Things device, characterized in that: include: A parsing module, used to perform intent parsing on natural language instructions through a large language model to generate intent information, wherein the large language model is deployed on a local server or control device; a generating module, used to match a target IoT device in a target area according to the intent information, and generate a device control instruction corresponding to the target IoT device, wherein the target area includes a plurality of IoT devices, and the plurality of IoT devices include the target IoT device; A sending module is used to send the device control instruction to the target Internet of Things device through a local area network protocol, so as to control the target Internet of Things device according to the natural language instruction.

11. A control device, characterized in that: include: The control device of the Internet of Things device as claimed in claim 10; a human-computer interaction unit connected to the control device, used to receive natural language instructions input by the target object; and an AI acceleration unit of the control device, used to improve the processing efficiency of the parsing module of the control device; A communication unit connected to the control device is used to exchange information with the multiple Internet of Things devices.

12. A control system for an Internet of Things device, characterized in that: include: A control device, configured to perform intent parsing on a natural language instruction through a large language model to generate intent information, wherein the large language model is deployed on a local server or the control device; match a target IoT device in a target area according to the intent information, and generate a device control instruction corresponding to the target IoT device, wherein the target area includes a plurality of IoT devices, and the plurality of IoT devices include the target IoT device; send the device control instruction to the target IoT device through a local area network protocol to control the target IoT device according to the natural language instruction; The target IoT device is used to receive the device control instruction through the local area network protocol and execute the device control instruction.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 9 when executed.

Images