Control method, system and terminal device of internet of things device

Abstract

This application provides a control method, system, and terminal device for IoT devices. The method includes: first, acquiring user demand information; then, if it is determined that operation information corresponding to the user demand information cannot be generated, generating first instruction information based on the user demand information and the device profile of the IoT device, wherein the device profile of the IoT device includes descriptive information of the operations supported by the IoT device; subsequently, sending the first instruction information to a model service platform, wherein the first instruction information instructs the model service platform to generate the operation information, the operation information instructs the IoT platform to generate a control command, and the control command instructs the target IoT device to perform a target operation. In this way, user intent recognition through edge-cloud collaboration can improve the accuracy of IoT device control, reduce information transmission latency, and enhance user experience.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a control method, system, and terminal device for IoT devices. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, especially in natural language processing, computer vision, and machine learning, AI models can better understand user needs and behavioral habits. Today, an increasing number of home appliances support voice assistant control. The voice assistant invokes AI models to recognize user intentions and then controls the corresponding home appliances based on the recognition results, greatly improving people's quality of life.

[0003] Currently, user intent recognition typically relies on AI models deployed on the device or in the cloud. However, limited by the computing power of device-side AI models, their performance is limited, making it difficult to handle complex or ambiguous intents, resulting in inaccurate control of home appliances. While cloud-deployed AI models possess stronger computing power and can more accurately recognize user intents, they require uploading voice data to the cloud for processing before transmitting the results back to the device. This process is affected by network conditions, and the reliability and speed of information transmission may negatively impact the user experience. Summary of the Invention

[0004] In view of this, this application provides a control method, system, and terminal device for Internet of Things (IoT) devices. This method can improve the accuracy of controlling IoT devices, reduce information transmission latency, and enhance user experience.

[0005] In a first aspect, this application provides a control method for an Internet of Things (IoT) device, applied to a terminal device. The method includes: firstly, acquiring user demand information; then, if it is determined that operation information corresponding to the user demand information cannot be generated, generating first instruction information based on the user demand information and the device profile of the IoT device, wherein the device profile of the IoT device includes descriptive information of the operations supported by the IoT device; subsequently, sending the first instruction information to a model service platform, wherein the first instruction information is used to instruct the model service platform to generate the operation information, the operation information is used to instruct the IoT platform to generate a control command, and the control command is used to instruct the target IoT device to perform a target operation.

[0006] The phrase "unable to generate operation information corresponding to the user's request information" can be understood as "unable to identify the user's intent corresponding to the user's request information." Furthermore, "unable to generate operation information corresponding to the user's request information" can include "unable to generate all or part of the operation information corresponding to the user's request information," that is, "unable to identify all or part of the user's intent corresponding to the user's request information." In other words, this application requests the cloud (such as a model service platform) to identify all or part of the user's intent when the terminal side (such as a set-top box) cannot identify all or part of the user's intent. This employs a terminal-cloud collaborative approach to achieve user intent identification. This solves the problem in existing technologies where the terminal side alone cannot identify complex or ambiguous intents, ensuring the accuracy of IoT device control. When the terminal side can identify the user's intent, there is no need to send a request to the cloud, reducing the number of requests from the terminal side to the cloud and reducing bandwidth pressure. This, in turn, reduces information transmission latency and ensures information transmission reliability, thus addressing to some extent the problem of low reliability and long latency in information transmission when only the cloud identifies user intent.

[0007] Secondly, the device profile of an IoT device includes descriptive information about the operations supported by the IoT device. This operational information is generated based on the device profile. Therefore, the operational information is independent of the IoT device's brand, model, software, or hardware information. In other words, when generating operational information, the terminal device and model service platform do not need to consider the IoT device's brand, model, software, or hardware information, thus reducing the complexity of generating operational information corresponding to user needs. Similarly, when users input their needs, they do not need to consider the IoT device's brand, model, software, or hardware information, further reducing the difficulty of user interaction with the terminal device. Furthermore, the control of IoT devices in this application is not limited by the IoT device's brand, model, software, or hardware, and has universal applicability.

[0008] For example, the terminal device can be a smart device used to control IoT devices; such as a smart speaker, a smart robot, a set-top box, etc.

[0009] For example, the operation information is used to describe what operation is performed on what type of IoT device, where it is located.

[0010] For example, this application does not limit the way users input their needs information. For instance, users can input their needs information through voice, gestures, remote control, etc.

[0011] For example, user request information can be information with a simple intent (or information with a simple operational intent). Information with a simple intent can refer to information that includes a specific device and a specific operation, such as "turn on the living room lights," "close all the curtains in the house," or "turn on the TV." User request information can also be information with a complex intent (or information with a complex operational intent). Information with a complex intent can refer to information that does not include a specific device and a specific operation, such as "turn on home theater mode" or "make the atmosphere in the living room more comfortable."

[0012] For example, the IoT platform can generate device information for an IoT device based on the original device information uploaded during the registration process and send this information to the terminal device. The terminal device can then generate a device profile for the IoT device based on this device information. Here, "IoT device" in "IoT device information obtained from the IoT platform" can refer to all IoT devices included in the current IoT application scenario. For example, if the current IoT application scenario is a smart home scenario and the current IoT application location is a home, then device information for all IoT devices in that home can be obtained from the IoT platform. Similarly, if the current IoT application scenario is a smart agriculture scenario and the current IoT application location is a farm greenhouse, then device information for all IoT devices in that farm greenhouse can be obtained from the IoT platform. The original device information for an IoT device can include its name, brand, location, model, software information, and hardware information, etc. The device information for an IoT device can include its name, location, and the names and parameters (optional) of the operations it supports, etc. The IoT platform can determine the name and parameters (optional) of the original operations supported by the IoT device based on the IoT device's brand, model, software information, and hardware information. Then, according to preset rules (set with the goal of decoupling the operations supported by the IoT device from the IoT device's brand, model, software information, and hardware information), it can adjust the name and parameters (optional) of the original operations supported by each IoT device to obtain the name and parameters (optional) of the operations supported by each IoT device. Finally, it can generate the conversion / mapping relationship between the original operations supported by the IoT device and the operations supported by the IoT device.

[0013] For example, if the original operation name supported by air conditioner brand 1 is "increase fan speed", the original operation name supported by air conditioner brand 2 is "increase fan speed", and the original operation name supported by air conditioner brand 3 is "increase fan speed", then according to the preset rules, after adjusting the original operation names supported by these three brands of air conditioners, the name of the operation supported by these three brands of air conditioners will be "increase fan speed".

[0014] For example, the original operation name supported by air conditioner brand 1 is "Set Fan Speed", with parameters "0" - low fan speed, "1" - medium fan speed, and "2" - high fan speed; the original operation name supported by air conditioner brand 2 is "Set Fan Speed", with parameters "0" - high fan speed, "1" - medium fan speed, and "2" - low fan speed; the original operation name supported by air conditioner brand 3 is "Set Fan Speed", with parameters "0" - low fan speed, "1" - low-medium fan speed, "2" - medium fan speed, "3" - medium-high fan speed, "4" - high fan speed, and "5" - strong fan speed. According to preset rules, after adjusting the parameters of the original operation supported by these three brands of air conditioners, the operation name supported by these three brands of air conditioners is now "Set Fan Speed", with parameters "0" - low fan speed, "1" - low-medium fan speed, "2" - medium fan speed, "3" - medium-high fan speed, "4" - high fan speed, and "5" - strong fan speed.

[0015] In this way, the name and parameters (optional) of the operations supported by each IoT device are independent of the IoT device's brand, model, software information, or hardware information.

[0016] For example, the model service platform and the IoT platform can be deployed on the same server or on different servers, and this application does not impose any restrictions on this.

[0017] For example, an IoT platform can provide IoT device connectivity and management services, IoT device data collection and analysis services, and IoT device control services, etc.

[0018] For example, a model service platform can provide services such as user intent recognition.

[0019] According to the first aspect, the descriptive information of the operations supported by the IoT device includes the name of the operations supported by the IoT device, and the device profile of the IoT device also includes: the name of the IoT device and the location of the IoT device.

[0020] For example, the device profile of an IoT device includes: air conditioner (name of the IoT device), living room / dining room / master bedroom / secondary bedroom (location of the IoT device), setting temperature / increasing temperature / decreasing temperature / setting fan speed / increasing fan speed / decreasing fan speed / setting mode (name of the operation supported by the IoT device); curtains (name of the IoT device), living room / master bedroom / secondary bedroom (location of the IoT device), opening curtains / closing curtains (name of the operation supported by the IoT device).

[0021] According to the first aspect, or any implementation of the first aspect above, the description information of the operations supported by the IoT device also includes parameters of the operations supported by the IoT device.

[0022] For example, the parameters for increasing the temperature include: service type (temperature setting), parameter value (step), parameter value type (integer), and description (the increased temperature value, with a default step of 1).

[0023] For example, the parameters for setting wind speed include: service type (wind speed setting), parameter value (wind speed value), parameter value type (enumeration type), and description (0: automatic, 1: silent, 2: low wind, 3: medium wind, 4: high wind, 5: very strong wind).

[0024] According to the first aspect, or any implementation of the first aspect above, the operation information includes one or more items, each item of operation information including: the name of the target IoT device, the target location of the target IoT device, and the name of the target operation to be performed by the target IoT device.

[0025] According to the first aspect, or any implementation of the first aspect above, each operation information also includes: parameters of the target operation to be performed by the target IoT device.

[0026] According to the first aspect, or any implementation of the first aspect above, the method further includes: identifying the user's identity; and, if it is determined that operation information corresponding to the user's demand information cannot be generated, generating first instruction information based on the user's demand information and the device profile of the IoT device, including: generating the first instruction information based on the user's demand information and the device profile of the IoT device corresponding to the user's identity if it is determined that operation information corresponding to the user's demand information cannot be generated.

[0027] Among them, the device profile of the IoT device corresponding to the user identity can refer to the device profile of the IoT device that the user identity has control over.

[0028] In one possible implementation, the device profile of the IoT device corresponding to the user's identity may include: descriptive information of all operations supported by the IoT device for which the user has control authority.

[0029] In one possible implementation, the device profile of the IoT device corresponding to the user's identity may include: descriptive information of some operations supported by the IoT device that the user has control over (operations that the user has control over).

[0030] This ensures that each user can only control IoT devices for which they have control authority, thus achieving personalization and security in IoT device control. This approach not only improves the user experience but also enhances the security and accuracy of device management. Furthermore, since this application uses terminal devices for identification, the terminal devices do not need to send user personal information such as voiceprints, fingerprints, or facial features to the model service platform, ensuring the security of user personal information.

[0031] In addition, terminal devices and IoT devices will not upload other personal data (such as personal health data, user audio and video data, device usage data, etc.) to the model service platform or IoT platform; this can ensure the security of personal information.

[0032] According to the first aspect, or any implementation of the first aspect above, when it is determined that the operation information corresponding to the user demand information cannot be generated, a first instruction information is generated based on the user demand information and the device profile of the IoT device, including: when it is determined that all operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of all IoT devices; when it is determined that some operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of some IoT devices, wherein the some IoT devices are IoT devices other than the IoT devices corresponding to the operation information generated by the terminal device among all IoT devices.

[0033] In this way, when the user's intent cannot be recognized on the device side, the cloud can recognize the user's intent, which can improve the accuracy of controlling IoT devices. When the device side can recognize part of the user's intent, the cloud can recognize the other part of the user intent. This reduces the amount of data exchanged between the device side and the cloud side, which can further reduce the control latency of IoT devices and improve the user experience.

[0034] According to the first aspect, or any implementation of the first aspect above, the method further includes: identifying the current scenario based on the user demand information; and, if it is determined that operation information corresponding to the user demand information cannot be generated, generating first instruction information based on the user demand information and the device profile of the IoT device, including: if it is determined that the current scenario is an IoT device control scenario, and it is determined that operation information corresponding to the user demand information cannot be generated, generating the first instruction information based on the user demand information, the current scenario, and the device profile of the IoT device. In other words, the current scenario can be used to initially determine whether the user intends to control the IoT device; if it is initially determined that the user intends to control the IoT device, the user's intention is further identified; if it is initially determined that the user does not intend to control the IoT device, the process can be terminated; thus, computing power on both the edge and cloud sides can be saved.

[0035] Based on the first aspect, or any implementation of the first aspect above, if the control complexity of the user's demand information for the IoT device exceeds a threshold, then it is determined that the operation information corresponding to the user's demand information cannot be generated. This eliminates the need to call a model to determine whether the edge can generate user intent, saving computing power on the edge and allowing for a quick determination of whether the edge can generate user intent.

[0036] According to the first aspect, or any implementation of the first aspect above, the control complexity includes the number of devices of the target IoT device and / or the number of target operations of the target IoT device, and the threshold includes a first sub-threshold and a second sub-threshold; when the number of devices of the target IoT device is greater than the first sub-threshold, and / or the number of target operations of the target IoT device is greater than the second sub-threshold, it is determined that the control complexity is greater than the threshold.

[0037] It should be understood that this application does not limit the meaning of "threshold", "first sub-threshold" and "second sub-threshold", how to set them, or whether they can be adjusted, etc.

[0038] According to the first aspect, or any implementation of the first aspect above, the method further includes: receiving operation information sent by the model service platform; and sending second instruction information to the IoT platform, the second instruction information being generated based on operation information corresponding to user demand information. Since terminal devices on the market already have built-in interfaces for communication with the IoT platform, while the model service platform does not have such interfaces, sending operation information to the IoT platform through the terminal device instead of the model service platform eliminates the need for additional hardware and software development for the communication interface, thereby significantly reducing costs and system complexity.

[0039] For example, in this case, the operation information corresponding to the user's demand information may include operation information generated by the model service platform, or the operation information corresponding to the user's demand information may include operation information generated by the model service platform and operation information generated locally by the terminal device.

[0040] Secondly, this application provides a control system for an Internet of Things (IoT) device, the control system comprising: a terminal device, a model service platform, an IoT platform, and the IoT device.

[0041] The terminal device is used to obtain user demand information; when it is determined that the operation information corresponding to the user demand information cannot be generated, it generates first instruction information based on the user demand information and the device profile of the IoT device, the device profile of the IoT device including description information of the operations supported by the IoT device; and sends the first instruction information to the model service platform.

[0042] The model service platform is used to generate the operation information based on the first instruction information; and to send the operation information to the terminal device.

[0043] The terminal device is also used to generate second instruction information based on the operation information; and to send the second instruction information to the Internet of Things platform;

[0044] The IoT platform is used to generate control commands based on the operation information in the second instruction information and send the control commands to the target IoT device.

[0045] The target IoT device is used to execute the target operation corresponding to the control command.

[0046] It should be understood that the terminal devices in this control system can also be used to execute the methods in any of the implementations of the first aspect described above.

[0047] Thirdly, this application provides a terminal device for:

[0048] Obtain user demand information;

[0049] If it is determined that the operation information corresponding to the user's demand information cannot be generated, a first instruction information is generated based on the user's demand information and the device profile of the IoT device. The device profile of the IoT device includes a description of the operations supported by the IoT device.

[0050] The first instruction information is sent to the model service platform, which instructs the model service platform to generate the operation information. The operation information instructs the IoT platform to generate control instructions, which instruct the target IoT device to perform the target operation.

[0051] According to the third aspect, the terminal device includes at least one of the following: a set-top box, a smart speaker, or a smart robot.

[0052] It should be understood that the terminal device can also be used to execute the methods in any of the implementations of the first aspect described above.

[0053] Fourthly, this application provides a control device for an Internet of Things (IoT) device, the device comprising:

[0054] The requirement elicitation module is used to obtain user requirement information;

[0055] The instruction generation module is used to generate first instruction information based on the user demand information and the device profile of the IoT device when it is determined that the operation information corresponding to the user demand information cannot be generated. The device profile of the IoT device includes description information of the operations supported by the IoT device.

[0056] The communication module is used to send the first instruction information to the model service platform. The first instruction information is used to instruct the model service platform to generate the operation information. The operation information is used to instruct the IoT platform to generate control instructions. The control instructions are used to instruct the target IoT device to perform the target operation.

[0057] It should be understood that the control device of the Internet of Things device can also be used to execute the methods in any of the implementations of the first aspect described above.

[0058] Fifthly, this application provides a terminal device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the terminal device to perform the method in the first aspect or any possible implementation thereof.

[0059] In a sixth aspect, this application provides a chip including one or more interface circuits and one or more processors; the one or more processors receive or transmit data through the one or more interface circuits, and when the one or more processors execute computer instructions, cause a terminal device to perform the method in the first aspect or any possible implementation of the first aspect.

[0060] In a seventh aspect, this application provides a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to perform the method of the first aspect or any possible implementation thereof.

[0061] Eighthly, this application provides a computer program product including computer instructions that, when executed by a computer or processor, cause the computer or processor to perform the method in the first aspect or any possible implementation thereof.

[0062] In this embodiment, the terminal device, computer-readable storage medium, computer program product, chip or device, system, etc. are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above. Attached Figure Description

[0063] Figure 1A This is a schematic diagram illustrating one application scenario of an embodiment of this application;

[0064] Figure 1B This is a schematic diagram illustrating another application scenario of an embodiment of this application;

[0065] Figure 1C This is a schematic diagram of the framework of a control system 100 for an Internet of Things (IoT) device according to an embodiment of this application.

[0066] Figure 1D This is a schematic diagram of the framework of a control system 100 for another IoT device according to an embodiment of this application;

[0067] Figure 2 This is a schematic diagram of a control process 200 of an Internet of Things (IoT) device according to an embodiment of this application;

[0068] Figure 3 This is a schematic diagram of the control process 300 of another Internet of Things (IoT) device according to an embodiment of this application;

[0069] Figure 4 This is a schematic diagram of the control process 400 of another Internet of Things (IoT) device according to an embodiment of this application;

[0070] Figure 5 This is a schematic diagram of a control device 500 for an Internet of Things (IoT) device according to an embodiment of this application;

[0071] Figure 6 This is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0074] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0075] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0076] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0077] In the embodiments of this application, the modules / components shown in the framework diagram (or structural diagram or system diagram) are merely examples of this application. The actual framework (or structure or system) may include more or fewer modules / components than those shown in the diagram, or may have different component configurations. Furthermore, the various components / modules shown in the diagrams may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0078] Figure 1A This is a schematic diagram illustrating one application scenario of an embodiment of this application. Figure 1A The application scenario shown is a smart home (or intelligent home) scenario.

[0079] Figure 1AThe home shown includes multiple Internet of Things (IoT) devices, such as lights, televisions, projectors, routers, and air conditioners. It should be understood that the home may also include other IoT devices such as curtains, water heaters, refrigerators, and stereo systems; this application does not limit this. Each IoT device can connect to the network via a router; thus, these IoT devices can establish communication connections with an IoT platform (which provides connectivity and management services for IoT devices, data collection and analysis services for IoT devices, and control services for IoT devices, etc.).

[0080] Figure 1A The home shown also includes smart devices for controlling IoT devices, such as set-top boxes and smart speakers. It should be understood that the home may also include other smart devices for controlling IoT devices, such as smart robots. These smart devices for controlling IoT devices can connect to the internet via a router; thus, they can establish communication connections with the model service platform (which can provide user intent recognition services, etc.) and the IoT platform.

[0081] For example, one application scenario could be: On a hot summer day, after arriving home from get off work, a user can wake up their smart speaker via voice and issue the voice command, "Turn on the air conditioner and adjust the temperature to 26 degrees Celsius." The smart speaker can then interact with the model service platform and the IoT platform to switch the air conditioner to cooling mode and adjust the temperature to 26 degrees Celsius.

[0082] For example, one application scenario could be: a large family gathering on a rainy day, after lunch, where everyone wants to watch a movie. The homeowner can wake up the set-top box using the remote control and perform the gesture to "activate home theater mode." Afterward, the set-top box can interact with the model service platform and IoT platform to activate home theater mode (where activating home theater mode requires actions such as closing the curtains, turning off the lights, turning on the projector, and turning on the stereo speakers, etc.).

[0083] Figure 1B This is a schematic diagram illustrating another application scenario of this application embodiment. Figure 1B The application scenario shown is a smart agriculture scenario (or intelligent agriculture scenario).

[0084] Figure 1B The farm greenhouse shown includes multiple IoT devices, such as supplemental lighting, temperature control equipment, ventilation equipment, and irrigation equipment. It should be understood that the farm greenhouse may also include other IoT devices such as humidity sensors, temperature sensors, and soil sensors; this application does not impose any limitations on this. Each IoT device can be connected via a router (…). Figure 1B(Not shown in the image) Networked; in this way, these IoT devices can establish a communication connection with the IoT platform.

[0085] Figure 1B The farm greenhouse shown also includes a monitoring robot (a type of intelligent robot). This robot can monitor the greenhouse conditions (such as crop growth, pests and diseases, and land utilization) and control IoT devices within the greenhouse. It should be understood that the farm greenhouse may also include other intelligent devices such as smart speakers. The monitoring robot can connect to the internet via a router, allowing it to establish communication connections with model service platforms and IoT platforms.

[0086] For example, one application scenario could be: after entering a farm greenhouse and finding poor air circulation, monitoring personnel can wake up a monitoring robot via voice and issue the voice command "Open the ventilation equipment." The monitoring robot can then interact with a model service platform and an IoT platform to activate the ventilation equipment.

[0087] For example, one application scenario could be: during a hot summer, monitoring personnel determine that the humidity in a farm greenhouse is too low based on humidity sensor readings. They can then wake up a monitoring robot via voice and issue the voice command "turn on the irrigation equipment." The monitoring robot can then interact with the model service platform and IoT platform to activate the irrigation equipment.

[0088] It should be understood that this application may also include other IoT application scenarios such as smart industry, smart healthcare, and smart logistics, and this application does not impose any restrictions on these.

[0089] Figure 1C This is a schematic diagram of the framework of a control system 100 for an Internet of Things (IoT) device according to an embodiment of this application.

[0090] Figure 1C The control system 100 for IoT devices may include: terminal device 110, model service platform 120, IoT platform 130, IoT device 1 (141), IoT device 2 (142), ..., IoT device N (14N). Wherein, N is a positive integer.

[0091] For example, terminal device 110 can establish communication connections with model service platform 120 and IoT platform 130 respectively. IoT platform 130 can establish communication connections with N IoT devices.

[0092] Optionally, the model service platform 120 and the IoT platform 130 can establish a communication connection.

[0093] For example, the terminal device 110 may include the aforementioned smart devices for controlling IoT devices, such as set-top boxes, smart speakers, and smart robots. Compared to IoT devices, the terminal device 110 has relatively high computing power, can perform relatively complex computing tasks locally, has large storage capacity, and can support complex interactive systems, etc.

[0094] For example, terminal device 110 deploys a first AI model, which can be a small model (a small model usually refers to a machine learning model with fewer parameters and a relatively simple structure). The first AI model can identify simple user intentions, current scene, user identity, etc.

[0095] For example, the model service platform 120 deploys a second AI model, which can be a large model (a large model typically refers to a machine learning model with a huge number of parameters and a complex structure). For example, the performance of the second AI model is better than that of the first AI model; the second AI model can identify complex user intents.

[0096] For example, the IoT platform 130 can generate control commands to instruct IoT devices to perform corresponding operations.

[0097] Figure 1D This is a schematic diagram of the framework of a control system 100 for another IoT device according to an embodiment of this application. Figure 1D Is Figure 1C Based on the above, Figure 1D The structure of terminal device 110 has been added.

[0098] Figure 1D In this system, the terminal device 110 may include a processing module 111, a first agent module 112, a second agent module 113, and a first AI model. The processing module 111 can call the first AI model to identify simple user intentions, current scenarios, user identities, etc. Furthermore, the processing module 111 can communicate with the model service platform 120 through the first agent module 112 and with the IoT platform 130 through the second agent module 113.

[0099] Optionally, a voice assistant (or smart butler, such as a smart home butler in a smart home scenario, or a smart farm butler in a smart agriculture scenario) can be deployed in the processing module 111. The voice assistant can call the first AI model to identify simple user intentions, current scenarios, user identities, etc.

[0100] It should be noted that the model service platform 120 and the IoT platform 130 can be deployed on the same server or on different servers; this application does not impose any restrictions on this. The specific implementation form of the server in this application can be a cloud server, a physical (dedicated) server, a website cluster server, etc.; this application does not impose any restrictions on this.

[0101] The following is based on Figure 1C and Figure 1D This section explains the control process of IoT devices.

[0102] Figure 2 This is a schematic diagram of a control process 200 for an Internet of Things (IoT) device according to an embodiment of this application. Figure 2 The control process of IoT devices 200 can be controlled by Figure 1C or Figure 1D The terminal device 110 in the process executes the command; the following explanation takes the terminal device 110 as a set-top box, which includes a voice assistant.

[0103] S201, Obtain user requirement information.

[0104] For example, a microphone is deployed inside the set-top box 110; this allows users to input their needs via voice. Specifically, after waking up the voice assistant with their voice, the user can speak their needs; the microphone can then capture the user's voice. The microphone can then output the user's voice to the voice assistant, which can call a speech recognition module or a first AI model to perform automatic speech recognition (ASR) to obtain the user's needs.

[0105] For example, the set-top box 110 has an image acquisition module (such as a camera) internally deployed; thus, the user can input user request information through gestures. Specifically, after waking up the set-top box 110, the user can make a gesture corresponding to the user request information within a certain distance from the set-top box 110, so that the camera of the set-top box 110 can capture the user image. Then, the camera can output the user image to the processing module 111, which can call the gesture recognition module or the first AI model to perform gesture recognition based on the user image to obtain the user request information.

[0106] For example, the set-top box 110 has a matching remote control. The set-top box 110 has an infrared module or a Bluetooth module deployed within it, and the remote control has both an infrared module and a Bluetooth module deployed within it. The set-top box 110 and its matching remote control can communicate via the infrared module or the Bluetooth module. This allows the user to input user-required information via the remote control. Specifically, the user presses a button on the remote control according to the user-required information. The remote control can respond to the user's operation by emitting an infrared signal via the infrared module or a Bluetooth signal via the Bluetooth module. Thus, the set-top box 110 can receive the Bluetooth signal emitted by the remote control via its Bluetooth module, or it can receive the infrared signal emitted by the remote control via its infrared module. Then, the Bluetooth module of the set-top box 110 can parse the Bluetooth signal to obtain the load data (such as button identifiers) in the received Bluetooth signal and output this load data to the processing module 111; or, the infrared module of the set-top box 110 can parse the infrared signal to obtain the load data in the received infrared signal and output this load data to the processing module 111. Then, the processing module 111 of the set-top box 110 can determine the corresponding operation of the key identifier included in the load data according to the mapping relationship between the key and the operation, so as to obtain the user demand information.

[0107] It should be understood that this application does not restrict the way users input their needs; correspondingly, it does not restrict the way the set-top box obtains user needs information. This application uses the example of a user inputting user needs information via voice for illustration.

[0108] It should be understood that user request information can be simple in intent (or simple in operation intent). Simple intent information can refer to information that includes specific devices and specific operations, such as "turn on the living room lights," "close all the curtains," or "turn on the TV." User request information can also be complex in intent (or complex in operation intent). Complex intent information can refer to information that does not include specific devices and specific operations, such as "turn on home theater mode" or "make the living room atmosphere more comfortable."

[0109] The statement "The first AI model can identify simple user intents" can be understood as "The first AI model can identify user intents corresponding to information with simple intents." Similarly, the statement "The second AI model can identify complex user intents" can be understood as "The first AI model can identify user intents corresponding to information with complex intents."

[0110] S202, if it is determined that operation information corresponding to user demand information cannot be generated, first instruction information is generated based on user demand information and device profile of IoT device. Device profile of IoT device includes description information of operations supported by IoT device.

[0111] For example, after obtaining user request information, the voice assistant determines whether it can locally recognize the user intent corresponding to the user request information. For example, recognizing the user intent corresponding to the user request information in this application may refer to generating operation information corresponding to the user request information; wherein, the operation information is used to describe what operation is performed on what type of IoT device at what location.

[0112] In one possible implementation, the voice assistant can extract one or more sets of keywords from the user's request information; then, based on the one or more sets of keywords, it can determine whether the local system can generate operation information corresponding to the user's request information.

[0113] For example, when each set of keywords includes the name of a specific device and the name of a specific operation, it can be determined that the local system can generate operation information corresponding to the user's request information. For instance, if the user's request information is "turn on the living room light," and the extracted set of keywords is "turn on" + "living room light," it can be determined that the local system can generate operation information corresponding to the user's request information.

[0114] For example, when any set of keywords does not include the name of a specific device or the name of a specific operation, the voice assistant can call the first AI model to process the user's request information. Subsequently, based on the output of the first AI model, it determines whether the local system can generate the operation information corresponding to the user's request information. For example, if the user's request information is "turn on home theater mode", the extracted set of keywords is "turn on" + "home theater mode". Since the keywords do not include the name of a specific device, "turn on home theater mode" can be input into the first AI model. Based on the output of the first AI model, it determines whether the local system can generate the operation information corresponding to the user's request information (the specific process will be explained later).

[0115] In one implementation method, the voice assistant can directly input user demand information into the first AI model, and determine whether the local system can generate operation information corresponding to the user demand information based on the output of the first AI model (the specific process will be explained later).

[0116] It should be understood that this application may also use other methods to determine whether the local system can generate user intent corresponding to user demand information.

[0117] For example, if the voice assistant determines that it cannot generate operation information corresponding to the user's request information locally, the voice assistant can request the model service platform to identify the user's intent corresponding to the user's request information, that is, the operation information is generated by the model service platform.

[0118] For example, the voice assistant can first generate a device profile of the IoT device based on the device information obtained from the IoT platform. Then, the voice assistant can use user request information and the IoT device profile to generate first instruction information. This first instruction information can be used to instruct the model service platform to generate operation information.

[0119] The "IoT devices" in "device information of IoT devices obtained from the IoT platform" can refer to all IoT devices included in the current IoT application scenario. For example, if the current IoT application scenario is a smart home scenario and the current IoT application scenario is a home, then the voice assistant in the set-top box 110 can obtain device information of all IoT devices included in that home from the IoT platform. Similarly, if the current IoT application scenario is a smart agriculture scenario and the current IoT application scenario is a farm greenhouse, then the voice assistant in the monitoring robot 110 can obtain device information of all IoT devices included in that farm greenhouse from the IoT platform.

[0120] For example, when an IoT device is used for the first time, it can register on the IoT platform. During this process, the IoT device can send its raw device information to the IoT platform for storage. This raw device information may include the IoT device's name, brand, location, model, software information, hardware information, etc. For example, after obtaining the raw device information, the IoT platform can also generate and store additional device information based on it. This additional device information may include the IoT device's name, location, and the names and parameters (optional) of the operations supported by the IoT device. The IoT platform can determine the name and parameters (optional) of the original operations supported by the IoT device based on the IoT device's brand, model, software information, and hardware information. Then, according to preset rules (set with the goal of decoupling the operations supported by the IoT device from the IoT device's brand, model, software information, and hardware information), it can adjust the name and parameters (optional) of the original operations supported by each IoT device to obtain the name and parameters (optional) of the operations supported by each IoT device. Finally, it can generate the conversion / mapping relationship between the original operations supported by the IoT device and the operations supported by the IoT device.

[0121] For example, if the original operation name supported by air conditioner brand 1 is "increase fan speed", the original operation name supported by air conditioner brand 2 is "increase fan speed", and the original operation name supported by air conditioner brand 3 is "increase fan speed", then according to the preset rules, after adjusting the original operation names supported by these three brands of air conditioners, the name of the operation supported by these three brands of air conditioners will be "increase fan speed".

[0122] For example, the original operation name supported by air conditioner brand 1 is "Set Fan Speed", with parameters "0" - low fan speed, "1" - medium fan speed, and "2" - high fan speed; the original operation name supported by air conditioner brand 2 is "Set Fan Speed", with parameters "0" - high fan speed, "1" - medium fan speed, and "2" - low fan speed; the original operation name supported by air conditioner brand 3 is "Set Fan Speed", with parameters "0" - low fan speed, "1" - low-medium fan speed, "2" - medium fan speed, "3" - medium-high fan speed, "4" - high fan speed, and "5" - strong fan speed. According to preset rules, after adjusting the parameters of the original operation supported by these three brands of air conditioners, the operation name supported by these three brands of air conditioners is now "Set Fan Speed", with parameters "0" - low fan speed, "1" - low-medium fan speed, "2" - medium fan speed, "3" - medium-high fan speed, "4" - high fan speed, and "5" - strong fan speed.

[0123] In this way, the name and optional parameters of the operations supported by each IoT device are independent of the IoT device's brand, model, software information, and hardware information. Consequently, when a user inputs their needs, they don't need to consider the IoT device's brand, model, software information, or hardware information, thus reducing the difficulty of interaction between the user and the set-top box 110. Furthermore, the first AI model of the terminal device 110 and the second AI model of the model service platform 120 also don't need to consider the IoT device's brand, model, software information, or hardware information when generating operation information, thus reducing the complexity of generating the operation information corresponding to the user's needs.

[0124] For example, the description information of the operations supported by the IoT device may include the name of the operation supported by the IoT device. Optionally, the description information of the operations supported by the IoT device may also include the parameters of the operation supported by the IoT device.

[0125] In one possible implementation, the device information of the IoT device can be used to determine the device profile of the IoT device.

[0126] For example, the device profile of an IoT device includes: the name of the IoT device, the location of the IoT device, and the names and parameters (optional) of the operations supported by the IoT device.

[0127] For example, the device profile of an IoT device can be shown in Table 1:

[0128] Table 1

[0129]

[0130]

[0131] One possible implementation is to filter the device information of IoT devices to obtain a device profile of the IoT devices; this will be explained in detail later.

[0132] For example, "unable to generate operation information corresponding to the user's request information" may include "unable to generate all or part of the operation information corresponding to the user's request information".

[0133] S203, send first instruction information to the model service platform. The first instruction information is used to instruct the model service platform to generate operation information. The operation information is used to instruct the IoT platform to generate control instructions. The control instructions are used to instruct the target IoT device to perform the target operation.

[0134] Subsequently, the voice assistant can send a first instruction message to the model service platform 120 through the first agent module 112. After receiving the first instruction message, the model service platform 120 can call the second AI model to generate operation information corresponding to the user's needs based on the first instruction message.

[0135] For example, the model service platform 120 can generate one or more operation information corresponding to the user's requirement information. Each operation information may include: the name of the target IoT device, the target location of the target IoT device, the name and parameters (optional) of the target operation to be performed by the target IoT device.

[0136] In one possible implementation, the model service platform 120 can directly send the generated operation information to the IoT platform 130. After receiving the operation information, the IoT platform 130 can generate corresponding control commands based on the received operation information; one operation information can be used to generate one control command. Subsequently, the IoT platform 130 can send each control command to the corresponding IoT device (i.e., the target IoT device), and the target IoT device will execute the target operation corresponding to the received control command.

[0137] In one possible implementation, the model service platform 120 can send the generated operation information to the set-top box 110. The voice assistant can then receive the operation information sent by the model service platform 120 through the first proxy module 112. The voice assistant can then send the operation information to the IoT platform 130 through the second proxy module 113. Upon receiving the operation information, the IoT platform 130 can generate corresponding control commands based on the received information. Subsequently, the IoT platform 130 can send each control command to the corresponding target IoT device, which will then execute the target operation corresponding to the received control command.

[0138] For example, when operation information corresponding to user needs can be generated, the voice assistant can send the generated operation information to the IoT platform 130 through the second agent module 113. After receiving the operation information, the IoT platform 130 can generate corresponding control commands based on the received operation information. Subsequently, the IoT platform 130 can send each control command to the corresponding target IoT device, and the target IoT device will execute the target operation corresponding to the received control command.

[0139] In summary, this application addresses the issue of the edge (e.g., set-top box) being unable to identify all or part of the user's intent, by requesting the cloud (e.g., a model service platform) to identify all or part of the user's intent. This employs an edge-cloud collaborative approach to achieve user intent recognition. This solves the problem in existing technologies where the edge alone cannot identify complex or ambiguous intents, ensuring accurate control of IoT devices. Conversely, if the edge (e.g., set-top box) can identify the user's intent, there is no need to send requests to the cloud, reducing the number of requests and bandwidth pressure. This, in turn, reduces information transmission latency and ensures information transmission reliability, thus mitigating the low reliability and long latency issues associated with cloud-based user intent recognition to some extent.

[0140] Secondly, the device profile of an IoT device includes descriptive information about the operations supported by the IoT device. This operational information is generated based on the device profile. Therefore, the operational information is independent of the IoT device's brand, model, software, or hardware information. In other words, the terminal device and model service platform do not need to consider the IoT device's brand, model, software, or hardware information when generating operational information. This reduces the complexity of generating operational information corresponding to user needs using the first AI model of the terminal device 110 and / or the second AI model of the model service platform 120. Furthermore, when users input user needs information, they do not need to consider the IoT device's brand, model, software, or hardware information, thus reducing the difficulty of user interaction with the set-top box 110. In addition, the control of IoT devices in this application is not limited by the IoT device's brand, model, software, or hardware, and has universal applicability.

[0141] The following section will provide a more detailed explanation of the IoT control method of this application through the interaction process between the various devices included in the control system of the IoT device.

[0142] For example, the inability of terminal device 110 to generate operation information corresponding to user request information may include: terminal device 110 failing to generate all operation information corresponding to user request information, or terminal device 110 failing to generate some operation information corresponding to user request information. The interaction process corresponding to the case where terminal device 110 fails to generate all operation information corresponding to user request information differs from the interaction process corresponding to the case where terminal device 110 fails to generate some operation information corresponding to user request information. This will be explained below using the control process 300 and control process 400 of the IoT device.

[0143] Figure 3 This is a schematic diagram of another control process 300 for an IoT device according to an embodiment of this application. The control process 300 for the IoT device describes the interaction process between various devices included in the control system of the IoT device when the terminal device 110 is unable to generate all the control information corresponding to the user's demand information.

[0144] S301, the terminal device obtains user demand information.

[0145] For example, a user can input their needs via voice; for instance, inputting the voice command "Turn on home theater mode". The voice assistant can then perform ASR (Automatic Speech Recognition) on the user's voice to obtain the user's desired information, i.e., the text "Turn on home theater mode".

[0146] S302, the terminal device identifies the current scene based on user demand information.

[0147] For example, the first AI model may include multiple sub-models, such as a scene classification sub-model, an identity recognition sub-model, and an operation generation sub-model (hereinafter referred to as the first operation generation sub-model for ease of distinction and description). It should be understood that the first AI model may also include other sub-models such as a speech recognition sub-model (for ASR), a gesture recognition sub-model (for gesture recognition), etc., and this application does not impose any limitations on this.

[0148] For example, the output of the scene classification sub-model can include three values: "0", "1", and "2". "0" represents an IoT device control scene, "1" represents a closed-circuit television scene, and "2" represents a network television scene.

[0149] For example, the output of the identity recognition sub-model may include four values: "0", "1", "2", and "3". "0" represents a family member (adult), "1" represents a family member (child / teenager), "2" represents a family member (elderly), and "3" represents a non-family member.

[0150] For example, the output of the first operation generation sub-model may include two: "NULL" and "Name of the target IoT device + Target location of the target IoT device + Name of the target operation of the target IoT device + Parameters of the target operation of the target IoT device (optional)". "NULL" indicates that operation information cannot be generated, while "Name of the target IoT device + Target location of the target IoT device + Name of the target operation to be performed by the target IoT device + Parameters of the target operation to be performed by the target IoT device (optional)" indicates that some or all of the operation information can be generated.

[0151] For example, the voice assistant can input user request information into the scene classification sub-model, and then determine the current scene based on the output of the scene classification sub-model. For instance, when the scene classification sub-model outputs "0", the current scene can be determined to be an IoT device control scene; when the scene classification sub-model outputs "1", the current scene can be determined to be a closed-circuit television scene; and when the scene classification sub-model outputs "2", the current scene can be determined to be a network television scene.

[0152] S303, terminal device identifies user identity.

[0153] For example, a voice assistant can input a user's voice into an identity recognition sub-model, and then determine the user's identity based on the output of the identity recognition sub-model. For instance, when the identity recognition sub-model outputs "0", the user's identity can be determined to be a family member (adult); when the identity recognition sub-model outputs "1", the user's identity can be determined to be a family member (child / teenager); when the identity recognition sub-model outputs "2", the user's identity can be determined to be a family member (elderly); and when the identity recognition sub-model outputs "3", the user's identity can be determined to be a non-family member.

[0154] It should be noted that this application does not restrict the execution order of S302 and S303, and both S302 and S303 are executed after S301.

[0155] S304, the terminal device sends a third instruction message to the IoT platform.

[0156] For example, the voice assistant can use the device information of the terminal device (such as the name of the terminal device, the identifier of the terminal device, the location of the terminal device, etc.) to generate third instruction information. Then, the voice assistant can send the third instruction information to the IoT platform 130 through the second agent module 113; the third instruction information is used to instruct the IoT platform 130 to return the device information of the IoT device.

[0157] S305, the IoT platform sends device information of IoT devices to terminal devices.

[0158] It should be noted that this application does not restrict whether S304 and S305 are executed before or after S301.

[0159] S306, The terminal device generates a device profile of the IoT device corresponding to the user's identity based on the device information of the IoT device.

[0160] For example, the voice assistant can obtain device information of IoT devices through the second agent module 113.

[0161] For example, the device profile of an IoT device corresponding to a user identity can refer to the device profile of an IoT device for which the user identity has control permissions.

[0162] In one possible implementation, the device profile of the IoT device corresponding to the user's identity may include: descriptive information of all operations supported by the IoT device for which the user has control authority.

[0163] For example, the device information of IoT devices is shown in Table 1. If the user's identity is a family member (child / teenager), the generated device profile of the IoT devices corresponding to the family member (child / teenager) can be shown in Table 2.

[0164] Table 2

[0165]

[0166] The first column in Table 2 represents the optional items.

[0167] In one possible implementation, the device profile of the IoT device corresponding to the user's identity may include: descriptive information of some operations supported by the IoT device that the user has control over (operations that the user has control over).

[0168] For example, the device information of IoT devices is shown in Table 1. If the user's identity is a family member (child / teenager), the generated device profile of the IoT devices corresponding to the family member (child / teenager) can be shown in Table 3.

[0169] Table 3

[0170]

[0171] The first column in Table 3 represents optional items.

[0172] For example, when the user's identity is a family member (adult), the device profile of the IoT device corresponding to the user's identity can be shown in Table 1.

[0173] Afterwards, the terminal device can determine whether it can generate all the operation information corresponding to the user's required information.

[0174] In one possible implementation, the voice assistant can analyze the control complexity of the user's request information for the IoT device (i.e., the target IoT device). Then, the voice assistant can determine whether the control complexity of the user's request information for the target IoT device exceeds a threshold. If the control complexity of the user's request information for the target IoT device exceeds the threshold, it is determined that all operation information corresponding to the user's request information cannot be generated. If the control complexity of the user's request information for the target IoT device is less than or equal to the threshold, it is determined that all operation information corresponding to the user's request information can be generated. For example, the control complexity may include the target number of operations for the target IoT device and / or the number of target IoT devices; the threshold may include a first sub-threshold and a second sub-threshold. When the number of target IoT devices exceeds the first sub-threshold, or the target number of operations for the target IoT device exceeds the second sub-threshold, it is determined that the control complexity of the user's request information for the target IoT device exceeds the threshold. This application does not limit the meaning of "threshold," "first sub-threshold," and "second sub-threshold," how they are set, or whether they can be adjusted, etc.

[0175] In one possible implementation, the voice assistant can extract one or more sets of keywords from the user's request information. Then, based on these keywords, it determines whether all operation information corresponding to the user's request information can be generated. For example, when each set of keywords includes the name of a specific device and the name of a specific operation, it can be determined that all operation information corresponding to the user's request information can be generated. When any set of keywords does not include the name of a specific device and the name of a specific operation, the voice assistant can call a first operation generation sub-model to process the user's request information. Subsequently, based on the output of the first operation generation sub-model, it determines whether all operation information corresponding to the user's request information can be generated (the specific process can be referred to the description in the next implementation). If the first operation generation sub-model outputs "NULL", it is determined that all operation information corresponding to the user's request information cannot be generated. If the first operation generation sub-model outputs "name of the target IoT device + target location of the target IoT device + name of the target operation of the target IoT device + parameters of the target operation of the target IoT device (optional)", the voice assistant can analyze the control complexity of the user's request information for the IoT device (i.e., the target IoT device); where the control complexity may include the number of target operations of the target IoT device. Then, the voice assistant can compare the number of operation information entries output by the first operation generation sub-model with the target number of operations for the target IoT device. If the number of operation information entries output by the first operation generation sub-model is equal to the target number of operations for the target IoT device, it is determined that all operation information corresponding to the user's needs can be generated; if the number of operation information entries output by the first operation generation sub-model is less than the target number of operations for the target IoT device, it is determined that all operation information corresponding to the user's needs cannot be generated.

[0176] In one possible implementation, the voice assistant can first use user request information, a device profile of the IoT device corresponding to the user's identity, and preset prompts corresponding to the first operation generation sub-model to generate prompts for the first operation generation sub-model. For example, the preset prompts for the first operation generation sub-model might be "Based on _(user request information) + _(device profile), return operation information (name of target IoT device + target location of target IoT device + name of target operation of target IoT device + parameters of target operation (optional))"; the generated prompts for the first operation generation sub-model could be "Based on (open home theater mode) + (Table 1), return operation information (name of target IoT device + target location of target IoT device + name of target operation of target IoT device + parameters of target operation (optional))". Then, the voice assistant can input the prompts for the first operation generation sub-model to instruct it to generate operation information. For example, when the first operation generation sub-model outputs "NULL", it can be determined that the first operation generation sub-model cannot generate all operation control information. For example, when the first operation generation sub-model outputs "name of the target IoT device + target location of the target IoT device + name of the target operation of the target IoT device + parameters of the target operation of the target IoT device (optional)," the voice assistant can analyze the control complexity of the user's request information for the IoT device (i.e., the target IoT device); wherein, the control complexity may include the number of target operations of the target IoT device. Then, the voice assistant can compare the number of operation information entries output by the first operation generation sub-model with the number of target operations of the target IoT device. If the number of operation information entries output by the first operation generation sub-model equals the number of target operations of the target IoT device, it is determined that all operation information corresponding to the user's request information can be generated; if the number of operation information entries output by the first operation generation sub-model is less than the number of target operations of the target IoT device, it is determined that all operation information corresponding to the user's request information cannot be generated.

[0177] In addition, the terminal device can determine whether the current scenario is an IoT device control scenario.

[0178] For example, if the scene classification sub-model outputs "0", then the current scene is determined to be an IoT device control scene.

[0179] For example, if the voice assistant determines that the current scenario is an IoT device control scenario and cannot generate all the operation information corresponding to the user's needs, it can execute S307; if the current scenario is an IoT device control scenario and can generate all the operation information corresponding to the user's needs, it can execute S313; otherwise, the process ends.

[0180] S307, In the case that the current scenario is an IoT device control scenario and it is impossible to generate all the operation information corresponding to the user's needs, the terminal device generates model prompt words based on the user's needs, the current scenario and the device profile of all IoT devices corresponding to the user's identity.

[0181] For example, when the voice assistant determines that the current scenario is an IoT device control scenario and cannot generate all the operation information corresponding to the user's needs, it can generate model prompts based on the user's needs, the current scenario, and the device profiles of all IoT devices corresponding to the user's identity. The second AI model may also include an operation generation sub-model (hereinafter referred to as the second operation generation sub-model), and the model prompts may refer to the prompts of the second operation generation sub-model.

[0182] For example, the voice assistant can first use user demand information, the current scene, and device profiles of all IoT devices corresponding to the user's identity, as well as preset prompts corresponding to the second operation generation sub-model, to generate prompts for the second operation generation sub-model. For instance, the preset prompts for the second operation generation sub-model could be "Based on _(user demand information) + _(current scene) + _(device profile), return operation information (name of target IoT device + target location of target IoT device + name of target operation of target IoT device + parameters of target operation (optional))"; the generated prompts for the second operation generation sub-model could be "Based on (open home theater mode) + (IoT device control scene) + (Table 1), return operation information (name of target IoT device + target location of target IoT device + name of target operation of target IoT device + parameters of target operation (optional))"

[0183] S308, the terminal device generates the first instruction information based on the model prompt words.

[0184] For example, the voice assistant can encapsulate model prompt words according to the communication protocol between the set-top box 110 and the model service platform 120 to obtain first instruction information. The first instruction information includes the model prompt words.

[0185] S309, the terminal device sends the first instruction information to the model service platform.

[0186] S310, the model service platform generates all operation information corresponding to the user's requirement information based on the first instruction information.

[0187] For example, if the current season is summer and the device profiles of IoT devices corresponding to user identities are shown in Table 1, then the operation information generated by the model service platform 120 may include:

[0188] Operation information 1: Air conditioner + living room + turn on air conditioner

[0189] Operation information 2: Air conditioner + living room + setting mode + "1"

[0190] Operation information 3: Air conditioner + living room + set temperature + "26℃"

[0191] Operation information 4: Curtains + Living Room + Close Curtains

[0192] Operation information 5: Spotlights + Living room + Turn off spotlights

[0193] Operation information 6: Recessed light + Living room + Turn off recessed light

[0194] Operation information 7: Ambient light + living room + turn on ambient light

[0195] Operation Information 8: Stereo Speakers + Living Room + Turn on Stereo Speakers

[0196] S311, the model service platform sends all operation information corresponding to the user's request information to the terminal device.

[0197] For example, after the model service platform 120 encapsulates all the operation information corresponding to the user requirement information, it sends all the encapsulated operation information corresponding to the user requirement information to the set-top box 110.

[0198] S312, the terminal device generates second instruction information based on the operation information received from the model service platform.

[0199] For example, the voice assistant can encapsulate the operation information received from the model service platform (that is, all operation information corresponding to the user demand information) according to the communication protocol between the set-top box 110 and the IoT platform 130 to obtain the second instruction information. The second instruction information includes all operation information corresponding to the user demand information.

[0200] S313, In the current scenario of IoT device control, and when all operation information corresponding to user demand information can be generated, the terminal device generates second instruction information based on the locally generated operation information.

[0201] For example, the voice assistant can encapsulate locally generated (i.e., generated by calling the first operation generation sub-model) operation information (that is, all operation information corresponding to the user demand information) according to the communication protocol between the set-top box 110 and the IoT platform 130 to obtain the second instruction information. The second instruction information includes all operation information corresponding to the user demand information.

[0202] S314, the terminal device sends a second instruction message to the IoT platform.

[0203] S315, the IoT platform generates control commands based on the operation information in the second instruction information.

[0204] For example, the IoT platform 130 can generate a control instruction based on each piece of operation information in the second instruction information. Each control instruction may include the identifier of the target IoT device and the name of the target operation to be performed by the target IoT device.

[0205] S316, the IoT platform sends control commands to the target IoT device.

[0206] For example, for each control command, the IoT platform 130 can determine the address of the target IoT device based on the identifier of the target IoT device in the control command; then, the control command can be sent to the target IoT device based on the address of the target IoT device.

[0207] S317, the IoT platform sends the first prompt message to the terminal device.

[0208] For example, the first prompt message is used to remind the user that the IoT device should start performing the operation corresponding to the user's requested information.

[0209] It should be noted that this application does not restrict the execution order of S317, S315, and S316.

[0210] S318, the terminal device outputs the first prompt message.

[0211] For example, if the first prompt is a voice message, the voice assistant can use the speaker to read it aloud. If the first prompt is a text message, the voice assistant can use the display interface to show it.

[0212] S319, the target IoT device executes the target operation corresponding to the control command.

[0213] For example, after receiving a control command, any target IoT device can execute the target operation corresponding to the name of the operation carried by the control command.

[0214] S320, the target IoT device sends the execution result to the IoT platform.

[0215] For example, after any target IoT device completes the target operation, it can send the execution result to the IoT platform 130. The execution result may include whether the execution was successful or failed.

[0216] S321, the IoT platform generates a second prompt message based on the execution result.

[0217] For example, after receiving the execution result sent by the IoT device, the IoT platform 130 can generate a second prompt message based on the execution result; the second prompt message is used to remind the user whether the IoT device has executed successfully or failed.

[0218] S322, the IoT platform sends a second prompt message to the terminal device.

[0219] S323, the terminal device outputs a second prompt message.

[0220] For example, S323 can be described with reference to the above description of S318, and will not be repeated here.

[0221] For example, steps S317, S318, and S320 to S323 are optional.

[0222] In the IoT device control process 300, firstly, a device profile of the IoT device corresponding to the user's identity is generated; then, operation information is generated based on the device profile of the IoT device corresponding to the user's identity. This ensures that each user can only control IoT devices for which they have control authority, thereby achieving personalization and security for IoT device control. This approach not only improves the user experience but also enhances the security and accuracy of device management.

[0223] Furthermore, this application uses terminal devices for identity verification, eliminating the need for terminal devices to send user personal information such as voiceprints, fingerprints, and facial features to the model service platform, thus ensuring the security of user personal information.

[0224] Furthermore, terminal devices and IoT devices will not upload other personal data (such as personal health data, audio and video data of users in home scenarios, device usage data, etc.) to the model service platform or IoT platform; this can ensure the security of personal information.

[0225] Figure 4 This is a schematic diagram of another control process 400 for an IoT device according to an embodiment of this application. The timing of the IoT device control process 400 and the timing of the terminal device obtaining the device information of the IoT device from the IoT platform and generating the device profile of the IoT device in the IoT device control process 300 are different; and in the IoT device control process 400, the terminal device generates a part of the operation information corresponding to the user demand information, while the model service platform generates another part of the operation information corresponding to the user demand information.

[0226] S401, the terminal device sends a third instruction message to the IoT platform.

[0227] S402, the IoT platform sends device information of IoT devices to terminal devices.

[0228] S403, The terminal device generates a device profile of the IoT device based on the device information of the IoT device.

[0229] For example, S401 to S403 can be described with reference to the above description of S304 to S306, and will not be repeated here.

[0230] S404, The terminal device obtains user request information.

[0231] For example, a user can input their needs via voice; for instance, inputting the voice command "Make the living room atmosphere more comfortable." The terminal device's processing module can then perform ASR (Automatic Speech Recognition) on the user's voice to obtain the user's needs, i.e., the text "Make the living room atmosphere more comfortable."

[0232] S405: The terminal device identifies the current scenario based on user demand information.

[0233] S406, terminal devices identify user identity.

[0234] For example, S404 to S406 can be described with reference to the above description of S301 to S303, and will not be repeated here.

[0235] S407, when the current scenario is an IoT device control scenario and the terminal device cannot generate partial operation information corresponding to the user's needs, it generates model prompt words based on the user's needs, the current scenario, and the device profile of some IoT devices corresponding to the user's identity.

[0236] For example, the way the terminal device determines whether the current scene is an IoT device control scene in the control process 400 of the IoT device can refer to the way the terminal device determines whether the current scene is an IoT device control scene in the control process 300 of the IoT device described above, and will not be repeated here.

[0237] For example, in the control process 400 of the IoT device, the process by which the terminal device determines whether it can generate partial operation information corresponding to the user's request information can be as follows: The voice assistant can first use the user's request information, the device profile of the IoT device corresponding to the user's identity, and the preset prompts corresponding to the first operation generation sub-model to generate the prompts for the first operation generation sub-model. For example, the preset prompts for the first operation generation sub-model are "Based on _(user request information) + _(device profile), return operation information (name of target IoT device + target location of target IoT device + name of target operation of target IoT device + parameters of target operation (optional))"; the generated prompts for the first operation generation sub-model can be "Based on (open home theater mode) + (Table 1), return operation information (name of target IoT device + target location of target IoT device + name of target operation of target IoT device + parameters of target operation (optional))". Afterwards, the voice assistant can input the prompts for the first operation generation sub-model into the first operation generation sub-model to instruct the first operation generation sub-model to generate the operation information. For example, when the first operation generation sub-model outputs "name of the target IoT device + target location of the target IoT device + name of the target operation of the target IoT device + parameters of the target operation of the target IoT device (optional)," the voice assistant can analyze the control complexity of the user's request information for the IoT device (i.e., the target IoT device); wherein, the control complexity may include the number of target operations of the target IoT device. Then, the voice assistant can compare the number of operation information items output by the first operation generation sub-model with the number of target operations of the target IoT device. If the number of operation information items output by the first operation generation sub-model is less than the number of target operations of the target IoT device, it is determined that the partial operation information corresponding to the user's request information cannot be generated.

[0238] For example, in a scenario where the current situation is an IoT device control scenario and partial operation information corresponding to the user's needs cannot be generated, the process by which the terminal device generates model prompts based on the user's needs, the current scenario, and the device profiles of some IoT devices corresponding to the user's identity can be as follows: The voice assistant can determine the IoT device corresponding to the second part of the operation information generated locally; then, from the device profiles of all IoT devices corresponding to the user's identity, the device profile of the IoT device corresponding to the second part of the operation information generated locally is removed to obtain the device profile of the partial IoT device corresponding to the user's identity. Then, referring to the description in S307, model prompts can be generated based on the user's needs, the current scenario, and the device profiles of the partial IoT devices corresponding to the user's identity. For example: Suppose the device profiles of all IoT devices corresponding to the user's identity are shown in Table 1. If the IoT device corresponding to the second part of the operation information generated locally is an ambient light, then the device profile of the partial IoT devices corresponding to the user's identity can be Table 1 excluding the device profile of "ambient light".

[0239] Optionally, if one or more sets of keywords can be extracted from the user's request information, the voice assistant can also remove one or more sets of keywords from the user's request information that correspond to the second part of the operation information generated locally. Then, based on the user's request information after keyword removal, the current scene, and the device profiles of some IoT devices corresponding to the user's identity, model prompts are generated. For example, if the user's request information is "turn off spotlights, turn off downlights, turn on ambient lights, close curtains", and if one set of keywords corresponding to the second part of the operation information generated locally is "turn on" + "ambient lights", and the other set of keywords is "turn off" + "curtains", then the keywords "turn on", "ambient lights", "turn off", and "curtains" can be removed from the user's request information, resulting in the user's request information after keyword removal as "turn off spotlights, turn off downlights".

[0240] S408, the terminal device generates the first instruction information based on the model prompt words.

[0241] S409, the terminal device sends the first instruction information to the model service platform.

[0242] S410, the model service platform generates the first part of the operation information corresponding to the user's requirement information based on the first instruction information.

[0243] For example, if the user's requirement is "to make the living room atmosphere more comfortable," and the device profiles of some IoT devices corresponding to the user's identity are shown in Table 1 (excluding "ambiance lights"), then the first part of the operation information generated by the model service platform may include:

[0244] Operation information 1: Spotlights + Living room + Turn off spotlights

[0245] Operation information 2: Recessed light + Living room + Turn off recessed light

[0246] Operation message 3: Spotlights + Dining room + Turn off spotlights

[0247] Operation information 4: Recessed light + Dining room + Turn off recessed light

[0248] Operation message 5: Spotlights + Master bedroom + Turn off spotlights

[0249] Operation message 6: Recessed light + Master bedroom + Turn off recessed light

[0250] Operation information 7: Spotlight + Second bedroom + Turn off spotlight

[0251] Operation information 8: Recessed light + secondary bedroom + turn off recessed light

[0252] S411, the model service platform sends the first part of the operation information corresponding to the user's request information to the terminal device.

[0253] For example, after the model service platform encapsulates all the operation information corresponding to the user's requirement information, it sends the encapsulated operation information corresponding to the user's requirement information to the terminal device.

[0254] S412, The terminal device obtains the second part of the operation information corresponding to the user demand information generated locally.

[0255] The voice assistant can generate the second part of the operation information by calling the first operation generation sub-model as follows:

[0256] Operation Information 1: Curtains + Living Room + Close Curtains

[0257] Operation information 2: Ambient light + living room + turn on ambient light

[0258] S413, the terminal device generates second instruction information based on the received first part of operation information and the locally generated second part of operation information.

[0259] For example, the voice assistant can fuse (or merge) the first part of the operation information and the second part of the operation information to obtain all the operation information corresponding to the user's demand information; then, based on all the operation information corresponding to the user's demand information, a second instruction information is generated; the description of S312 above can be referred to, and will not be repeated here.

[0260] S414, the terminal device sends a second instruction message to the IoT platform.

[0261] S415, the IOT platform generates control commands based on the operation information in the second instruction information.

[0262] S416, the IoT platform sends control commands to the target IoT device.

[0263] S417, the IoT platform sends the first notification information to the terminal device.

[0264] S418, the terminal device outputs the first prompt message.

[0265] S419, the target IoT device executes the target operation corresponding to the control command.

[0266] S420, the target IoT device sends the execution result to the IoT platform.

[0267] S421, the IoT platform generates a second prompt message based on the execution result.

[0268] S422, the IoT sends a second prompt message to the terminal device.

[0269] S423, the terminal device outputs a second prompt message.

[0270] For example, S414 to S423 can be referred to the description of S314 to S323 above, and will not be repeated here.

[0271] Compared to the control process 300 of the IoT device, the control process 400 of the IoT device generates some operation information from the edge and another part of operation information from the cloud. In this way, the amount of data exchanged between the edge and the cloud can be reduced, thereby further reducing the control latency of the IoT device and improving the user experience.

[0272] It should be understood that after the set-top box generates the first part of the operation information locally, it can first send this first part of the operation information to the IoT platform; after receiving the second part of the operation information from the model service platform, it then sends the second part of the operation information to the IoT platform. In this way, the IoT platform can first generate the corresponding control command based on the first part of the operation information, and then send the control command to the target IoT device for the target IoT device to execute the target operation; this can also reduce the control latency of IoT devices and improve the user experience.

[0273] Figure 5 This is a schematic diagram of a control device 500 for an Internet of Things (IoT) device according to an embodiment of this application. The control device 500 for the IoT device can be used to execute the methods of the foregoing embodiments. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0274] For example, the control device 500 of the Internet of Things (IoT) device may include:

[0275] The requirement acquisition module 501 is used to acquire user requirement information;

[0276] The instruction generation module 502 is used to generate first instruction information based on the user demand information and the device profile of the IoT device when it is determined that the operation information corresponding to the user demand information cannot be generated. The device profile of the IoT device includes description information of the operations supported by the IoT device.

[0277] The communication module 503 is used to send first instruction information to the model service platform. The first instruction information is used to instruct the model service platform to generate operation information. The operation information is used to instruct the IoT platform to generate control commands. The control commands are used to instruct the target IoT device to perform the target operation.

[0278] The demand acquisition module 501 and the instruction generation module 502 can be deployed in the processing module 111 of the terminal device 110, and the communication module 503 can include the first agent module 112 and the second agent module 113 of the terminal device 110.

[0279] For example, the description information of the operations supported by the IoT device includes the name of the operations supported by the IoT device, and the device profile of the IoT device also includes: the name of the IoT device and the location of the IoT device.

[0280] For example, the description information of the operations supported by the IoT device may also include parameters of the operations supported by the IoT device.

[0281] For example, the operation information includes one or more pieces, each piece of operation information including: the name of the target IoT device, the target location of the target IoT device, and the name of the target operation that the target IoT device needs to perform.

[0282] For example, each operation information also includes parameters of the target operation that the target IoT device needs to perform.

[0283] Exemplarily, the device further includes:

[0284] The identity recognition module is used to identify the user's identity;

[0285] The instruction generation module 502 is used to generate first instruction information based on the user's requirement information and the device profile of the IoT device corresponding to the user's identity when it is determined that operation information corresponding to the user's requirement information cannot be generated.

[0286] The identity recognition module can be deployed in the processing module 111.

[0287] For example, the instruction generation module 502 is used to generate first instruction information based on the user demand information and the device profile of all IoT devices when it is determined that all operation information corresponding to the user demand information cannot be generated; and to generate first instruction information based on the user demand information and the device profile of some IoT devices when it is determined that some operation information corresponding to the user demand information cannot be generated. The some IoT devices are IoT devices other than the IoT devices corresponding to the operation information generated by the terminal device among all IoT devices.

[0288] Exemplarily, the device further includes:

[0289] The scene recognition module is used to identify the current scene based on user needs information;

[0290] The instruction generation module 502 is used to generate first instruction information based on user demand information, the current scenario, and the device profile of the IoT device when it is determined that the current scenario is an IoT device control scenario and it is determined that operation information corresponding to user demand information cannot be generated.

[0291] The scene recognition module can be deployed in the processing module 111.

[0292] Exemplarily, the device further includes:

[0293] The judgment module is used to determine whether the control complexity of the user's request information for the IoT device is greater than a threshold; if the control complexity is greater than the threshold, it is determined that the operation information corresponding to the user's request information cannot be generated.

[0294] The judgment module can be deployed in the processing module 111.

[0295] For example, the control complexity includes the number of target IoT devices and / or the number of target operations of the target IoT devices, and the threshold includes a first sub-threshold and a second sub-threshold; when the number of target IoT devices is greater than the first sub-threshold, and / or the number of target operations of the target IoT devices is greater than the second sub-threshold, the control complexity is determined to be greater than the threshold.

[0296] For example, the communication module 503 is also used to receive operation information sent by the model service platform; and to send second instruction information to the Internet of Things platform, wherein the second instruction information is generated based on the operation information corresponding to the user demand information.

[0297] In one example, Figure 6 The schematic block diagram illustrating an embodiment of the present application shows an apparatus 600. The apparatus 600 may include a processor 601 and a transceiver 602, and optionally, a memory 603.

[0298] The various components of device 600 are coupled together via bus 604, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 604 in the figure.

[0299] Optionally, the memory 603 can be used to store instructions from the foregoing method embodiments. The processor 601 can be used to execute the instructions in the memory 603, and to control the transceiver 602 to receive signals and transmit signals.

[0300] The device 600 may be an electronic device or a chip of an electronic device as described in the above method embodiments. The electronic device may be a terminal device or a server.

[0301] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0302] This application also provides a chip, including one or more interface circuits and one or more processors; the one or more processors receive or transmit data through the one or more interface circuits, and when the one or more processors execute computer instructions, the steps of the above-described related method steps that implement the method in the above embodiments are executed. The interface circuit is a transceiver 602.

[0303] This application also provides a computer-readable storage medium storing computer instructions. When these computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the methods described in the above embodiments. Exemplarily, the computer-readable storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0304] This application also provides a computer program product comprising computer instructions that, when executed by a computer or processor, cause the computer to perform the aforementioned steps to implement the methods described in the above embodiments. Exemplarily, the computer program product may be stored in random access memory (RAM), flash memory, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, read-only optical discs (CD-ROMs), or any other form of storage medium known in the art.

[0305] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0306] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0307] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0308] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0309] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0310] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A control method for an Internet of Things (IoT) device, characterized in that, Applied to a terminal device, the method includes: Obtain user demand information; If it is determined that the operation information corresponding to the user demand information cannot be generated, a first instruction information is generated based on the user demand information and the device profile of the IoT device. The device profile of the IoT device includes a description of the operations supported by the IoT device. The first instruction information is sent to the model service platform. The first instruction information is used to instruct the model service platform to generate the operation information. The operation information is used to instruct the IoT platform to generate control instructions. The control instructions are used to instruct the target IoT device to perform the target operation.

2. The method according to claim 1, characterized in that, The description information of the operations supported by the IoT device includes the name of the operations supported by the IoT device, and the device profile of the IoT device also includes: the name of the IoT device and the location of the IoT device.

3. The method according to claim 2, characterized in that, The description information of the operations supported by the IoT device also includes the parameters of the operations supported by the IoT device.

4. The method according to any one of claims 1 to 3, characterized in that, The operation information includes one or more pieces, each piece of operation information including: the name of the target IoT device, the target location of the target IoT device, and the name of the target operation that the target IoT device needs to perform.

5. The method according to claim 4, characterized in that, Each piece of operation information also includes: parameters of the target operation that the target IoT device needs to perform.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Identify user identity; In the event that it is determined that operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of the IoT device, including: If it is determined that the operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of the IoT device corresponding to the user identity.

7. The method according to any one of claims 1 to 5, characterized in that, In the event that it is determined that operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of the IoT device, including: If it is determined that it is impossible to generate all the operation information corresponding to the user demand information, the first instruction information is generated based on the user demand information and the device profile of all IoT devices. If it is determined that partial operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of partial IoT devices. The partial IoT devices are IoT devices other than the IoT devices corresponding to the operation information generated by the terminal device among all IoT devices.

8. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Based on the user demand information, identify the current scenario; In the event that it is determined that operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of the IoT device, including: If the current scenario is determined to be an IoT device control scenario, and it is determined that the operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information, the current scenario, and the device profile of the IoT device.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Receive operation information sent by the model service platform; Send a second instruction to the IoT platform. The second instruction is generated based on the operation information corresponding to the user demand information.

10. A control system for an Internet of Things (IoT) device, characterized in that, The control system includes: terminal devices, a model service platform, an Internet of Things (IoT) platform, and the IoT devices. The terminal device is used to acquire user demand information; if it is determined that operation information corresponding to the user demand information cannot be generated, it generates first instruction information based on the user demand information and the device profile of the IoT device, wherein the device profile of the IoT device includes description information of the operations supported by the IoT device; and sends the first instruction information to the model service platform. The model service platform is used to generate the operation information based on the first indication information; and to send the operation information to the terminal device. The terminal device is further configured to generate second instruction information based on the operation information; and to send the second instruction information to the Internet of Things platform; The IoT platform is used to generate control commands based on the operation information in the second indication information; and to send the control commands to the target IoT device. The target IoT device is used to execute the target operation corresponding to the control command.

11. A terminal device, characterized in that, The terminal device is used for Obtain user demand information; If it is determined that the operation information corresponding to the user demand information cannot be generated, a first instruction information is generated based on the user demand information and the device profile of the IoT device. The device profile of the IoT device includes a description of the operations supported by the IoT device. The first instruction information is sent to the model service platform. The first instruction information is used to instruct the model service platform to generate the operation information. The operation information is used to instruct the IoT platform to generate control instructions. The control instructions are used to instruct the target IoT device to perform the target operation.

12. The terminal device according to claim 11, characterized in that, The terminal device includes at least one of the following: a set-top box, a smart speaker, or a smart robot.

13. The terminal device according to claim 11 or 12, characterized in that, The terminal device is also used for Receive operation information sent by the model service platform; Send a second instruction to the IoT platform. The second instruction is generated based on the operation information corresponding to the user demand information.

14. The terminal device according to any one of claims 11 to 13, characterized in that, The description information of the operations supported by the IoT device includes the name of the operations supported by the IoT device, and the device profile of the IoT device also includes: the name of the IoT device and the location of the IoT device.

15. The terminal device according to any one of claims 11 to 14, characterized in that, The terminal device is used to identify the user's identity; if it is determined that the operation information corresponding to the user's demand information cannot be generated, the first instruction information is generated based on the user's demand information and the device profile of the IoT device corresponding to the user's identity.

16. The terminal device according to any one of claims 11 to 14, characterized in that, The terminal device is configured to generate the first instruction information based on the user demand information and the device profile of all IoT devices when it is determined that it is impossible to generate all the operation information corresponding to the user demand information. If it is determined that partial operation information corresponding to the user demand information cannot be generated, the first instruction information is generated based on the user demand information and the device profile of partial IoT devices. The partial IoT devices are IoT devices other than the IoT devices corresponding to the operation information generated by the terminal device among all IoT devices.

17. A control device for an Internet of Things (IoT) device, characterized in that, The device includes: The requirement elicitation module is used to obtain user requirement information; The instruction generation module is used to generate first instruction information based on the user demand information and the device profile of the IoT device when it is determined that operation information corresponding to the user demand information cannot be generated. The device profile of the IoT device includes description information of the operations supported by the IoT device. The communication module is used to send the first indication information to the model service platform. The first indication information is used to instruct the model service platform to generate the operation information. The operation information is used to instruct the IoT platform to generate control instructions. The control instructions are used to instruct the target IoT device to perform the target operation.

18. A terminal device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the terminal device to perform the method as described in any one of claims 1 to 9.

19. A chip, characterized in that, It includes one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the steps of the method as described in any one of claims 1 to 9 are performed.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 1 to 9.

21. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a computer or processor, cause the steps of the method as described in any one of claims 1 to 9 to be performed.

Images