Smart home equipment control method and device, equipment, medium and product

Abstract

The invention provides a smart home equipment control method and device, equipment, a medium and a product, and is applied to the technical field of smart home, and the method comprises the steps: obtaining an equipment capability map of smart home equipment in a current smart home environment; determining a target device capability node in the device capability map according to a scene intention input by a user, and determining a target smart home device based on the target device capability node; according to the invention, the scene control script for the target smart home device is generated, and the target smart home device is controlled according to the scene control script, so that the target smart home device corresponding to the user scene intention is accurately matched based on the device capability map, the scene control script is automatically generated, and intelligent control is completed. The operation process of manual configuration of the user is simplified, the operation conflict between the devices can be avoided by means of the device capacity atlas, the stability and flexibility of linkage control of the devices are improved, and then the use experience of the user is optimized.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a method, apparatus, device, medium, and product for controlling smart home devices. Background Technology

[0002] With the development of smart home technology, smart home devices can connect and communicate with each other through networks, enabling interconnection and intelligent control between devices.

[0003] In existing technologies, users can manually configure scene modes, such as adding smart home devices through an app, setting scene trigger conditions and actions, to automatically control smart home devices.

[0004] However, when using the above methods, the manual configuration process is cumbersome, requiring users to add devices and set parameters one by one, which is prone to configuration errors. Furthermore, due to users' lack of understanding of the capabilities of smart home devices, it is difficult to make accurate scene configurations based on the capabilities of the devices, resulting in poor control effects in scene modes, such as device conflicts, failure to achieve the expected control effects, and affecting the user experience. Summary of the Invention

[0005] In view of the above problems, a method, apparatus, device, medium, and product for controlling smart home devices are proposed to overcome or at least partially solve the above problems, including: A method for controlling smart home devices, the method comprising: Obtain a device capability map of smart home devices in the current smart home environment; wherein, the device capability map includes device capability nodes, and each device capability node corresponds to one or more smart home devices; Based on the user's input scenario intent, the target device capability node in the device capability map is determined, and based on the target device capability node, the target smart home device is determined; Generate a scene control script for the target smart home device, and control the target smart home device according to the scene control script.

[0006] Optionally, based on the user's input scenario intent, the target device capability node in the device capability map is determined, including: Determine keywords from the user's input contextual intent; Determine the semantic similarity between the keywords and the device capability nodes in the device capability map; Based on the semantic similarity, the target device capability node in the device capability map is determined.

[0007] Optionally, based on the target device capability node, the target smart home device is determined, including: Based on the target device capability nodes, a list of candidate smart home devices is determined; Obtain user historical behavior data, and determine the target smart home device from the candidate smart home device list based on the user historical behavior data.

[0008] Optionally, based on the user's historical behavior data, the target smart home device is determined from the list of candidate smart home devices, including: Based on the user's historical behavior data, the weights of the candidate smart home devices in the candidate smart home device list are determined, and the weights are dynamically adjusted based on real-time environmental data. Based on the dynamically adjusted weights, the matching degree of candidate smart home devices or combinations of candidate smart home devices is determined, and the target smart home device is determined based on the matching degree.

[0009] Optionally, a scene control script for the target smart home device is generated, including: Based on the scene intent, a target scene control scheme is generated; wherein, the target scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; Based on the scene triggering conditions and action sequence in the target scene control scheme, generate device control instructions, and based on the device control instructions, generate scene control scripts.

[0010] Optionally, based on the scene intent, a target scene control scheme is generated, including: Based on the scene intent, multiple candidate scene control schemes are generated; wherein, each candidate scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; Based on the scene triggering conditions and action sequences in each candidate scene control scheme, an evaluation result of the multiple candidate scene control schemes is generated, and based on the evaluation result, a target scene control scheme is determined from the multiple candidate scene control schemes.

[0011] Optionally, before generating the evaluation results of the multiple candidate scene control schemes based on the scene triggering conditions and action sequences in each candidate scene control scheme, the method further includes: performing conflict detection on the action sequences in each candidate scene control scheme, and adjusting the action sequences in the candidate scene control schemes based on the conflict detection results.

[0012] Optionally, before obtaining the device capability map of smart home devices in the current smart home environment, the following steps are also included: Obtain the instruction manuals for smart home devices and extract key information from them; Based on the key information, construct a device capability map of smart home devices.

[0013] Optionally, after constructing the device capability map of smart home devices based on the key information, the method further includes: establishing a mapping relationship between the device capability nodes in the device capability map and the control primitives in the smart home devices.

[0014] A device for controlling smart home devices, the device comprising: The device capability map acquisition module is used to acquire the device capability map of smart home devices in the current smart home environment; wherein, the device capability map includes device capability nodes, and each device capability node corresponds to one or more smart home devices; The target smart home device determination module is used to determine the target device capability node in the device capability map based on the user's input scene intent, and to determine the target smart home device based on the target device capability node; The scene control script generation and control module is used to generate scene control scripts for the target smart home device and control the target smart home device according to the scene control scripts.

[0015] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.

[0016] A computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method described above.

[0017] A computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0018] The embodiments of the present invention have the following advantages: In this embodiment of the invention, a device capability map of smart home devices in the current smart home environment is obtained. The device capability map includes device capability nodes, each corresponding to one or more smart home devices. Based on the user's input scene intent, a target device capability node in the device capability map is determined, and based on the target device capability node, a target smart home device is identified. A scene control script for the target smart home device is generated, and the target smart home device is controlled according to the scene control script. This achieves accurate matching of the target smart home device corresponding to the user's scene intent based on the device capability map, automatically generating scene control scripts and completing intelligent control. This not only simplifies the user's manual configuration process but also avoids operational conflicts between devices by leveraging the device capability map, improving the stability and flexibility of device linkage control, thereby optimizing the user experience. Attached Figure Description

[0019] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a method for controlling smart home devices according to some embodiments of the present invention; Figure 2 This is a flowchart of the steps of a method for controlling smart home devices provided in some embodiments of the present invention; Figure 3 This is a flowchart of the steps of a second method for controlling smart home devices provided in some embodiments of the present invention; Figure 4 This is a structural block diagram of a smart home device control device provided in some embodiments of the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0022] In related technologies, smart home scene mode orchestration schemes rely on manual configuration by users. For example, users add devices one by one through an app, setting trigger conditions and actions, a process that is complex and inefficient. Secondly, users often lack understanding of the combined capabilities of smart home devices and find it difficult to discover potential scene possibilities. For example, users may not know how to achieve a reading mode through the linkage of dimming lights, air conditioners, and environmental sensors, or they may not be able to predict conflicts between device actions (such as simultaneously turning on heating and cooling devices).

[0023] The limitations of the relevant technologies can be mainly reflected in the following aspects: 1. The user operation threshold is high. Users need to actively input device parameters and logical relationships, and there is a lack of automated discovery and orchestration capabilities.

[0024] 2. Poor scenario adaptability; the solution relies on a pre-defined rule base and cannot dynamically adapt to the diverse needs of user intent.

[0025] 3. Lack of conflict detection: Resource competition or logical contradictions between device actions (such as conflicts between temperature control devices) are not effectively identified, leading to scenario execution failure or energy waste.

[0026] 4. Knowledge is fragmented, and equipment capability information is scattered across different manufacturers' protocols, lacking a unified semantic expression and correlation analysis.

[0027] Based on this, such as Figure 1 This invention proposes to introduce a visual-language model based on a multimodal Transformer architecture to automate the parsing and structured modeling of device manuals. This transforms unstructured textual and graphical information into a JSON-LD format device capability graph, reducing the configuration complexity of device access. Simultaneously, through a semantic-protocol mapping rule base and a deep learning model, abstract device capabilities (such as wind speed settings) are accurately mapped to specific control primitives, resolving cross-protocol device control compatibility issues. Combined with scene intent parsing, dynamic conflict detection, and multi-objective optimization algorithms, the system can automatically generate highly adaptable and efficient scene control scripts and support dynamic knowledge graph iteration after device manual updates. This achieves full-process automation from device access to scene orchestration, significantly improving the intelligence level, scalability, and user operation efficiency of smart home systems.

[0028] The following combination Figure 2 The present invention will be further described as follows: Reference Figure 2 The diagram illustrates a flowchart of steps for controlling a smart home device according to some embodiments of the present invention, which is applied to a smart home system.

[0029] As examples, a smart home system can consist of multiple smart home devices, including but not limited to smart lights, smart air conditioners, smart curtains, and smart sensors, which can communicate via network connections.

[0030] The smart home system may also include a home host, which has the ability to process, store, and communicate data, and can interact with other smart home devices.

[0031] In some examples, the home host can be a standalone hardware device or an integrated home host (such as one integrated into a smart control screen).

[0032] Specifically, it may include the following steps: Step 201: Obtain the device capability map of smart home devices in the current smart home environment; wherein, the device capability map includes device capability nodes, and each device capability node corresponds to one or more smart home devices.

[0033] In some examples, the device manuals of smart home devices in the current smart home environment can be obtained, and a device capability map can be constructed based on the key information in the device manuals.

[0034] As examples, a device capability graph may include multiple nodes created based on device type, multiple nodes created based on device capability attributes, instance nodes for each smart home device, and multiple interaction edges describing the interaction relationships between device instance nodes, as well as multiple association edges between device instance nodes, device capability nodes, and device type nodes.

[0035] For example, the device type nodes include three types of nodes: air conditioner, smart lighting, and environmental sensor; the instance nodes are air conditioner A and air conditioner B, smart lighting Q and smart lighting E. In the device capability graph, the instance nodes can be assigned to the corresponding type nodes through inclusion relationship edges, that is, the instance nodes of air conditioner A and B are associated with the air conditioner node, and the instance nodes of smart lighting Q and E are associated with the smart lighting node.

[0036] For example, the device capability nodes include four types of nodes: temperature adjustment, brightness adjustment, color temperature adjustment, and fan speed setting; the instance nodes are still air conditioner A, air conditioner B, fan C, smart light fixture Q, and smart light fixture E. Among them, air conditioners A and B have temperature adjustment and fan speed settings, fan C has fan speed settings, and smart lights Q and E have brightness adjustment and color temperature adjustment capabilities. In the device capability graph, instance nodes can be associated with corresponding capability nodes through relational edges. That is, air conditioners A and B are associated with the temperature adjustment and fan speed settings nodes, fan C is associated with the fan speed settings node, and smart lights Q and E are associated with the brightness adjustment and color temperature adjustment nodes, respectively.

[0037] In practical applications, interaction relationships refer to the logical connections between devices that can be linked (such as the linkage between dimming lights and environmental sensors), which are used for automatic scene discovery.

[0038] For example, the lighting fixture's capability attribute is brightness adjustment; the sensor's capability attribute is ambient light detection; the interaction relationship is that the lighting fixture's brightness adjustment can be triggered when the sensor detects that the ambient light is below a threshold; the interaction relationship between device instance nodes can be mined using graph traversal algorithms (such as breadth-first search) to generate corresponding interaction relationship edges.

[0039] As examples, the relationship refers to the logical connection (such as possession relationship, inclusion relationship) between the device instance node, the device capability node, and the device type node, which is used to describe the capabilities that the device has and the type to which it belongs.

[0040] For example, a smart lighting fixture Q is an instance node that is associated with two device capability nodes, namely brightness adjustment and color temperature adjustment, through a relational edge, and is also associated with a device type node, namely smart lighting fixture, through an inclusion relational edge.

[0041] In some embodiments of the present invention, before obtaining the device capability map of smart home devices in the current smart home environment, the method further includes: obtaining the device manual of the smart home device and extracting key information from the device manual; and constructing the device capability map of the smart home device based on the key information.

[0042] In practical applications, users can upload the instruction manuals of smart home devices to the home host for parsing, or the home host can retrieve the instruction manuals from the network based on the device model of the paired smart home devices and parse them to extract key information from the instruction manuals.

[0043] As examples, a device instruction manual refers to a written or illustrated document (such as a printed manual, scanned copy, or electronic PDF) that explains how to use a smart home device. Key information may include device type (such as an air conditioner), function description (such as "fan speed adjustment"), control parameters (such as "temperature range 16℃-30℃"), and communication protocols (such as Matter (Internet of Things protocol) and Zigbee (Zigbee protocol)).

[0044] In some examples, home consoles can use multimodal Transformer models (such as LayoutLM (LayoutLanguage Model)) to perform OCR (Optical Character Recognition) recognition and semantic parsing on the text and graphics content of the device manual, extract key information such as device type, function description, control parameters and communication protocols, and construct a device capability map in JSON-LD format.

[0045] Taking a smart air conditioner manual as an example, key information may include: device type (smart air conditioner); function description (supports automatic fan speed adjustment to achieve comfortable temperature control); control parameters (fan speed levels: low (1-3), medium (4-6), high (7-9)); temperature adjustment range (16℃~30℃); communication protocol (such as Matter).

[0046] In practical applications, key information can be automatically extracted from keywords or semantic analysis results in the equipment manual.

[0047] After extracting key information, we can determine the device type, device capability attributes, and interaction relationships between smart home devices, and construct a device capability map of smart home devices.

[0048] Among them, the equipment capability attributes can be determined by extracting the core capabilities (such as wind speed regulation and temperature control) from the equipment manual through semantic parsing, and then modeling the relationship through graph neural network (GNN).

[0049] For example, the original text of the device manual states: "The smart bulb supports multiple brightness levels to match different scene requirements." The parsed result is: "Function attribute = [Brightness adjustment, Scene matching]". "Brightness adjustment" can be used as a device capability node of the smart home device and associated with a node whose device type is "lighting fixture" through an edge.

[0050] In some embodiments of the present invention, after constructing a device capability map of smart home devices based on the key information, the method further includes: establishing a mapping relationship between device capability nodes in the device capability map and control primitives in the smart home devices.

[0051] As examples, control primitives refer to the smallest instruction units used in smart home device control to describe the operation or state of a device (such as SetAttribute / GetAttribute in the Matter protocol and read / write instructions in the Zigbee protocol), which specify the object to be operated based on parameter identifiers defined by the protocol (such as Cluster ID, Attribute ID, register address).

[0052] For example, the "fan speed adjustment" capability of a smart air conditioner can be mapped to the Fan Control Cluster in the Matter protocol, where low fan speed corresponds to the "Low" attribute value in the Cluster, medium speed corresponds to "Medium", and high speed corresponds to "High".

[0053] In some examples, semantic parsing and visualization of each device capability node in the device capability graph can be performed through a semantic-protocol mapping rule base or deep learning modules (such as BERT, Bidirectional Encoder Representations from Transformers, a bidirectional encoder representation based on Transformers), generating control primitives that smart home devices can recognize and execute, and thus establishing a mapping relationship between device capability nodes and control primitives.

[0054] For example, the device capability node "wind speed level" can be mapped to Attribute0x0000 of Matter Cluster 0x0102, and the mapping strategy can be dynamically adjusted in combination with data from the device manufacturer.

[0055] Here, Cluster 0x0102 refers to the Fan Control Cluster defined in the Matter protocol; Attribute 0x0000 refers to the Fan Speed ​​Attribute defined in the cluster; the fan speed level (such as "high") is mapped to a specific value in the Matter protocol through Attribute 0x0000 (such as 7 corresponding to high).

[0056] For example, user intent: Turn on the air conditioner at high fan speed; System mapping: Device capability node semantic "wind speed" → semantic-protocol rule base → Matter Cluster0x0102.Attribute 0x0000=7; Generate control commands: Matter command, SetAttribute(FanSpeed=7); where SetAttribute is the control primitive and FanSpeed=7 is the specific parameter; Device response: The air conditioner sets the fan speed to high (level 7).

[0057] In the above embodiments, by establishing a mapping relationship between device capability nodes and control primitives, unified control of cross-protocol devices can be achieved, and compatibility with new devices can be achieved without developing an adaptation layer.

[0058] Step 202: Based on the user's input scenario intent, determine the target device capability node in the device capability map, and based on the target device capability node, determine the target smart home device.

[0059] As examples, scenario intent refers to a specific goal or scenario requirement (such as "reading mode" or "sleep mode") that a user inputs via voice or text through smart home devices. For instance, a user might input, "I need a quiet sleeping environment."

[0060] In practical applications, it can parse scene intent and identify the associated target device capability nodes (such as brightness adjustment of lamps and temperature control of air conditioners) and target smart home devices (such as smart lamps and air conditioners).

[0061] In some embodiments of the present invention, determining a target device capability node in the device capability map based on a user-inputted scenario intent includes: determining keywords from the user-inputted scenario intent; determining the semantic similarity between the keywords and device capability nodes in the device capability map; and determining the target device capability node in the device capability map based on the semantic similarity.

[0062] In practical applications, keywords representing scene intent (such as quiet and sleep) can be extracted and mapped to device capability nodes in the device capability graph (such as noise reduction mode and ambient silence).

[0063] In some examples, semantic similarity can be determined by calculating the cosine similarity between the word vectors of keywords and the semantics of device capability nodes.

[0064] For example, the similarity between the keyword "quiet" and device capability nodes such as "noise reduction mode" and "ambient quiet" in the device capability map is calculated, and the node with the highest similarity is selected as the target device capability node.

[0065] As examples, a home host can use a pre-trained NLP (Natural Language Processing) model (such as BERT) to perform semantic parsing on the user's input contextual intent (such as "I need a quiet sleeping environment"), extract core keywords (such as "quiet" and "sleep"), and then match the corresponding device capability nodes in the device capability knowledge graph based on semantic similarity (such as "quiet" matching the "ambient mute" node, representing the mute function of the environmental sensor; "sleep" matching the "noise cancellation mode" node, representing the noise cancellation function of the smart headphones) as the target device capability nodes.

[0066] In the example above, when a user inputs a scenario intent, the system can automatically map the keywords to the corresponding device capability nodes in the device capability graph, triggering a graph traversal algorithm to mine related smart home devices (such as environmental sensors and smart headphones) to generate a scenario script for "turning on the environmental sensor's silent mode + starting the smart headphones' noise reduction," thus achieving a seamless conversion from natural language to device linkage.

[0067] In some embodiments of the present invention, determining a target smart home device based on the target device capability node includes: determining a candidate smart home device list based on the target device capability node; obtaining user historical behavior data, and determining the target smart home device from the candidate smart home device list based on the user historical behavior data.

[0068] After identifying the target device capability node, one or more smart home devices associated with the target device capability node can be identified to build a candidate smart home device list.

[0069] For example, a graph traversal algorithm (such as breadth-first search) can be used to mine and match smart home devices (such as "smart curtains", "noise-canceling headphones", "environmental sensors") that are associated with the target device's capability nodes, generating a preliminary list of candidate smart home devices.

[0070] As some examples, user history behavior data can include user's past operation records of smart home devices, device usage frequency, device operation preference settings, user operation behavior in the APP, and other behavioral data to reflect user usage habits and demand preferences.

[0071] After creating a list of candidate smart home devices, historical user behavior data can be obtained, and target smart home devices can be identified from the list based on this data. The target smart home device can be one or more smart home devices.

[0072] For example, the optimal combination of user historical behavior data (such as "energy saving priority" or "comfort priority") can be selected from the list of candidate smart home devices as the target smart home devices.

[0073] In some embodiments of the present invention, determining a target smart home device from the candidate smart home device list based on the user's historical behavior data includes: determining the weights of candidate smart home devices in the candidate smart home device list based on the user's historical behavior data, and dynamically adjusting the weights based on real-time environmental data; determining the matching degree of the candidate smart home devices or combinations of candidate smart home devices based on the dynamically adjusted weights, and determining the target smart home device based on the matching degree.

[0074] As examples, the weights for candidate smart home devices can be based on energy-saving attributes. For instance, if historical user behavior data shows that energy saving is a priority in most scenarios, then a higher value can be assigned to the energy-saving attribute.

[0075] After obtaining users' historical behavior data, energy-saving attribute weights can be assigned to candidate smart home devices in the candidate smart home device list based on the user's historical behavior data, and the energy-saving attribute weights can be dynamically adjusted in combination with real-time environmental data.

[0076] In some examples, real-time environmental data may include environmental parameters such as the current time (e.g., night or day), indoor and outdoor temperatures, and light intensity.

[0077] For example, at night when the indoor temperature is low, the energy-saving attribute weight of air conditioning equipment with temperature regulation function can be appropriately reduced to maintain a comfortable indoor temperature, so as to ensure that energy saving is taken into account while meeting the user's comfort needs.

[0078] As examples, a weighted scoring model can be applied to filter candidate smart home devices from multiple dimensions. Based on users' historical behavior data (such as the device operation mode that users have previously set to "energy saving priority"), the energy-saving attribute weights of smart home devices can be assigned (e.g., the energy consumption coefficient of smart curtains +0.3). The energy-saving attribute weights can be dynamically adjusted by combining real-time environmental data (e.g., whether it is nighttime and indoor noise levels) to calculate the comprehensive matching degree between combinations of smart home devices. Then, the top 3 optimal combinations (e.g., "curtains + sensor + air conditioner") can be sorted according to the comprehensive matching degree and output, and a visual solution can be generated for users to confirm.

[0079] Step 203: Generate a scene control script for the target smart home device, and control the target smart home device according to the scene control script.

[0080] As examples, scene control scripts may include device control instructions, and scene control scripts are structured program files that can be parsed and executed by a home host.

[0081] After identifying the target smart home device, a scene control scheme can be generated based on the scene intent. The scene control scheme can then be converted into device control commands using a protocol conversion middleware. Finally, a scene control script for the target smart home device can be generated using a script synthesizer to control the target smart home device.

[0082] In some examples, after controlling a target smart home device, when a user manually adjusts scene parameters (such as light brightness in "reading mode"), the system captures and records this user behavior in real time as a basis for optimization. By analyzing and adjusting the correlation with the scene, the semantic weight of the "brightness adjustment" device capability node in the device capability graph is dynamically increased, strengthening its correlation strength in the "reading mode" subgraph. Subsequently, the optimized parameters are encrypted and synchronously uploaded to the cloud through a federated learning framework, and the device capability graph is iteratively updated in combination with global user behavior data. This makes subsequent scene orchestration more accurately reflect user habits, achieving a closed-loop optimization from single adjustment to continuous evolution of the knowledge base.

[0083] For example, when a user manually increases the light brightness from the default 30% to 60% in a "reading mode" scenario, the system immediately captures this behavior data, analyzes the correlation between this operation and "reading mode" through a federated learning framework, and dynamically increases the semantic weight of the "brightness adjustment" node in the device capability graph (for example, from the original weight 0.5 to 0.75), so that when the system generates a "reading mode" scenario in the future, it will prioritize setting the default light brightness to 60% instead of 30%.

[0084] In some embodiments of the present invention, generating a scene control script for the target smart home device includes: generating a target scene control scheme based on the scene intent; wherein the target scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; generating device control instructions based on the scene triggering conditions and action sequence in the target scene control scheme, and generating a scene control script based on the device control instructions.

[0085] In some examples, scene triggering conditions can be generated based on the user's scene intent and the user's historical behavior data.

[0086] For example, users can input scene intents via voice or text (such as "I need a quiet sleeping environment"), obtain users' historical behavior data (such as smart lights being off at 22:00), and generate scene trigger conditions based on users' scene intents and historical behavior data (such as mapping the "sleep" scene intent to the scene trigger condition "detecting that the user enters the bedroom and turns on after 22:00").

[0087] In some examples, the action sequence of a target smart home device refers to a set of sequential operation instructions that the target smart home device needs to execute after the scene triggering conditions are met (such as closing the curtains → muting the environmental sensor).

[0088] As examples, a protocol conversion middleware can be used to combine a "semantic-protocol mapping rule base or deep learning module (such as BERT)" with the "mapping relationship between device capability nodes and control primitives" to convert the generated target scene control scheme into device control instructions that are adapted to different communication protocols. Based on these device control instructions, scene control scripts that can be directly issued and executed can be further generated.

[0089] For example, scene intents (such as "sleep") can be transformed into specific trigger condition logic expressions (such as "time ≥ 22:00 and user detected entering bedroom"). Then, based on the identified target smart home devices, each action to be performed (such as "turn off smart lights" or "lower air conditioner fan speed") is converted into device control instructions for that brand and model of device through a protocol conversion middleware. Finally, a script synthesizer can be used to assemble and encapsulate these elements according to a template format of "when [trigger condition] is met, execute the action sequence [instruction 1, instruction 2...]" to generate a scene control script, which is then sent to the home host for execution.

[0090] As examples, during the semantic-protocol mapping stage, the system can automatically construct a protocol weight matrix based on protocol identifiers in the device manual and users' historical protocol preferences. When device control commands need to be generated, the system calculates the matching degree for each protocol (Matter, Zigbee, Wi-Fi, etc.), with the following priority rules: if the device manufacturer explicitly supports Matter and the user has not specified other preferences, the Matter protocol weight is set to the highest (e.g., 1.0), and the weights of other protocols are dynamically reduced according to manufacturer compatibility; if the device only supports Zigbee, the Zigbee protocol weight is automatically increased and the protocol conversion middleware is triggered. This real-time calculation ensures that the optimal protocol path is always prioritized for command generation, avoiding cross-protocol conflicts, while also allowing users to manually switch protocol preferences in the app to override the default policy.

[0091] When a user does not specify a protocol preference, the system will prioritize the protocol that the user has selected most frequently in the past (e.g., if user A is used to using the Matter protocol to control smart home devices, then Matter will be enabled by default). Users can also manually switch through the App (e.g., user A clicks "Switch to Zigbee" on the "Device Protocol Settings" page in the App) to increase the protocol conversion weight of the Zigbee protocol to 1.0 and override the default policy. All subsequent device control commands will be executed through the Zigbee protocol first, achieving personalized protocol adaptation.

[0092] As examples, users can also modify scene triggering conditions at any time through the App (such as changing the default "after 22:00" to "the user's mobile phone GPS location was detected in the bedroom"). The system can regenerate and optimize the scene control script according to the new scene triggering conditions, achieving a seamless transformation from natural language intent to dynamic triggering logic, ensuring that the scene triggering conditions are both in line with user habits and fit the actual usage scenario.

[0093] In some embodiments of the present invention, generating a target scene control scheme based on the scene intent includes: generating multiple candidate scene control schemes based on the scene intent; wherein each candidate scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; generating an evaluation result of the multiple candidate scene control schemes based on the scene triggering conditions and action sequence in each candidate scene control scheme; and determining the target scene control scheme from the multiple candidate scene control schemes based on the evaluation result.

[0094] As examples, multiple candidate scenario control schemes can be generated by invoking multi-objective optimization algorithms (such as NSGA-II, Non-dominated Sorting Genetic Algorithm II) based on scenario intent.

[0095] In some examples, after determining the candidate scenario control schemes, an evaluation model can be invoked to comprehensively evaluate each candidate scenario control scheme based on dimensions such as energy consumption, execution success rate, and user experience, and obtain the evaluation results.

[0096] For example, when a user inputs the scenario intent "I need an energy-efficient sleep environment," the multi-objective optimization algorithm (NSGA-II) is invoked to automatically generate three candidate scenario solutions: Option 1 (close curtains + raise air conditioner temperature by 1℃ + silence ambient sensor) consumes 25W, has a 98% success rate, and provides a user experience score of 8.5 / 10.

[0097] Option 2 (close the curtains + lower the air conditioner temperature by 1℃ + start the noise-canceling headphones) consumes 35W, has a 92% success rate, and provides a user experience score of 7.0 / 10.

[0098] Option 3 (close curtains + air conditioner maintains original temperature + ambient sensor keeps quiet) consumes 20W, has a 95% success rate, and a user experience score of 9.0 / 10.

[0099] Then, the evaluation results (i.e., the comprehensive score) are calculated according to the preset evaluation weights in the evaluation model (such as energy consumption 30%, execution success rate 40%, user experience 30%). For example, Option 1 scores 0.85, Option 2 scores 0.82, and Option 3 scores 0.94.

[0100] Based on the evaluation results, the third option with the highest comprehensive score was selected as the optimal solution from multiple candidate scene control schemes. This option was then used as the target scene control scheme. A scene control script containing the trigger condition ("After detecting that the user has entered the bedroom at 22:00") and the action sequence ("Close the curtains → Mute the environmental sensor") was generated and executed to ensure that the scene is both energy-efficient and comfortable.

[0101] In some embodiments of the present invention, before generating the evaluation results of the plurality of candidate scene control schemes based on the scene triggering conditions and action sequences in each candidate scene control scheme, the method further includes: performing conflict detection on the action sequences in each candidate scene control scheme, and adjusting the action sequences in the candidate scene control schemes based on the conflict detection results.

[0102] In practical applications, conflict detection refers to the identification and judgment of possible operational command conflicts in the action sequence of candidate scenario control schemes.

[0103] In some examples, the rule engine can be invoked to verify whether there are action conflicts in the action sequence (such as "turn on noise-canceling headphones" and "turn off environmental sensors" may affect sleep monitoring), identify potential conflict points, and if they exist, the action sequence can be adjusted, such as changing "turn off environmental sensors" to "adjust environmental sensors to low sensitivity mode", to ensure that basic monitoring of the sleep environment can still be maintained while noise-canceling headphones are turned on, and to avoid missing important environmental information due to completely turning off environmental sensors.

[0104] In this embodiment of the invention, a device capability map of smart home devices in the current smart home environment is obtained. The device capability map includes device capability nodes, each corresponding to one or more smart home devices. Based on the user's input scene intent, a target device capability node in the device capability map is determined, and based on the target device capability node, a target smart home device is identified. A scene control script for the target smart home device is generated, and the target smart home device is controlled according to the scene control script. This achieves accurate matching of the target smart home device corresponding to the user's scene intent based on the device capability map, automatically generating scene control scripts and completing intelligent control. This not only simplifies the user's manual configuration process but also avoids operational conflicts between devices by leveraging the device capability map, improving the stability and flexibility of device linkage control, thereby optimizing the user experience.

[0105] Reference Figure 3 This document illustrates a flowchart of another method for controlling smart home devices according to some embodiments of the present invention, which is applied to a smart home system.

[0106] Specifically, it may include the following steps: Step 301: Obtain the device manual for the smart home device and extract key information from the device manual.

[0107] Step 302: Based on the key information, construct a device capability map of smart home devices.

[0108] Step 303: Establish the mapping relationship between the device capability nodes in the device capability graph and the control primitives in the smart home device.

[0109] Step 304: Obtain the device capability map of smart home devices in the current smart home environment; wherein, the device capability map includes device capability nodes, and each device capability node corresponds to one or more smart home devices; Step 305: Based on the user's input scenario intent, determine the target device capability node in the device capability map, and based on the target device capability node, determine the target smart home device.

[0110] Step 306: Generate a scene control script for the target smart home device, and control the target smart home device according to the scene control script.

[0111] In this embodiment of the invention, a device capability map of smart home devices in the current smart home environment is obtained. The device capability map includes device capability nodes, each corresponding to one or more smart home devices. Based on the user's input scene intent, a target device capability node in the device capability map is determined, and based on the target device capability node, a target smart home device is identified. A scene control script for the target smart home device is generated, and the target smart home device is controlled according to the scene control script. This achieves accurate matching of the target smart home device corresponding to the user's scene intent based on the device capability map, automatically generating scene control scripts and completing intelligent control. This not only simplifies the user's manual configuration process but also avoids operational conflicts between devices by leveraging the device capability map, improving the stability and flexibility of device linkage control, thereby optimizing the user experience.

[0112] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0113] Reference Figure 4The diagram shows a structural schematic of a smart home device control apparatus provided in some embodiments of the present invention, which is applied to a smart home system.

[0114] Specifically, it can include the following modules: The device capability map acquisition module 401 is used to acquire the device capability map of smart home devices in the current smart home environment; wherein, the device capability map includes device capability nodes, and each device capability node corresponds to one or more smart home devices; The target smart home device determination module 402 is used to determine the target device capability node in the device capability map according to the scene intent input by the user, and determine the target smart home device based on the target device capability node; The scene control script generation and control module 403 is used to generate a scene control script for the target smart home device, and control the target smart home device according to the scene control script.

[0115] In some embodiments of the present invention, the target smart home device determination module 402 includes: The keyword determination submodule is used to determine keywords from the user's input scenario intent; The semantic similarity determination submodule is used to determine the semantic similarity between the keyword and the device capability nodes in the device capability map; The target device capability node determination submodule is used to determine the target device capability node in the device capability map based on the semantic similarity.

[0116] In some embodiments of the present invention, the target smart home device determination module 402 includes: The candidate smart home device list determination submodule is used to determine the candidate smart home device list based on the target device capability nodes; The smart home device identification submodule is used to acquire user historical behavior data and, based on the user historical behavior data, determine the target smart home device from the candidate smart home device list.

[0117] In some embodiments of the present invention, the smart home device determination submodule includes: The weight adjustment unit is used to determine the weight of the candidate smart home devices in the candidate smart home device list based on the user's historical behavior data, and to dynamically adjust the weight based on real-time environmental data. The matching degree determination unit is used to determine the matching degree of candidate smart home devices or combinations of candidate smart home devices based on dynamically adjusted weights, and to determine the target smart home device based on the matching degree.

[0118] In some embodiments of the present invention, the scene control script generation and control module 403 includes: The target scene control scheme determination submodule is used to generate a target scene control scheme based on the scene intent; wherein, the target scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; The scene control script generation submodule is used to generate device control instructions based on the scene triggering conditions and action sequences in the target scene control scheme, and to generate scene control scripts based on the device control instructions.

[0119] In some embodiments of the present invention, the target scene control scheme determination submodule includes: The candidate scene control scheme determination unit is used to generate multiple candidate scene control schemes based on the scene intent; wherein each candidate scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; The evaluation result determination unit generates evaluation results for the multiple candidate scene control schemes based on the scene triggering conditions and action sequences in each candidate scene control scheme, and determines the target scene control scheme from the multiple candidate scene control schemes based on the evaluation results.

[0120] In some embodiments of the present invention, the apparatus further includes: The action sequence adjustment module is used to perform conflict detection on the action sequences in each candidate scene control scheme, and adjust the action sequences in the candidate scene control scheme according to the conflict detection results.

[0121] In some embodiments of the present invention, the apparatus further includes: The key information extraction module is used to obtain the device manual of the smart home device and extract key information from the device manual. The device capability map construction module is used to construct a device capability map of smart home devices based on the key information.

[0122] In some embodiments of the present invention, the apparatus further includes: The mapping relationship determination module is used to establish the mapping relationship between the device capability nodes in the device capability map and the control primitives in the smart home device.

[0123] In this embodiment of the invention, a device capability map of smart home devices in the current smart home environment is obtained. The device capability map includes device capability nodes, each corresponding to one or more smart home devices. Based on the user's input scene intent, a target device capability node in the device capability map is determined, and based on the target device capability node, a target smart home device is identified. A scene control script for the target smart home device is generated, and the target smart home device is controlled according to the scene control script. This achieves accurate matching of the target smart home device corresponding to the user's scene intent based on the device capability map, automatically generating scene control scripts and completing intelligent control. This not only simplifies the user's manual configuration process but also avoids operational conflicts between devices by leveraging the device capability map, improving the stability and flexibility of device linkage control, thereby optimizing the user experience.

[0124] Some embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.

[0125] Some embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, and which, when executed by a processor, implements the method described above.

[0126] Some embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0127] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0129] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0130] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0135] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.

[0136] The above provides a detailed description of the method, apparatus, device, medium, and product for controlling smart home devices. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for controlling smart home devices, characterized in that, The method includes: Obtain a device capability map of smart home devices in the current smart home environment; wherein, the device capability map includes device capability nodes, and each device capability node corresponds to one or more smart home devices; Based on the user's input scenario intent, the target device capability node in the device capability map is determined, and based on the target device capability node, the target smart home device is determined; Generate a scene control script for the target smart home device, and control the target smart home device according to the scene control script.

2. The method according to claim 1, characterized in that, Based on the user's input scenario intent, the target device capability node in the device capability map is determined, including: Determine keywords from the user's input contextual intent; Determine the semantic similarity between the keywords and the device capability nodes in the device capability map; Based on the semantic similarity, the target device capability node in the device capability map is determined.

3. The method according to claim 1, characterized in that, Based on the target device capability nodes, the target smart home devices are determined, including: Based on the target device capability nodes, a list of candidate smart home devices is determined; Obtain user historical behavior data, and determine the target smart home device from the candidate smart home device list based on the user historical behavior data.

4. The method according to claim 3, characterized in that, Based on the user's historical behavior data, target smart home devices are determined from the list of candidate smart home devices, including: Based on the user's historical behavior data, the weights of the candidate smart home devices in the candidate smart home device list are determined, and the weights are dynamically adjusted based on real-time environmental data. Based on the dynamically adjusted weights, the matching degree of candidate smart home devices or combinations of candidate smart home devices is determined, and the target smart home device is determined based on the matching degree.

5. The method according to any one of claims 1-4, characterized in that, Generate scene control scripts for the target smart home device, including: Based on the scene intent, a target scene control scheme is generated; wherein, the target scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; Based on the scene triggering conditions and action sequence in the target scene control scheme, generate device control instructions, and based on the device control instructions, generate scene control scripts.

6. The method according to claim 5, characterized in that, Based on the stated scene intent, a target scene control scheme is generated, including: Based on the scene intent, multiple candidate scene control schemes are generated; wherein, each candidate scene control scheme includes scene triggering conditions and an action sequence for the target smart home device; Based on the scene triggering conditions and action sequences in each candidate scene control scheme, an evaluation result of the multiple candidate scene control schemes is generated, and based on the evaluation result, a target scene control scheme is determined from the multiple candidate scene control schemes.

7. The method according to claim 6, characterized in that, Before generating the evaluation results of the multiple candidate scene control schemes based on the scene triggering conditions and action sequences in each candidate scene control scheme, the method further includes: performing conflict detection on the action sequences in each candidate scene control scheme, and adjusting the action sequences in the candidate scene control schemes based on the conflict detection results.

8. The method according to any one of claims 1-4, characterized in that, Before obtaining the device capability map of smart home devices in the current smart home environment, the following steps are also included: Obtain the instruction manuals for smart home devices and extract key information from them; Based on the key information, construct a device capability map of smart home devices.

9. The method according to claim 8, characterized in that, After constructing a device capability map of smart home devices based on the key information, the method further includes: establishing a mapping relationship between device capability nodes in the device capability map and control primitives in the smart home devices.

10. A device for controlling smart home devices, characterized in that, The device includes: The device capability map acquisition module is used to acquire the device capability map of smart home devices in the current smart home environment; wherein, the device capability map includes device capability nodes, and each device capability node corresponds to one or more smart home devices; The target smart home device determination module is used to determine the target device capability node in the device capability map based on the user's input scene intent, and to determine the target smart home device based on the target device capability node; The scene control script generation and control module is used to generate scene control scripts for the target smart home device and control the target smart home device according to the scene control scripts.

11. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 9.

Images