A home environment device linkage method and system

Abstract

The application provides a home environment device linkage method and system, and relates to the field of Internet of Things.The method comprises the following steps: constructing a home environment ontology knowledge base and a rule knowledge base; obtaining object information in the home environment, transmitting the object information to the home environment ontology knowledge base for storage and reasoning to obtain real-time states of the objects; inquiring whether a solution corresponding to the real-time states of the objects is previously set in the rule knowledge base; if no solution is previously set, calling a task planner to plan a task and generate a service task corresponding to the real-time states of the objects; and delivering the generated service task to a home control center or a service robot.The application uses the task planner to avoid task planning failure caused by rule omission, improves the stability and intelligence of device linkage, and increases the expandability of the system by adding the ontology knowledge of a new device to the home environment ontology knowledge base to realize linkage control of the device.

Description

Technical Field

[0001] This invention belongs to the field of Internet of Things (IoT) technology, and in particular relates to a method and system for linking devices in a home environment. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With increasing demands for intelligent home environments and the development of smart homes, more and more devices are being connected to home networks, generating ever-growing amounts of data. How to effectively manage, analyze, and utilize this data, and improve the interoperability of devices within the home environment, is a crucial issue facing the smart home field. An ontology knowledge base can provide a unified semantic description framework, enabling data from different devices to be described and represented in a consistent format, thereby achieving better data interaction and sharing. The system can utilize the ontology knowledge base to provide decision-making support for smart home device linkage and service robot task execution, thereby improving user experience and convenience.

[0004] Currently, manufacturers mostly use pre-set rules for device linkage in the home environment. For example, when a temperature sensor detects that the temperature is too high, the air conditioner is turned on; when it reaches 6 PM, the curtains are closed. However, manually writing rules requires considering many scenarios and conditions, and a significant investment of time and effort in rule design and implementation. Omissions or oversights may occur during the rule writing process, leading to incomplete or incorrect rules, thus affecting the reliability and stability of device linkage. Rule writing typically requires considering numerous scenarios and conditions; if new devices or scenarios are added, new rules must be written, significantly reducing the system's scalability.

[0005] Therefore, the current usage rules have some shortcomings in the linkage of devices in the home environment. For complex and diverse smart home scenarios, they cannot well meet people's requirements for more intelligent linkage of devices in the home environment. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention provides a method and system for linking devices in a home environment, which solves the problems of instability and low scalability of using rules to link devices in a home environment.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] The first aspect of this invention provides a method for linking home environment devices.

[0009] A method for linking home environment devices includes the following steps:

[0010] Step S101: Construct a home environment ontology knowledge base, describe and represent data between different devices in a consistent format, and at the same time construct a rule knowledge base to store user-preset rules, wherein the user-preset rules are the solutions corresponding to the corresponding states of objects;

[0011] Step S102: Acquire object information in the home environment in real time, transmit the object information to the home environment ontology knowledge base for storage and reasoning, and obtain the real-time state of the objects;

[0012] Step S103: Check if a solution corresponding to the real-time state of the object is pre-set in the rule knowledge base;

[0013] Step S104: If there is no pre-set solution in the ontology knowledge base, the task planner is invoked to perform task planning and generate service tasks corresponding to the real-time state of the object.

[0014] Step S105: The generated service task is sent to the home control center or service robot, and the home control center or service robot controls the execution terminal to execute the service task.

[0015] Preferably, a home environment ontology knowledge base is constructed, which includes user knowledge, robot knowledge, environmental knowledge, and rule knowledge, wherein:

[0016] User knowledge includes the user's physical state and personal information; robot knowledge includes the parameters and functions of the home environment service robot; environmental knowledge includes information about items and rooms in the environment; and rule knowledge is used for service reasoning, automatically inferring the required service task when the rules are met.

[0017] Preferably, the home environment ontology knowledge base is constructed using a web ontology language, and uses classes, attributes, and instances to describe and represent data between different devices in a consistent format.

[0018] Preferably, real-time acquisition of object information in the home environment includes:

[0019] The sensor system in the home environment is used to acquire the values ​​of physical quantity sensors in real time and transmit them to the home environment ontology knowledge base for storage and reasoning.

[0020] For user or object states that physical quantity sensors cannot obtain, cameras are used for real-time detection, semantic information is extracted from the images, and transmitted to the home environment ontology knowledge base for storage and reasoning.

[0021] Preferably, user-defined rules are written using the SWRL language. SWRL consists of premises and results, and the corresponding result can only be obtained when the conditions are met.

[0022] Preferably, a planning domain definition language is used for task planning. The task planning problem is defined as a binary tuple including a domain description and a problem description under a specific domain. The domain description includes three parts: a set of types, a set of predicates, and a set of actions, specifying the action set, action preconditions, and action result states of the atomic task. The problem description under a specific domain consists of the domain, object, initial state, and target state. The goal of task planning is to solve for an action sequence that realizes the transformation from the initial state to the target state.

[0023] Preferably, after the task is completed, the user or item status in the home environment ontology knowledge base is updated based on the generated service task.

[0024] A second aspect of the present invention provides a home environment equipment linkage system.

[0025] A home environment device linkage system, comprising:

[0026] The ontology knowledge base construction module is configured to: construct a home environment ontology knowledge base, describe and represent data between different devices in a consistent format, and construct a rule knowledge base that stores user-preset rules, wherein the user-preset rules are solutions corresponding to the corresponding states of objects;

[0027] The sensor detection module is configured to: acquire object information in the home environment in real time, transmit the object information to the home environment ontology knowledge base for storage and reasoning, and obtain the real-time state of the objects;

[0028] The rule knowledge base query module is configured to: query whether a solution corresponding to the real-time state of an object is pre-set in the rule knowledge base;

[0029] The task planning module is configured to: if there is no pre-set solution in the ontology knowledge base, call the task planner to perform task planning and generate service tasks corresponding to the real-time state of the object.

[0030] The task distribution module is configured to distribute the generated service tasks to the home control center or service robot, and then the home control center or service robot controls the execution terminal to execute the service tasks.

[0031] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the home environment device linkage method as described in the first aspect of the present invention.

[0032] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the home environment device linkage method as described in the first aspect of the present invention.

[0033] The above one or more technical solutions have the following beneficial effects:

[0034] This invention uses a home environment ontology knowledge base and introduces a task planner to automatically plan service tasks while using rules to link devices.

[0035] 1) It reduced the number of manually written rules and improved efficiency;

[0036] 2) Using a task planner can avoid task planning failures due to rule omissions, thus improving the stability and intelligence of device linkage;

[0037] 3) When new devices are added to the scene, simply add the device's ontology knowledge to the home environment ontology knowledge base to achieve device linkage control, increasing the system's scalability.

[0038] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0040] Figure 1 This is a flowchart of the method in the first embodiment.

[0041] Figure 2 This is a system structure diagram of the second embodiment. Detailed Implementation

[0042] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0043] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0044] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0045] Example 1

[0046] This embodiment discloses a method for linking home environment devices.

[0047] like Figure 1 As shown, a method for linking home environment devices includes the following steps:

[0048] Step S101: Construct a home environment ontology knowledge base, describe and represent data between different devices in a consistent format, and at the same time construct a rule knowledge base to store user-preset rules, wherein the user-preset rules are the solutions corresponding to the corresponding states of objects;

[0049] Step S102: Acquire object information in the home environment in real time, transmit the object information to the home environment ontology knowledge base for storage and reasoning, and obtain the real-time state of the objects;

[0050] Step S103: Check if a solution corresponding to the real-time state of the object is pre-set in the rule knowledge base;

[0051] Step S104: If there is no pre-set solution in the ontology knowledge base, the task planner is invoked to perform task planning and generate service tasks corresponding to the real-time state of the object.

[0052] Step S105: The generated service task is sent to the home control center or service robot, and the home control center or service robot controls the execution terminal to execute the service task.

[0053] Specifically, it includes:

[0054] Step S101: Construct a home environment ontology knowledge base

[0055] An ontology knowledge base provides a unified semantic description framework, enabling data from different devices to be described and represented in a consistent format, thereby achieving better data interaction and sharing. The home environment ontology knowledge base stores various knowledge within the home environment, which can help smart devices in the home environment perform task planning and decision-making, improve device interoperability, and enhance the level of intelligence in the home environment.

[0056] The ontology knowledge base is constructed using the Web Ontology Language (OWL) recommended by the W3C organization. Resources in OWL include three types: classes, properties, and instances.

[0057] In the ontology domain, related concepts are represented by classes, and the hierarchical relationships between different concepts are described using the "class-subclass" format. For example, the home environment ontology knowledge base includes user, robot, and environment classes. The environment class can be further divided into item and room classes, and the relationship between the environment class and the item class is described using the "class-subclass" format.

[0058] Attributes include object attributes and data attributes. Relationships between different classes or instances are described through object attributes; for example, in the statement "an apple is on a table," the relationship between the apple and the table is described as "on...". Relationships between instances and data types are described through data attributes; for example, the information "the apple's color is red" can be represented by defining an instance of the Apple class, apple1, and the data attribute of instance apple1 is red.

[0059] An instance refers to an object within a concept, that is, a specific member of a class. In OWL ontology, a class can have multiple instances simultaneously. For example, the Home Appliances class can have instances of Refrigerator, Washing Machine, etc. An instance can also be a member of multiple classes; for example, a refrigerator of brand A is not only an instance of the Home Appliances class but also an instance of the Product class of brand A.

[0060] In this solution, the home environment ontology knowledge base includes user knowledge, robot knowledge, environmental knowledge, and rule knowledge. User knowledge includes the user's physical state and personal information; robot knowledge includes the parameters and functions of the home environment service robot; environmental knowledge includes information about items and rooms in the environment; and rule knowledge is used for service reasoning. When the rules are met in the early stages, the required service task is automatically inferred, and the robot or home control center is invoked to coordinate device actions to complete the task.

[0061] Step S102: The sensor system detects the status of the user and the items.

[0062] In this solution, the sensor system in the home environment acquires the values ​​of physical quantity sensors in real time and transmits them to the home environment ontology knowledge base for storage and inference. For the status of users or objects that cannot be obtained by the physical quantity sensors, the camera performs real-time detection, extracts semantic information from the images, and transmits it to the home environment ontology knowledge base for storage and inference.

[0063] The real-time status of a user or object is acquired through sensors or cameras and saved into a knowledge base. Then, task reasoning is performed based on the status of the object in step 3 or step 4, and the reasoning yields the task.

[0064] Step S103: Check if the rule knowledge base has a pre-set solution.

[0065] In this solution, user-defined rules are written using the Semantic Web Rule Language (SWRL) within the home environment ontology knowledge base. SWRL is a rule description language proposed by the W3C that presents rules semantically. SWRL consists of antecedents and consequents; a result is obtained only when the conditions are met.

[0066] For example, a user can pre-set a rule: if a book falls to the floor, the service robot will pick it up and put it back on the shelf. This can be represented using SWRL as: `on(book, floor)->pick_up(robot, book)^put(book, bookshelf)`. The camera detects a book falling to the floor and transmits its status to the ontology knowledge base. The ontology knowledge base queries the rule knowledge base and finds a pre-set rule for handling this situation. It then sends the pre-set rule to the service robot, which executes the corresponding task.

[0067] Step S104: Use the task planner to plan tasks.

[0068] In this approach, if there are no pre-set rules in the ontology knowledge base to handle exceptional situations, the task planner is invoked to plan tasks.

[0069] Task planning is performed using the Planning Domain Definition Language (PDDL). PDDL defines the task planning problem as a tuple (Domain, Problem). The Domain represents the domain description, mainly consisting of three parts: types, predicates, and actions. It specifies the action set, preconditions, and result states of the atomic task. The Problem is the problem description within a specific domain, composed of the domain, objects, initial state, and goal state. The goal of task planning is to find a sequence of actions that transforms the initial state into the goal state.

[0070] For example, if a book falls to the floor and the user hasn't pre-defined rules for handling it, the task planner will be invoked for task planning. First, sensors detect the book's state on the floor, while the ontology knowledge base stores the book's state on the bookshelf. Different book states trigger the task planner to generate service tasks. The initial conditions and execution results for each action of the service robot are predefined in the Domain file. For example, the initial condition for the "pick up" action is that the item is not in the robot's hand, and the execution result is that the item is in the robot's hand. The initial condition for the "move" action is that the robot is at point A, and the execution result is that the robot is at point B. The initial condition for the "place" action is that the item is in the robot's hand, and the execution result is that the item is neither in the robot's hand nor in a specific location. In the Problem file, the initial state is that the book is on the floor, and the target state is that the book is on the bookshelf. Invoking the task planner generates a series of service tasks (robot picks up the book, robot moves to the bookshelf, robot places the book on the bookshelf) for the service robot to execute.

[0071] Step S105: Issue tasks according to the task sequence

[0072] The home control center or service robot generates a task sequence based on step S103 or S104 and issues tasks accordingly. Upon completion of the task, the user or item status in the home environment ontology knowledge base is updated. The user or item status is updated based on the generated service tasks.

[0073] The rule knowledge base (step S103) or task planner (step S104) mentioned above will generate a series of service tasks. The completion of each service task will change the state of the user or item. Therefore, the state of the user or item in the home environment ontology knowledge base should be updated in a timely manner after the service task is completed.

[0074] For example, if the robot's initial state is in the living room, and the task assigned is for the robot to move to the bedroom, then after the task is completed, the robot's location in the ontology knowledge base needs to be updated to the bedroom.

[0075] The state of a user or item corresponds to the value of an object attribute or data attribute in the home environment ontology knowledge base (step S101). Therefore, updating the state of a user or item is done by modifying the corresponding object attribute value or data attribute value.

[0076] Example 2

[0077] This embodiment discloses a home environment equipment linkage system.

[0078] like Figure 2 As shown, a home environment device linkage system includes:

[0079] The ontology knowledge base construction module is configured to: construct a home environment ontology knowledge base, describe and represent data between different devices in a consistent format, and construct a rule knowledge base that stores user-preset rules, wherein the user-preset rules are solutions corresponding to the corresponding states of objects;

[0080] The sensor detection module is configured to: acquire object information in the home environment in real time, transmit the object information to the home environment ontology knowledge base for storage and reasoning, and obtain the real-time state of the objects;

[0081] The rule knowledge base query module is configured to: query whether a solution corresponding to the real-time state of an object is pre-set in the rule knowledge base;

[0082] The task planning module is configured to: if there is no pre-set solution in the ontology knowledge base, call the task planner to perform task planning and generate service tasks corresponding to the real-time state of the object.

[0083] The task distribution module is configured to distribute the generated service tasks to the home control center or service robot, and then the home control center or service robot controls the execution terminal to execute the service tasks.

[0084] Example 3

[0085] The purpose of this embodiment is to provide a computer-readable storage medium.

[0086] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the home environment device linkage method as described in Embodiment 1 of this disclosure.

[0087] Example 4

[0088] The purpose of this embodiment is to provide an electronic device.

[0089] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the home environment device linkage method as described in Embodiment 1 of this disclosure.

[0090] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0091] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0092] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for linking home environment devices, characterized in that, Includes the following steps: Step S101: Construct a home environment ontology knowledge base, describe and represent data between different devices in a consistent format, and at the same time construct a rule knowledge base to store user-preset rules, wherein the user-preset rules are the solutions corresponding to the corresponding states of objects; The home environment ontology knowledge base includes user knowledge, robot knowledge, environmental knowledge, and rule knowledge, among which: User knowledge includes the user's physical state and personal information; robot knowledge includes the parameters and functions of the home environment service robot; environmental knowledge includes information about items and rooms in the environment; and rule knowledge is used for service reasoning. When the rules are met, the required service task is automatically inferred. Step S102: Acquire object information in the home environment in real time, transmit the object information to the home environment ontology knowledge base for storage and reasoning, and obtain the real-time state of the objects; Step S103: Check if a solution corresponding to the real-time state of the object is pre-set in the rule knowledge base; Step S104: If there is no pre-set solution in the ontology knowledge base, the task planner is invoked to perform task planning and generate service tasks corresponding to the real-time state of the object. Task planning is performed using the Planning Domain Definition Language (PDDL). PDDL defines the task planning problem as a pair of tuples: Domain and Problem. Domain represents the domain description, mainly consisting of three parts: types, predicates, and actions. It specifies the action set, preconditions, and result states of the atomic task. Problem is the problem description under a specific domain, consisting of the domain, objects, initial state (init), and goal state (goal). The goal of task planning is to find a sequence of actions that transforms the initial state into the goal state. Step S105: The generated service task is sent to the home control center or service robot, and the home control center or service robot controls the execution terminal to execute the service task.

2. The home environment device linkage method as described in claim 1, characterized in that, The home environment ontology knowledge base is constructed using the Web Ontology language, and uses classes, attributes, and instances to describe and represent data across different devices in a consistent format.

3. The home environment device linkage method as described in claim 1, characterized in that, Real-time acquisition of object information in the home environment, specifically including: The sensor system in the home environment is used to acquire the values ​​of physical quantity sensors in real time and transmit them to the home environment ontology knowledge base for storage and reasoning. For user or object states that physical quantity sensors cannot obtain, cameras are used for real-time detection, semantic information is extracted from the images, and transmitted to the home environment ontology knowledge base for storage and reasoning.

4. The home environment device linkage method as described in claim 1, characterized in that, The SWRL language is used to write user-defined rules. SWRL consists of premises and results. The corresponding result can only be obtained when the conditions are met.

5. The home environment device linkage method as described in claim 1, characterized in that, Task planning is performed using a domain definition language. The task planning problem is defined as a tuple consisting of a domain description and a problem description under a specific domain. The domain description includes three parts: a set of types, a set of predicates, and a set of actions, specifying the action set, action preconditions, and action result states of the atomic task. The problem description under a specific domain consists of the domain, the object, the initial state, and the target state. The goal of task planning is to solve for a sequence of actions that transforms the initial state into the target state.

6. The home environment device linkage method as described in claim 1, characterized in that, It also includes updating the user or item status in the home environment ontology knowledge base based on the generated service task after the task is completed.

7. A home environment equipment linkage system, characterized in that, The home environment device linkage method according to any one of claims 1-6 includes: The ontology knowledge base construction module is configured to: construct a home environment ontology knowledge base, describe and represent data between different devices in a consistent format, and construct a rule knowledge base that stores user-preset rules, wherein the user-preset rules are solutions corresponding to the corresponding states of objects; The sensor detection module is configured to: acquire object information in the home environment in real time, transmit the object information to the home environment ontology knowledge base for storage and reasoning, and obtain the real-time state of the objects; The rule knowledge base query module is configured to: query whether a solution corresponding to the real-time state of an object is pre-set in the rule knowledge base; The task planning module is configured to: if there is no pre-set solution in the ontology knowledge base, call the task planner to perform task planning and generate service tasks corresponding to the real-time state of the object. The task distribution module is configured to distribute the generated service tasks to the home control center or service robot, and then the home control center or service robot controls the execution terminal to execute the service tasks.

8. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the home environment device linkage method as described in any one of claims 1-6.

9. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the home environment device linkage method as described in any one of claims 1-6.

Images