A method and system for automatically constructing a home environment ontology knowledge base
By constructing a home environment ontology knowledge base template and using communication text and image recognition technologies to automatically update semantic information, the problems of unsuitability and low efficiency in constructing a home environment ontology knowledge base are solved, and efficient and standardized knowledge base construction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG UNIV
- Filing Date
- 2023-05-23
- Publication Date
- 2026-05-29
Smart Images
Figure CN116756331B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and knowledge engineering technology, and in particular relates to a method and system for automatically constructing a home environment ontology knowledge base. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] With the continuous advancement of artificial intelligence technology and the increasing demands for intelligent home environments, the demand for intelligent home devices and service robots is rapidly expanding. As a core component of home intelligence, ontology knowledge bases can provide decision-making support for the linkage of home devices and the execution of tasks by service robots, significantly improving the level of home intelligence.
[0004] Currently, the construction of ontology knowledge bases largely relies on information extraction from the internet or manual construction. However, in practical work, due to factors such as the complexity of the home environment, the variety of household items, and the dynamic placement of items, ontology knowledge bases built from online information are not suitable for the home environment. Secondly, manually constructing ontology knowledge bases is time-consuming and inefficient. Furthermore, the structure of ontology knowledge bases constructed by different people will vary, resulting in a lack of standardization and difficulty in reuse. Summary of the Invention
[0005] To overcome the shortcomings of the prior art, the present invention provides an automatic construction method and system for an ontology knowledge base in a home environment, which solves the problems of inapplicability, inefficiency and non-standardization in the construction of ontology knowledge bases in a home environment.
[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:
[0007] The first aspect of this invention provides a method for automatically constructing a home environment ontology knowledge base.
[0008] A method for automatically constructing a knowledge base on a home environment ontology includes the following steps:
[0009] Step S101: Construct a home environment ontology knowledge base template, which includes the ontology's classes, attributes, and instances;
[0010] Step S102: Establish communication with the target item, obtain the communication text of the target item, and extract the semantic information of the target item;
[0011] Step S103: Obtain an image of the target item and extract semantic information of the target item based on the image;
[0012] Step S104: The semantic information of the extracted items is used as an instance of the ontology and compared with the existing classes and attributes in the family environment ontology knowledge base. Based on the similarity comparison results, it is determined whether new classes and attributes need to be created, and the semantic information is updated to improve the family environment ontology knowledge base.
[0013] Preferably, the family environment ontology knowledge base includes four main categories of classes: robots, items, users, and environment. The attributes include object attributes and data attributes. The object attribute connection class is used to describe the association attributes between different classes in terms of composition and dependency. The data attribute connection class and attribute value are used to describe the data values of the class.
[0014] Preferably, after step S101, the method further includes: determining whether communication can be established with the target item; if communication cannot be established, step S102 is skipped and step S103 is executed directly.
[0015] Preferably, the communication text of the target item is transmitted in a triple format, which is either <class, object attribute, class> or <class, data attribute, attribute value>. After receiving the communication text transmitted by the target item, the service robot or control center extracts the semantic information stored in the text according to a predetermined format. The extracted semantic information is also in triple format.
[0016] Preferably, after acquiring the image of the target item, the service robot or control center uses YOLOv5 to detect the category and color semantic information of the target item, and uses Neural Motifs to detect the positional relationship information of the target item with other items. The category, color semantic information and positional relationship information of the target item are saved in triplet format.
[0017] Preferably, the extracted semantic information of the items is used as an instance of the ontology, and its similarity is compared with the classes and attributes already existing in the family environment ontology knowledge base. Specifically, this includes:
[0018] Preferably, the feature vectors of the newly extracted semantic information are compared with the information feature vectors in the existing family environment ontology knowledge base using cosine similarity. Cosine similarity measures the difference between the two vectors by the cosine value of the angle between them. The smaller the angle between the two vectors, the closer the cosine value is to 1, indicating that the two vectors are more similar. Conversely, the closer the angle between the two vectors is to 90°, the closer the cosine value is to 0, indicating that the two vectors are less similar.
[0019] Preferably, if the semantic similarity of the items is greater than a preset threshold, the match is successful, and the semantic information of the items is automatically added to the home environment ontology knowledge base; if the semantic similarity of the items is less than the preset threshold, the match fails, and a new class and attribute corresponding to the items are automatically created in the home environment ontology knowledge base, and the semantic information is automatically added therein.
[0020] A second aspect of the present invention provides an automatic construction system for a home environment ontology knowledge base.
[0021] An automated system for constructing a home environment ontology knowledge base includes:
[0022] The knowledge base template building module is configured to: build a home environment ontology knowledge base template, which contains the ontology's classes, properties, and instances;
[0023] The body communication module is configured to: establish communication with the target item, obtain the communication text of the target item, and extract the semantic information of the target item;
[0024] The image recognition module is configured to: acquire an image of the target item and extract semantic information of the target item based on the image;
[0025] The knowledge base update module is configured to: use the extracted semantic information of the items as instances of the ontology and compare their similarity with the existing classes and attributes in the family environment ontology knowledge base; determine whether to create new classes and attributes based on the similarity comparison results; update the semantic information; and improve the family environment ontology knowledge base.
[0026] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the method for automatically constructing a home environment ontology knowledge base as described in the first aspect of the present invention.
[0027] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the method for automatically constructing a home environment ontology knowledge base as described in the first aspect of the present invention.
[0028] The above one or more technical solutions have the following beneficial effects:
[0029] This invention extracts semantic information of a target item from its communication text by communicating with the target item; it also extracts semantic information from its image by acquiring the target item. Based on the extracted semantic information, it automatically adds it to an existing home environment ontology knowledge base. Compared to extracting home environment semantic information from online information, the method disclosed in this invention avoids the inapplicability of the ontology knowledge base due to changes in the home environment. Compared to manually constructing a home environment ontology knowledge base, the method disclosed in this invention effectively reduces manual workload, improves construction efficiency, and reduces the non-standardization inherent in manually constructed ontology knowledge bases.
[0030] After obtaining the semantic information of items, this invention uses a similarity comparison method to reduce the repeated construction of ontology knowledge base classes and attributes, thereby improving the standardization of the ontology knowledge base.
[0031] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0032] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0033] Figure 1 This is a flowchart of the method in the first embodiment.
[0034] Figure 2 This is a system structure diagram of the second embodiment. Detailed Implementation
[0035] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0036] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.
[0037] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.
[0038] Overall concept of the invention:
[0039] This invention discloses an automatic construction method and system for a home environment ontology knowledge base, specifically including: constructing a home environment ontology knowledge base template; extracting semantic information of target items based on communication text with target items; extracting semantic information of target items based on images; and improving the home environment ontology knowledge base based on the extracted semantic information. This invention simplifies the construction process of the home environment ontology knowledge base, improves construction efficiency, reduces labor costs, and reduces the inconsistencies inherent in manual construction, thereby improving the quality of the constructed ontology knowledge base.
[0040] Example 1
[0041] This embodiment discloses a method for automatically constructing a home environment ontology knowledge base.
[0042] like Figure 1 As shown, a method for automatically constructing a home environment ontology knowledge base includes the following steps:
[0043] Step S101: Construct a home environment ontology knowledge base template, which includes the ontology's classes, attributes, and instances;
[0044] Step S102: Establish communication with the target item, obtain the communication text of the target item, and extract the semantic information of the target item;
[0045] Step S103: Obtain an image of the target item and extract semantic information of the target item based on the image;
[0046] Step S104: The semantic information of the extracted items is used as an instance of the ontology and compared with the existing classes and attributes in the family environment ontology knowledge base. Based on the similarity comparison results, it is determined whether new classes and attributes need to be created, and the semantic information is updated to improve the family environment ontology knowledge base.
[0047] Specifically:
[0048] Step S101: Construct a home environment ontology knowledge base template.
[0049] In this embodiment of the invention, before acquiring the semantic information of the target item, a home environment ontology knowledge base template needs to be constructed. This template includes the ontology's classes, attributes, and instances, where attributes include object attributes and data attributes, etc. This template can provide a reference for subsequently adding semantic information to the home environment ontology knowledge base.
[0050] In one implementation, the classes in the home environment ontology knowledge base are divided into four main categories: "Robots," "Items," "Users," and "Environment."
[0051] The "robot" category is divided into two types: "service robots" and "sweeping robots".
[0052] The category of "goods" is further subdivided into ten categories: "furniture", "home appliances", "tableware", "food", "clothing", "medicine", "books", "cosmetics", "toys" and "decorative items".
[0053] "Environment" refers to a room in a home environment, used to indicate the location of a robot, object, or user.
[0054] In one implementation, attributes in the home environment ontology knowledge base are divided into object attributes and data attributes. Object attributes connect classes, used to describe the relationships between different classes in terms of composition, dependencies, etc. Specifically, object attributes include "classification relationships," "operational relationships," and "location relationships." Data attributes connect classes and attribute values, used to describe the data values of the classes. Specifically, data attributes include "physical attributes" and "functional attributes."
[0055] In this embodiment of the invention, as a supplement, after step S101, it is necessary to further determine whether the target item has a communication function. If the target item has a communication function, proceed to step S102. If the target item does not have a communication function, skip step S102 and proceed to step S103.
[0056] Step S102: Extract semantic information of the target item based on the communication text with the target item.
[0057] In this embodiment of the invention, the service robot or control center in a home environment first establishes communication with the target item. The service robot or control center and the target item use a TCP or UDP connection. After receiving a message sent by the target item, the service robot or control center analyzes the header. If the header meets the requirements, the service robot or control center and the target item begin information transmission.
[0058] Furthermore, the target item transmits structured text that stores its own semantic information. The text is transmitted in a triple format, either <class, object attribute, class> or <class, data attribute, attribute value>. The former represents the association attributes between the target item and other classes, while the latter represents the attribute value of the target item.
[0059] Furthermore, the service robot or control center extracts corresponding semantic information from the text transmitted by the target item. After receiving the text transmitted by the target item, the service robot or control center can extract the semantic information stored in the text according to a predetermined format. The extracted semantic information is also in triplet format.
[0060] Step S103: Extract semantic information of the target item based on the image.
[0061] In an example of the present invention, the service robot or control center in a home environment first acquires an image of the target item through a camera.
[0062] Furthermore, semantic information is extracted from the target item photos using deep learning methods. The service robot or control center uses YOLOv5 to detect the category and color semantic information of the target item, and Neural Motifs to detect the positional relationship between the target item and other items. The semantic information of the target item (i.e., the category, color semantic information, and positional relationship information of the target item) is extracted from the image and saved in triplet format.
[0063] YOLOv5 (You Only Look Once Version 5) is an object detection algorithm that employs a single-stage detection method. It can directly extract the location and category information of objects from an image without first extracting candidate regions. It boasts high detection accuracy and real-time performance, and is widely used in computer vision for tasks such as object detection, instance segmentation, and keypoint detection.
[0064] Neural Motifs are computer vision models and methods for image understanding and scene parsing. Proposed by a research team at Stanford University, it is a graph neural network-based model designed to extract semantic relationships and scene structure from images.
[0065] Step S104: Improve the family environment ontology knowledge base based on the extracted semantic information.
[0066] In this invention, the newly extracted semantic information is used as an instance of the ontology and its similarity is compared with the classes and attributes in the constructed family environment ontology knowledge base. The feature vectors of the newly extracted semantic information are then compared with the feature vectors of the existing family environment ontology knowledge base using cosine similarity. Cosine similarity measures the difference between two vectors by the cosine of the angle between them. The smaller the angle between the two vectors, the closer the cosine value is to 1, indicating that the two vectors are more similar. Conversely, the closer the angle between the two vectors is to 90°, the closer the cosine value is to 0, indicating that the two vectors are less similar.
[0067] Using similarity comparison can reduce the redundant construction of classes and attributes in the ontology knowledge base, thus improving the standardization of the ontology knowledge base. For example, in an existing home environment ontology knowledge base, there is a class "refrigerator," while the newly received text is "electric refrigerator." Through similarity comparison, we can know that these two classes are the same item, and we can directly add object attributes and data attributes to the "refrigerator" class without having to rebuild the "electric refrigerator" class in the knowledge base.
[0068] Furthermore, if the semantic similarity of the item is greater than a preset threshold, the match is successful, and the semantic information is automatically added to the constructed family environment ontology knowledge base; if the semantic similarity of the item is less than the preset threshold, the match fails, and the corresponding class and attributes for the item are automatically created in the constructed family environment ontology knowledge base, and the semantic information is automatically added to it.
[0069] Example 2
[0070] This embodiment discloses an automatic construction system for a home environment ontology knowledge base.
[0071] like Figure 2 As shown, an automatic construction system for a home environment ontology knowledge base includes:
[0072] The knowledge base template building module is configured to: build a home environment ontology knowledge base template, which contains the ontology's classes, properties, and instances;
[0073] The body communication module is configured to: establish communication with the target item, obtain the communication text of the target item, and extract the semantic information of the target item;
[0074] The image recognition module is configured to: acquire an image of the target item and extract semantic information of the target item based on the image;
[0075] The knowledge base update module is configured to: use the extracted semantic information of the items as instances of the ontology and compare their similarity with the existing classes and attributes in the family environment ontology knowledge base; determine whether to create new classes and attributes based on the similarity comparison results; update the semantic information; and improve the family environment ontology knowledge base.
[0076] Example 3
[0077] The purpose of this embodiment is to provide a computer-readable storage medium.
[0078] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for automatically constructing a home environment ontology knowledge base as described in Embodiment 1 of this disclosure.
[0079] Example 4
[0080] The purpose of this embodiment is to provide an electronic device.
[0081] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the automatic construction method of a home environment ontology knowledge base as described in Embodiment 1 of this disclosure.
[0082] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.
[0083] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.
[0084] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. A method for automatically constructing a knowledge base on a family environment ontology, characterized in that, Includes the following steps: Step S101: Construct a home environment ontology knowledge base template, which includes the ontology's classes, attributes, and instances; Step S102: Establish communication with the target item, obtain the communication text of the target item, and extract the semantic information of the target item; the communication text of the target item is transmitted in triple format, the triple format is <class, object attribute, class> or <class, data attribute, attribute value>. After the service robot or control center receives the communication text transmitted by the target item, it extracts the semantic information stored in the text according to the predetermined format. The extracted semantic information is also in triple format. Step S103: Obtain an image of the target item and extract semantic information of the target item based on the image; After acquiring the image of the target item, the service robot or control center uses YOLOv5 to detect the category and color semantic information of the target item, and uses Neural Motifs to detect the positional relationship information of the target item with other items. The category, color semantic information and positional relationship information of the target item are saved in triplet format. Step S104: The extracted semantic information of the item is used as an instance of the ontology and its similarity is compared with the existing classes and attributes in the home environment ontology knowledge base. Based on the similarity comparison result, it is determined whether to create new classes and attributes and update the semantic information to improve the home environment ontology knowledge base. Specifically, this includes: comparing the feature vector of the newly extracted semantic information with the information feature vector in the existing home environment ontology knowledge base using cosine similarity. If the semantic information similarity of the item is greater than a preset threshold, the match is successful, and the semantic information of the item is automatically added to the home environment ontology knowledge base; if the semantic information similarity of the item is less than the preset threshold, the match fails, and the class and attribute corresponding to the item are automatically created in the home environment ontology knowledge base, and the semantic information is automatically added to them.
2. The method for automatically constructing a family environment ontology knowledge base as described in claim 1, characterized in that, The home environment ontology knowledge base includes four main categories of classes: robots, items, users, and environment. Attributes include object attributes and data attributes. Object attribute connection classes are used to describe the association attributes between different classes in terms of composition and dependency. Data attribute connection classes and attribute values are used to describe the data values of the class.
3. The method for automatically constructing a family environment ontology knowledge base as described in claim 1, characterized in that, After step S101, the method further includes: determining whether communication can be established with the target item; if communication cannot be established, step S102 is skipped and step S103 is executed directly.
4. The method for automatically constructing a family environment ontology knowledge base as described in claim 1, characterized in that, The cosine similarity measures the difference between two vectors by the cosine value of the angle between them. The smaller the angle between the two vectors, the closer the cosine value is to 1, indicating that the two vectors are more similar. Conversely, the closer the angle between the two vectors is to 90°, the closer the cosine value is to 0, indicating that the two vectors are less similar.
5. An automatic construction system for a home environment ontology knowledge base, characterized in that: include: The knowledge base template building module is configured to: build a home environment ontology knowledge base template, which contains the ontology's classes, properties, and instances; The body communication module is configured to: establish communication with the target item, obtain the communication text of the target item, and extract the semantic information of the target item; The communication text of the target item is transmitted in triple format, which is either <class, object attribute, class> or <class, data attribute, attribute value>. After the service robot or control center receives the communication text transmitted by the target item, it extracts the semantic information stored in the text according to a predetermined format. The extracted semantic information is also in triple format. The image recognition module is configured to: acquire an image of the target item and extract semantic information of the target item based on the image; After acquiring the image of the target item, the service robot or control center uses YOLOv5 to detect the category and color semantic information of the target item, and uses Neural Motifs to detect the positional relationship information of the target item with other items. The category, color semantic information and positional relationship information of the target item are saved in triplet format. The knowledge base update module is configured to: use the extracted semantic information of items as instances of the ontology and compare their similarity with existing classes and attributes in the family environment ontology knowledge base; based on the similarity comparison results, determine whether to create new classes and attributes and update the semantic information to improve the family environment ontology knowledge base; specifically, it includes: comparing the feature vector of the newly extracted semantic information with the information feature vector of the existing family environment ontology knowledge base using cosine similarity; if the semantic information similarity of the item is greater than a preset threshold, the match is successful, and the semantic information of the item is automatically added to the family environment ontology knowledge base; if the semantic information similarity of the item is less than the preset threshold, the match fails, and the corresponding class and attribute are automatically created in the family environment ontology knowledge base, and the semantic information is automatically added to it.
6. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the automatic construction method of the home environment ontology knowledge base as described in any one of claims 1-4.
7. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the automatic construction method of the home environment ontology knowledge base as described in any one of claims 1-4.