Multi-source heterogeneous data fusion and management method for smart home scene

Abstract

The invention provides a multi-source heterogeneous data fusion and management method for a smart home scene, and the method generates a corresponding equipment linkage rule based on fusion feature vector representation data. And based on the equipment linkage rule, sending an action instruction to equipment in the smart home scene, so that the equipment executes a corresponding action. By integrating and analyzing the data from different devices and sensors, the technical problem of data isolation between intelligent devices is solved, and the real-time state and the user demand in the current scene can be known more accurately, so that the corresponding response can be made more accurately and quickly.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a multi-source heterogeneous data fusion and management method for smart home scenarios. Background Art

[0002] Smart home refers to a system that uses technologies such as the Internet of Things, sensors, and artificial intelligence to enable home devices and facilities to operate and interact automatically and intelligently. With the continuous development of intelligent technology, more and more smart devices are being used in people's lives, forming a smart home network covering multiple scenarios such as homes, vehicles, and offices.

[0003] In the prior art, in the application scenario of smart home, there is a technical problem of data isolation between smart devices. Summary of the invention

[0004] The purpose of the present invention is to overcome the above technical deficiencies and provide a multi-source heterogeneous data fusion and management method for smart home scenarios to solve the technical problem of data isolation between smart devices in related technologies.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a multi-source heterogeneous data fusion and management method for a smart home scenario, the method comprising: Acquire multi-source heterogeneous data in the smart home scenario; wherein the multi-source heterogeneous data is acquired from at least two data sources in the smart home scenario; Based on the multi-source heterogeneous data, corresponding graph data is constructed; wherein the graph data includes a plurality of nodes, and the nodes at least represent the data sources; For the graph data, extracting feature vector representation data of the node in the graph data; wherein the feature vector representation data is used to represent semantic information and association relationship information of the corresponding node in the graph data; Performing fusion processing on the feature vector representation data to obtain fused feature vector representation data; Based on the fused feature vector representation data, generating corresponding device linkage rules; In response to the received real-time data, based on the device linkage rules, an action instruction is issued to the device in the smart home scene so that the device performs a corresponding action.

[0006] In a second aspect, the present invention proposes a multi-source heterogeneous data fusion and management device for smart home scenarios, the device comprising: An acquisition module, which is used to acquire multi-source heterogeneous data in the smart home scene; wherein the multi-source heterogeneous data is acquired from at least two data sources in the smart home scene; A construction module, which is used to construct corresponding graph data based on the multi-source heterogeneous data; wherein the graph data includes a plurality of nodes, and the nodes at least represent the data sources; An extraction module, which is used to extract the feature vector representation data of the node in the graph data for the graph data; wherein the feature vector representation data is used to represent the semantic information and association relationship information of the corresponding node in the graph data; A fusion processing module, which is used to perform fusion processing on the feature vector representation data to obtain fused feature vector representation data; A generation module, which is used to generate corresponding device linkage rules based on the fused feature vector representation data; A response module is used to respond to the received real-time data and issue action instructions to the devices in the smart home scene based on the device linkage rules, so that the devices perform corresponding actions.

[0007] In a third aspect, the present invention proposes an electronic device comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors so that the one or more processors implement the above-mentioned method.

[0008] In a fourth aspect, the present invention proposes a computer-readable storage medium, wherein the computer program is stored in the computer-readable storage medium, and the computer program implements the above method when executed by a processor.

[0009] Beneficial effects: The present invention first obtains multi-source heterogeneous data in the smart home scenario, and then constructs corresponding graph data based on the multi-source heterogeneous data. Then, for the graph data, the feature vector representation data of the node in the graph data is extracted. The feature vector representation data is then fused to obtain fused feature vector representation data. Then, based on the fused feature vector representation data, corresponding device linkage rules are generated. Finally, based on the device linkage rules, action instructions are issued to the devices in the smart home scenario to enable the devices to perform corresponding actions. The present invention solves the technical problem of data isolation between smart devices by integrating and analyzing data from different devices and sensors, and can also more accurately understand the real-time status and user needs in the current scenario, so as to make corresponding responses more accurately and quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1It is a flowchart of a multi-source heterogeneous data fusion and management method for a smart home scenario provided by an embodiment of the present invention; Figure 2 It is a flowchart of a multi-source heterogeneous data fusion and management method for a smart home scenario provided by an embodiment of the present invention; Figure 3 It is a flowchart of a multi-source heterogeneous data fusion and management method for a smart home scenario provided by an embodiment of the present invention; Figure 4 It is a block diagram of a multi-source heterogeneous data fusion and management device for a smart home scenario provided by an embodiment of the present invention; Figure 5 It is a block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0012] Overview: Among the related technologies, multi-source heterogeneous data management driven by smart home scenarios is a research field that has emerged in recent years. With the continuous development of intelligent technology, more and more smart devices are being used in people's lives, forming a smart home network covering multiple scenarios such as homes, vehicles, and offices.

[0013] Multi-source heterogeneous data refers to data from different data sources with different structures and semantics.

[0014] In the smart home scenario, the sources of data include but are not limited to embedded sensors, video surveillance, voice recognition and other technical means. These data involve a wide range of content, covering family members' behavior habits, daily consumption, health status and other aspects. In order to efficiently manage these multi-source heterogeneous data, researchers have begun to explore various related technologies. Among them, data fusion technology is an important part of smart home data management. By fusing data from different sources, we can fully grasp the smart home scenario and provide users with smarter services. In addition, data storage and processing technology is also one of the key research directions. How to efficiently store and process a large amount of heterogeneous data has become one of the challenges of smart home data management. On the other hand, the management of multi-source heterogeneous data driven by smart home scenarios faces privacy and security issues. Since the data involves sensitive information such as users' personal privacy and family safety, how to ensure user privacy and data security during data management is also an urgent problem to be solved.

[0015] Among the related technologies, some solutions have clarified the content and classification of multi-source heterogeneous data in the manufacturing production process; explained the data processing methods and technologies used in data collection, data integration and data analysis in the processing of multi-source heterogeneous data, and analyzed the advantages and disadvantages of various methods and technologies as well as their applications. Some solutions have proposed multi-source heterogeneous big data fusion based on "physics-matter-humanity", analyzed the characteristics of multi-source heterogeneous data fusion at three different fusion levels: data layer, information layer and decision layer, and discussed the challenges that data fusion may face in terms of storage, use, analysis technology, data management and value determination. Some solutions have also studied a new multi-source heterogeneous information fusion framework based on deep learning for milling quality prediction, targeting the obstacles brought by the structural heterogeneity of various sensor data to information fusion and model construction. The preprocessing module extracts multi-domain features from structured time series data, and assigns modules based on convolutional neural networks to extract information from unstructured data. Then, multi-layer perceptron technology is used to achieve feature enhancement and cross-domain feature fusion.

[0016] Although the above schemes have proposed a variety of data fusion and management methods to address the structural heterogeneity and context information inconsistency problems faced in multi-source heterogeneous data fusion and management, in the smart home application scenario, the new characteristics of multi-source heterogeneous big data fusion have led to the expansion and extension of its connotation, and also brought new challenges and difficulties to the storage, integration, analysis, and management of multi-source heterogeneous big data fusion. For multi-source heterogeneous big data, how to clearly store data of different forms has become an inevitable problem. A large amount of data is generated every moment in the smart home scenario, but it is still difficult to truly use this data effectively. Most of the data exists in the form of "data islands" and is not connected to each other. There are obvious barriers to the use of data. Data fusion not only needs to realize the transformation and unified integrated analysis of multi-source heterogeneous data, but also needs to pay attention to the implicit knowledge behind the data, strengthen the understanding of the meaning of the data, and organically combine the consensus knowledge with the analysis of numbers. In the fusion model and method, solving the fusion of cross-domain, cross-media, cross-language, and multi-disciplinary is still the frontier and difficulty of research.

[0017] In summary, in the prior art, in the application scenarios of smart homes, there is a technical problem of data isolation between smart devices.

[0018] In response to this technical problem, the present invention first obtains multi-source heterogeneous data in the smart home scenario, and then constructs corresponding graph data based on the multi-source heterogeneous data. Then, for the graph data, extract the feature vector representation data of the node in the graph data. Then, the feature vector representation data is fused to obtain fused feature vector representation data. Then, based on the fused feature vector representation data, generate corresponding device linkage rules. Finally, based on the device linkage rules, issue action instructions to the devices in the smart home scenario so that the devices perform corresponding actions. The present invention solves the technical problem of data isolation between smart devices by integrating and analyzing data from different devices and sensors, and can also more accurately understand the real-time status and user needs in the current scenario, so as to make corresponding responses more accurately and quickly.

[0019] According to an embodiment of the present invention, a multi-source heterogeneous data fusion and management method for a smart home scenario is provided. The method can be applied to a smart home system. Figures 1 to 3 , which may include: Step S102: Acquire multi-source heterogeneous data in the smart home scenario; wherein the multi-source heterogeneous data is acquired from at least two data sources in the smart home scenario.

[0020] In this embodiment, the acquisition action can be represented as an action of acquiring data from at least two data sources in the smart home scene. Specifically, in the smart home scene, there may be multiple data sources, such as a sensor data source, a device status data source, an external data source, a user behavior data source, and a cloud data source.

[0021] More specifically, the sensor data source can store various parameters in the smart home scenario, and different types of parameters can be obtained by setting different types of sensors. For example, the temperature sensor can be used to monitor the indoor temperature, the humidity sensor can be used to monitor the indoor humidity, the light sensor can be used to monitor the light intensity, and the motion sensor can be used to monitor the user's movement trajectory. The device status data source can store the device status information, such as the device's switch status, working mode, energy consumption information, etc. The external data source can store weather forecasts, traffic information, etc. The user behavior data source can store user behavior data, such as the user's preferences for music, movies, lights, etc., as well as the user's activity trajectory and habits at home, etc. The cloud data source can be some smart devices that upload data to the cloud for storage and management, so these data can also be obtained from the cloud data source.

[0022] It is understandable that data obtained from different data sources can constitute multi-source heterogeneous data. In a specific implementation scheme, the following devices may be involved in the smart home scenario: smart health devices, meters, security devices, home appliance control devices, lighting devices, environmental monitoring devices, kitchen and bathroom equipment, living equipment, and audio-visual systems. For the data source of each device, its data format, interface, and communication protocol can be predetermined. Smart health devices may include smart bracelets, smart weight scales, blood pressure monitors, etc. These devices usually communicate with the smart home system via Bluetooth or Wi-Fi. Its data format is based on a specific health data standard (such as HL7, FHIR) format. Meters include electricity meters, water meters, gas meters, etc., which are used to monitor household energy consumption. These meters can be connected to the smart home system through IoT communication technologies (such as Zigbee, LoRaWAN) or wired interfaces. The data format can be standard meter readings, such as electricity meter readings, water meter readings, etc. Security equipment may include cameras, door and window sensors, smoke detectors, etc. These devices can be connected to the smart home system via Wi-Fi or dedicated security protocols (such as ONVIF). The data format is video stream, image data or event information (such as intrusion detection, smoke alarm). Home appliance control devices can include smart sockets, smart switches, etc., which are used to remotely control home appliances. These devices can interact with the smart home system via Wi-Fi or Bluetooth. The data format includes device status (on / off), power consumption, etc. Lighting devices can be smart bulbs, light strips, dimmers, etc., which are used to control home lighting. These devices can be connected to the smart home system through wireless communication (such as Wi-Fi, Zigbee) or wired interfaces (such as DALI). The data format is light brightness, color, etc. Environmental monitoring devices include temperature sensors, humidity sensors, air quality sensors, etc., which are used to monitor indoor environmental conditions. These devices can be connected to the smart home system through wireless communication (such as Zigbee, Bluetooth) or wired interfaces. The data format is temperature, humidity, air quality index, etc. Kitchen and bathroom equipment can include smart faucets, smart bathroom heaters, etc., which are used to provide convenience and safety. These devices usually interact with the smart home system through Wi-Fi or Bluetooth. The data format includes water flow, temperature, etc. Living equipment includes smart curtains, smart furniture, etc., which are used to enhance the living experience. These devices can be connected to the smart home system through wireless communication (such as Zigbee, Wi-Fi) or wired interface. The data format includes device status, location, etc. Audio and video equipment includes smart speakers, smart TVs, etc., which are used to provide entertainment and multimedia experience. These devices can interact with the smart home system through Wi-Fi or Bluetooth. The data format includes audio, video streams, etc.

[0023] Step S104: construct corresponding graph data based on the multi-source heterogeneous data; wherein the graph data includes a plurality of nodes, and the nodes at least represent the data sources.

[0024] In this embodiment, a graph data may be constructed for structured data, semi-structured data, and unstructured data obtained from different data sources. Specifically, the graph data may be a knowledge graph.

[0025] In this embodiment, the nodes in the graph data can be predefined. The nodes can be nodes representing the data source, nodes representing device names, such as device nodes, nodes representing sensor types, nodes representing user behavior, and nodes representing the environment. They can also be nodes representing time, such as morning, noon, or afternoon, etc.

[0026] In a specific implementation, a knowledge graph can be constructed based on similarity, which can be used to store and manage metadata and related information of different data sources. The knowledge graph can be in the form of a graph database or a graph model, which represents data sources and their connections through entities, attributes, and relationships. When constructing a knowledge graph, entity types, attributes, and relationships, as well as corresponding data models, can be defined.

[0027] First, for each data source, metadata can be extracted using natural language processing and data mining techniques. Metadata is data that describes data and can include information such as the structure, attributes, and meaning of the data. By analyzing the format and structure of the data, key metadata information can be extracted, including data table names, field names, data types, value ranges, etc. Based on metadata extraction, keyword extraction is performed on each data source. Keywords are important words or phrases that describe the content and characteristics of data. By using natural language processing and text mining techniques, keywords in the data, such as entity names, attribute names, phrases, etc., can be extracted. Keyword extraction can help identify and describe the characteristics of the data for subsequent data integration and analysis.

[0028] Then, based on the metadata and keyword extraction, ETL tools can be used for data cleaning and mapping. This includes data format conversion, data missing value processing, data duplicate value processing, etc. At the same time, according to the metadata and keyword information, data from different data sources are mapped and matched to ensure consistency and integration between data. After data sorting and cleaning are completed, data quality assessment is performed to assess the accuracy, completeness, consistency, and reliability of the data.

[0029] Finally, for metadata and keyword information, the distance between nodes can be used to measure the relationship between nodes in different smart home scenarios, and an edge can be generated between nodes that meet the distance threshold, thereby generating the knowledge graph. Specifically, the cosine similarity can be used to calculate the relationship between nodes in different smart home scenarios to obtain the knowledge graph in the smart home scenario. In some embodiments, the nodes in the smart home scenario can be predefined, and the nodes can be nodes representing the data source, or nodes representing device names, such as device nodes, or nodes representing sensor types, user behavior nodes, and environment nodes, and nodes representing time, such as morning, noon, or afternoon, etc.

[0030] Step S106: For the graph data, extract the feature vector representation data of the node in the graph data; wherein the feature vector representation data is used to characterize the semantic information and association relationship information of the corresponding node in the graph data.

[0031] In this embodiment, the extraction action can be represented as a process of generating a feature vector representing data of the node, and the information carried by the node can be converted into the feature vector representing data by a certain method.

[0032] In this embodiment, the feature vector representation data can characterize the semantic information and association relationship information of the node in the graph data, and the association relationship information can further include local topology information and global topology information. Specifically, in a possible scenario example, the node can represent various devices, sensors, user behaviors, etc. By extracting the feature vector representation data of the node, different types of nodes can be unified into a numerical representation, which is convenient for subsequent data fusion and processing. The feature vector representation data can represent information such as the node's attributes, status, historical data trends, and the association relationships between nodes, such as linkage rules between devices, user behavior patterns, and so on.

[0033] In this embodiment, a variety of methods can be used to extract the feature vector representation data of the node in the graph data. Specifically, a graph neural network can be used to learn and represent the graph data to extract the feature vector representation data of the node. For example, a GCN network (Graph Convolutional Network) or a GAT network (Graph Attention Network) can be used. Deep learning models, such as convolutional neural networks (CNN), autoencoders, etc., can also be used to learn and represent the attributes and relationships of nodes. Thereby extracting the feature vector representation data of the node.

[0034] Step S108: performing fusion processing on the feature vector representation data to obtain fused feature vector representation data.

[0035] In this embodiment, a plurality of methods may be used to fuse the feature vector representation data to obtain fused feature vector representation data. Specifically, the feature vector representation data of different nodes may be weighted averaged according to certain weights to obtain fused feature vector representation data. A multi-modal feature fusion strategy method may be used to fuse the feature vector representation data to obtain fused feature vector representation data.

[0036] In this embodiment, the fused feature vector representation data has richer information and can better reflect the characteristics and changes of the smart home scene.

[0037] Step S110: Generate corresponding device linkage rules based on the fused feature vector representation data.

[0038] In this embodiment, the generated action can be to automatically generate corresponding device linkage rules based on the fused feature vector representation data. It is understandable that the multi-source heterogeneous data in the smart home scenario can be continuously updated, and correspondingly, the extracted fused feature vector representation data is also continuously updated. It is also understandable that people's behavior habits may change, and the types and quantities of devices and sensors will also change over time. Therefore, the device linkage rules also need to adapt to various changes in the smart home scenario.

[0039] In this embodiment, the corresponding device linkage rules can be automatically generated by a variety of methods, and the fused feature vector representation data can be analyzed and mined to discover the rules and associations therein. Based on the rules and associations obtained by mining, the device linkage rules are generated based on logical rule modeling. The generated device linkage rules can be applied to the smart home system, and the rule engine can be used to implement the execution and control of the rules. The rule engine can automatically trigger the corresponding linkage actions according to the status of the device and the changes in the environment to achieve collaborative work between devices. Alternatively, the device linkage rules can be mined from the fused feature vector representation data using an association rule mining algorithm.

[0040] Step S112: In response to the received real-time data, based on the device linkage rules, an action instruction is issued to the device in the smart home scene so that the device performs a corresponding action.

[0041] In this implementation, the real-time data may be sensor data, user behavior data, external data source data, device status data, etc. acquired in real time.

[0042] In the above implementation, on the one hand, multi-source heterogeneous data in the smart home scene are integrated and managed. By constructing graph data and extracting the feature vector representation data of the node, the information of different data sources can be integrated and unified, avoiding the differences and inconsistencies between data sources. On the one hand, by representing the data through feature vectors, the semantic information and association relationship information of the node in the graph data can be characterized. In this way, the association relationship and rules between devices can be described more comprehensively, and the accuracy and reliability of the device linkage rules can be improved. On the other hand, the method can respond to the received real-time data in real time, and issue action instructions to the devices in the smart home scene according to the device linkage rules. In this way, the collaborative work between devices can be achieved, and the response speed and user experience of the smart home system can be improved.

[0043] In some implementations, the step of acquiring multi-source heterogeneous data in the smart home scenario is as follows: Figure 2 As shown, it may include: Step S1022: acquiring and storing multi-source heterogeneity in at least two data sources in the smart home scenario based on a preset central database; In this implementation, it is considered that multi-source heterogeneous data is processed, where multi-source data refers to data from different data sources, such as sensor data, social media data, and Internet data. Heterogeneous data refers to data with different structures, formats, and semantics. In the smart home scenario, smart home devices and sensors generate a large amount of data through a variety of different data sources, such as temperature, humidity, and home appliance consumption. These data come from a variety of different devices and platforms, and the data formats and structures are also different. Taking the smart temperature control system as an example, the system can automatically adjust the indoor temperature according to the home environment and residents' needs. This involves obtaining data from different data sources, such as reading the indoor temperature from a temperature sensor, and obtaining the outdoor temperature from a weather forecast API. Because these data come from different devices and platforms, and their data formats and structures are also different.

[0044] In order to achieve the fusion and management of multi-source heterogeneous data, this implementation method can use a data integration technology. Data integration refers to the fusion of data from different data sources into a unified data set and providing a unified interface to manage and query these data. In the smart home scenario, a central database can be established to store data from all data sources in a centralized manner, and an appropriate data model can be designed to handle different data formats and structures. In order to achieve real-time and accuracy of data, a data synchronization and update method can be used. By regularly obtaining the latest data from each data source and updating it, the timeliness and accuracy of the data can be guaranteed.

[0045] Step S1024: for the multi-source heterogeneous data stored in the central database, at least identifying abnormal data therein; In this implementation, the collected data often has different formats and structures, so the cleaning step becomes particularly important. Multi-source heterogeneous data is sorted and cleaned to ensure the quality and accuracy of the data. During the data cleaning process, data format conversion, duplicate data deletion, missing data filling, abnormal data identification and repair, etc. can also be performed.

[0046] In this embodiment, in addition to identifying abnormal data, repeated and redundant data can also be identified.

[0047] In this embodiment, the knowledge and experience of domain experts may be used to identify abnormal data, or statistical methods may be used to identify abnormal data.

[0048] Step S1026: Delete the identified abnormal data.

[0049] In some embodiments, in the process of acquiring data, the missing data is recorded at the same time. For abnormal data, the quantiles (Q1, Q3) of the data variable are first calculated, where Q1 represents the first quantile value in the obtained data distribution, which can represent the numerical point of 25% of the data on the left side of the data median; Q3 represents the third quantile value in the data distribution, which can represent the numerical point of 25% of the data on the right side of the data median. The data median can be the numerical point that divides the data into two equal parts.

[0050] Then, based on the two quantiles Q1 and Q3, we get the quartile range IQR=Q3-Q1. The range value IQR measures the degree of dispersion of the middle 50% of the data.

[0051] Then, two thresholds Q1-1.5*IQR and Q3+1.5*IQR are set. When a data value is smaller than Q1-1.5IQR or larger than Q3+1.5IQR, it can be screened out. The screened out value can be identified as an outlier, that is, abnormal data.

[0052] Through the steps of data cleaning and preprocessing, erroneous and redundant data can be removed, and the credibility and validity of the data can be improved.

[0053] In this implementation, on the one hand, by acquiring multi-source heterogeneous data in the smart home scenario based on a preset central database, data from different data sources can be easily acquired and stored. The central database can centrally manage and store data, simplify the process of data acquisition, and improve the accessibility and availability of data. On the one hand, by identifying abnormal data in multi-source heterogeneous data, the quality and accuracy of the data can be improved. Abnormal data may be caused by sensor failure, data collection errors or other reasons. By identifying and deleting abnormal data, the quality and accuracy of the data can be improved, and the influence of abnormal data on subsequent data analysis and rule generation can be avoided.

[0054] In some implementations, the step of acquiring and storing multi-source heterogeneity in at least two data sources in the smart home scenario based on a preset central database includes: Step S10222: Generate a data acquisition request; wherein the data acquisition request is an acquisition request for acquiring incremental data.

[0055] In this implementation, to avoid repeated recording and collection of data, an incremental extraction method is used. The business system records the time when the business occurs and marks the increment. Before each extraction, the largest time recorded in the data storage is first determined, and then all records greater than this time are obtained from the business system based on this time.

[0056] Step S10224: Based on a preset time frequency, the data acquisition request is sent to at least two data sources in the smart home scene.

[0057] In this embodiment, the preset time frequency may be daily or every 12 hours, etc.

[0058] Step S10226: storing the data received from at least two data sources in a preset central database; wherein the data received from at least two data sources constitute the multi-source heterogeneous data.

[0059] In the above implementation, on the one hand, by generating a data acquisition request and sending a request to the data source based on a preset time frequency, data acquisition can be automated. This can reduce the workload of manual intervention and improve the efficiency and accuracy of data acquisition. On the other hand, by using incremental data acquisition requests, only incremental data in the data source can be obtained instead of all data. This can reduce the overhead of data transmission and storage and improve the efficiency and performance of data acquisition.

[0060] In some implementations, the step of constructing corresponding graph data based on the multi-source heterogeneous data is specifically as follows: Figure 3 As shown, it may include: Step S1042: extracting first data and second data from the multi-source heterogeneous data; wherein the first data is data for representing metadata, and the second data is data for representing key information; wherein the key information is at least key information in the data source.

[0061] In this embodiment, the first data may be metadata or other forms of descriptive data, which are data that can describe and explain actual data, such as the properties, location, and status of a device. The second data may be, in addition to keyword information, numerical data, which is relatively more important than text data.

[0062] In this embodiment, the metadata table or system table of the database can be queried by a database query method to obtain metadata information of the data. It can be understood that the database will store metadata of the data, such as table structure, field type, index and other information. Keyword information can be extracted by text mining. For example, natural language processing technology, such as word segmentation, part-of-speech tagging, entity recognition, etc., can be used to extract keywords from text.

[0063] Step S1044: Based on the first data and the second data, determine the distance between preset nodes.

[0064] In this embodiment, the distance between preset nodes can be determined by the following method, specifically including: the relationship between different smart home scene nodes can be calculated using cosine similarity.

[0065] Step S1046: Generate edges between nodes that meet the distance threshold to obtain the graph data.

[0066] It can be understood that different smart home scene nodes can actually be divided into device nodes, sensor nodes, user behavior nodes, environment nodes, etc. The metadata and keyword information extracted are the attribute information of each of the above nodes. For example, the attributes of the device node can include metadata information such as device name, device type, device model, etc., and keyword information can also be used to describe the functional characteristics of the device. For example, the smart thermostat can add keyword information such as "temperature adjustment" and "timing switch". Therefore, the cosine similarity can be used to calculate the relationship between different nodes. A connecting edge can be constructed between nodes with high similarity.

[0067] In the above implementation, heterogeneous data from multiple data sources can be integrated, including metadata and key information. By extracting the first data and the second data, the information in the data source can be more comprehensively described and characterized. By generating edges between nodes that meet the distance threshold, graph data can be constructed. Graph data can more intuitively display the relationship and topological structure between nodes, facilitating subsequent graph analysis and mining.

[0068] In some embodiments, the step of extracting the feature vector representation data of the node in the graph data includes: extracting meta-path information from the graph data based on a preset extraction rule; using the meta-path information to navigate the graph data to obtain a meta-path-based wandering sequence; and inputting the meta-path-based wandering sequence into a preset word vector model for processing to obtain the feature vector representation data of the node in the graph data.

[0069] In the above implementation, the maximum length or minimum length of the meta-path may be set to construct the extraction rule, and the path type of the meta-path may be set to restrict the form of the meta-path to construct the extraction rule.

[0070] In a specific implementation scheme, a meta-path can be defined as a sequence that describes the association relationship between different data sources. A meta-path is a combination of a series of nodes and edges that is used to capture the semantic association between data sources. A meta-path can be defined based on the goals and requirements of the association. For example, a meta-path can represent the association relationship from a smart device to an environmental sensor. After the meta-path is defined, a set of representative meta-paths can be extracted from the graph data through meta-path sampling and generation methods. The meta-path sampling method can be selected based on the topological structure of the graph data and the importance of the nodes to ensure that the sampled meta-paths have high information richness and representativeness. For each meta-path, it is converted into a vector representation using a meta-path representation learning method. The meta-path representation learning method can use a graph neural network or an embedding learning-based method to map the nodes and edges in the meta-path to low-dimensional vectors. These vector representations can capture the semantic associations and information in the meta-path.

[0071] In a specific implementation scheme, the meta-path information in the knowledge graph can be extracted according to the type of home scene, and the and Two hyperparameters are used to control the random walk, and then the walk sequence based on the meta-path is obtained as follows: In the formula, Is a node and nodes The transition probability between is the normalization constant. Indicates Nodes visited in random walks. Indicates Walk to the node Under the conditions of Walk to the node The meaning of the above formula is as follows: if the node and nodes There is an edge between them, that is, , then the probability , otherwise it is equal to 0.

[0072] You can set hyperparameters based on node context information and The size of and control random walk strategy to generate node sequences. The preset word vector model can be a Skip-Gram model (skip word model). Specifically, the Skip-Gram model is a model for generating word vectors. It is a part of the Word2Vec model and aims to generate vector representations of words by learning the contextual relationship between words. The Skip-Gram model can be used to obtain feature vector representation data of the node, and the feature vector representation data reflects the local and global topological information of the node in the graph. In some embodiments, in addition to the Skip-Gram model, the CBOW model (Continuous Bag-of-Words) and the FastText model can also be used. The FastText model is a model for text classification and word vector learning. The core idea of ​​the model is: using the bag of words model to quickly generate feature vectors of the text, and then performing classification learning through a shallow neural network.

[0073] In this implementation, on the one hand, the structure and semantic information of the graph data are comprehensively considered. By extracting meta-path information and meta-path-based walk sequences, the topological structure between nodes and the contextual semantic information of nodes can be considered at the same time, thereby improving the representation ability of feature vectors. On the other hand, multi-source heterogeneous data are converted into feature representations with semantic information. These feature representations can reflect the similarities and differences between data and provide useful information for subsequent data fusion.

[0074] In some embodiments, before the step of issuing an action instruction to at least one device in the smart home scenario in response to the received real-time data based on the fused feature vector representation data and a preset action execution library so that the device performs a corresponding action, the method further includes: The fused feature vector representation data is stored in a graph database; wherein the graph database is configured in the cloud.

[0075] In this embodiment, the fused feature vector representation data can be properly stored and managed for subsequent data analysis and application. Specifically, the graph database can be a graph database based on Neo4j, in which nodes represent entities and edges represent relationships between entities. Neo4j is a graph database, which is a database management system based on graph theory and is specifically used to store and process graph data. The Neo4j database adopts a graph database model, in which data is stored in the form of nodes and relationships, nodes represent entities, and relationships represent connections between entities.

[0076] This database structure can query and analyze complex relationships very efficiently. Using the Cypher query language in the graph database to query and operate on data and create indexes on the properties of nodes and relationships can quickly locate and retrieve nodes and relationships that meet the conditions. Cypher query language is the query language of the Neo4j graph database, which is used to query and operate graph data.

[0077] In terms of management, a data warehouse or data lake is used to establish a unified data repository to facilitate data query, analysis and application development. In terms of storage, cloud storage is selected. At the same time, in order to better manage and utilize these integrated data, standardized data formats, data classification and indexing are also used to improve data readability and query efficiency.

[0078] At least one data desensitization method is used to perform desensitization processing on the graph database.

[0079] In this embodiment, for sensitive data that needs to protect privacy, a generalized data desensitization method can be used to replace sensitive information in the data with a broader category or range to hide specific details. For example, the user's age range is generalized to "adult" or "minor" instead of a specific age number. This method can reduce the risk of sensitive information leakage while maintaining data availability.

[0080] In some embodiments, the device linkage rules include: a rule for linking temperature with air conditioning: if the current indoor temperature is greater than the upper temperature limit set by the user, the air conditioning is turned on; a rule for linking light with lighting: if the current light intensity is less than the lower limit of the light intensity set by the user, the lighting is turned on; a rule for linking motion sensor with security system: if the motion sensor detects abnormal motion and the current time is night, the security system triggers an alarm; a rule for linking air quality with ventilation system: if the air quality index is greater than the set upper limit of the air quality index, the ventilation system is turned on; or a rule for linking time with scene mode: if the current time is at night and it is a weekday, the scene mode is switched to sleep mode. Therefore, on the one hand, based on the fused multi-source heterogeneous data, intelligent home automation control can be achieved. For example, by analyzing sensor data such as temperature, humidity, and light and user preference data, the system can automatically adjust the indoor temperature and lighting to provide a comfortable living experience. Using multi-source heterogeneous data, the smart home system can achieve scene-based control. For example, by analyzing user behavior, environmental sensor data, and device status data, the system can automatically identify the user's scene (such as dining, sleeping, or working), and intelligently adjust the device's working mode and interaction method according to the scene. On the other hand, based on the fused multi-source heterogeneous data, remote management and maintenance of smart devices can be achieved. By analyzing the device's status data, fault data, and user feedback data, the system can monitor the device's operating status in real time, and perform fault detection and early warning to improve the reliability and maintainability of the device. Provide personalized device management services using multi-source heterogeneous data. For example, by analyzing user preference data, device usage records, and environmental data, the system can recommend the smart device that best suits each user's needs and scenarios, and provide a personalized device management and control interface.

[0081] In some implementations, permission control can also be performed using user authentication and dynamic permission adjustment to ensure data security and privacy protection. For example, a user needs to enter the correct username and password or match the correct fingerprint to log in to the smart home system. When a family member leaves home, the access rights to their smart home devices are automatically closed.

[0082] According to an embodiment of the present invention, a multi-source heterogeneous data fusion and management device for smart home scenarios is provided. Figure 4 , the device may include: An acquisition module, which is used to acquire multi-source heterogeneous data in the smart home scene; wherein the multi-source heterogeneous data is acquired from at least two data sources in the smart home scene; A construction module, which is used to construct corresponding graph data based on the multi-source heterogeneous data; wherein the graph data includes a plurality of nodes, and the nodes at least represent the data sources; An extraction module, which is used to extract the feature vector representation data of the node in the graph data for the graph data; wherein the feature vector representation data is used to represent the semantic information and association relationship information of the corresponding node in the graph data; A fusion processing module, which is used to perform fusion processing on the feature vector representation data to obtain fused feature vector representation data; A generation module, which is used to generate corresponding device linkage rules based on the fused feature vector representation data; A response module is used to respond to the received real-time data and issue action instructions to the devices in the smart home scene based on the device linkage rules, so that the devices perform corresponding actions.

[0083] According to an embodiment of the present invention, an electronic device is provided. Figure 5 The electronic device in this embodiment may include one or more of the following components: a processor, a network interface, a memory, a non-volatile memory, and one or more applications, wherein the one or more applications may be stored in the non-volatile memory and configured to be executed by one or more processors, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0084] According to an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer executes the method described in any of the above embodiments.

[0085] According to an embodiment of the present invention, a computer program product comprising instructions is also provided. When the instructions are executed by a computer, the computer executes a multi-source heterogeneous data fusion and management method for a smart home scenario in any of the above-mentioned embodiments.

[0086] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0087] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0088] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0089] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0090] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multi-source heterogeneous data fusion and management method for smart home scenarios, characterized in that: The method comprises: Acquire multi-source heterogeneous data in the smart home scenario; wherein the multi-source heterogeneous data is acquired from at least two data sources in the smart home scenario; Based on the multi-source heterogeneous data, corresponding graph data is constructed; wherein the graph data includes a plurality of nodes, and the nodes at least represent the data sources; For the graph data, extracting feature vector representation data of the node in the graph data; wherein the feature vector representation data is used to represent semantic information and association relationship information of the corresponding node in the graph data; Performing fusion processing on the feature vector representation data to obtain fused feature vector representation data; Based on the fused feature vector representation data, generating corresponding device linkage rules; In response to the received real-time data, based on the device linkage rules, an action instruction is issued to the device in the smart home scene so that the device performs a corresponding action.

2. The method according to claim 1, characterized in that: The step of acquiring multi-source heterogeneous data in the smart home scenario includes: Acquire and store multi-source heterogeneity in at least two data sources in the smart home scenario based on a preset central database; For the multi-source heterogeneous data stored in the central database, at least identifying abnormal data therein; Delete the identified abnormal data.

3. The method according to claim 2, characterized in that The step of acquiring and storing the multi-source heterogeneity of at least two data sources in the smart home scenario based on a preset central database includes: Generate a data acquisition request; wherein the data acquisition request is an acquisition request for acquiring incremental data; Based on a preset time frequency, sending the data acquisition request to at least two data sources in the smart home scene; The data received from at least two data sources are stored in a preset central database; wherein the data received from at least two data sources constitute the multi-source heterogeneous data.

4. The method according to claim 1, characterized in that: The step of constructing corresponding graph data based on the multi-source heterogeneous data includes: Extracting first data and second data from the multi-source heterogeneous data; wherein the first data is data for representing metadata, and the second data is data for representing key information; wherein the key information is at least key information in the data source; Based on the first data and the second data, determining the distance between preset nodes; Edges are generated between nodes that meet the distance threshold to obtain the graph data.

5. The method according to claim 1, characterized in that The step of extracting the feature vector representation data of the node in the graph data for the graph data comprises: Extracting meta-path information from the graph data based on a preset extraction rule; Using the meta-path information, walking in the graph data, and obtaining a walking sequence based on the meta-path; The meta-path-based walk sequence is input into a preset word vector model for processing to obtain feature vector representation data of the node in the graph data.

6. The method according to claim 1, characterized in that Before the step of issuing an action instruction to at least one device in the smart home scenario in response to the received real-time data based on the fused feature vector representation data and a preset action execution library so that the device performs a corresponding action, the method further includes: Storing the fused feature vector representation data in a graph database; wherein the graph database is configured in the cloud; At least one data desensitization method is used to perform desensitization processing on the graph database.

7. The method according to claim 1, characterized in that The device linkage rules include: The rules for linking temperature and air conditioning: If the current indoor temperature is greater than the upper temperature limit set by the user, turn on the air conditioning; The rules for the linkage between light and lighting: If the current light intensity is less than the lower limit of light intensity set by the user, turn on the lighting; Rules for linking motion sensors with security systems: If the motion sensor detects abnormal motion and the current time is nighttime, the security system triggers an alarm; Rules for linking air quality with the ventilation system: If the air quality index is greater than the set air quality index upper limit, the ventilation system is turned on; or Rules for linking time and scene mode: If the current time is evening and it is a weekday, the scene mode switches to sleep mode.

8. A multi-source heterogeneous data fusion and management device for smart home scenarios, characterized in that: The device comprises: An acquisition module, which is used to acquire multi-source heterogeneous data in the smart home scene; wherein the multi-source heterogeneous data is acquired from at least two data sources in the smart home scene; A construction module, which is used to construct corresponding graph data based on the multi-source heterogeneous data; wherein the graph data includes a plurality of nodes, and the nodes at least represent the data sources; An extraction module, which is used to extract the feature vector representation data of the node in the graph data for the graph data; wherein the feature vector representation data is used to represent the semantic information and association relationship information of the corresponding node in the graph data; A fusion processing module, which is used to perform fusion processing on the feature vector representation data to obtain fused feature vector representation data; A generation module, which is used to generate corresponding device linkage rules based on the fused feature vector representation data; A response module is used to respond to the received real-time data and issue action instructions to the devices in the smart home scene based on the device linkage rules, so that the devices perform corresponding actions.

9. An electronic device, characterized in that: include: a memory, and one or more processors communicatively coupled to the memory; The memory stores instructions that can be executed by the one or more processors. The instructions are executed by the one or more processors to enable the one or more processors to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images