Multi-source data information fusion method and system based on Internet of Things protocol

Abstract

The invention discloses a multi-source data information fusion method and system based on an Internet of Things protocol, and belongs to the technical field of data processing, and the multi-source data information fusion method comprises the following steps: obtaining original data of a plurality of data sources based on different Internet of Things protocols, carrying out the preprocessing, and obtaining the preprocessed multi-source heterogeneous data; constructing a multi-source data mapping model based on a semantic ontology, mapping the preprocessed multi-source heterogeneous data to a corresponding node in the semantic ontology, and generating fusion data of a semantic level; constructing a data fusion model based on deep learning, inputting the fusion data of the semantic level into the data fusion model, and generating fusion features through multi-level feature extraction and feature fusion; and inputting the fusion features into a decision model, and outputting a final result after multi-source heterogeneous data fusion in combination with the demand of the Internet of Things application. According to the invention, efficient fusion processing of the multi-source heterogeneous data is realized, and the performance and reliability of the application of the Internet of Things are improved.

Description

Technical Field

[0001] The present invention relates to a multi-source data information fusion method and system based on Internet of Things protocol, belonging to the technical field of data processing. Background Art

[0002] With the rapid development of Internet of Things technology, more and more devices and sensors are connected to the network, generating a massive amount of heterogeneous data. These data come from different devices, sensors and applications, using different communication protocols and data formats, such as ZigBee, Wi-Fi, Bluetooth, RFID, NB-IoT, etc. How to effectively collect, transmit, store and process these multi-source heterogeneous data and realize data fusion and application has become an important challenge facing the field of Internet of Things.

[0003] Traditional data fusion methods mainly target homogeneous data, such as multi-sensor data fusion and image fusion, but lack the ability to effectively process multi-source heterogeneous data. These methods usually assume that the data comes from the same field or application, has similar characteristics and formats, and cannot fully utilize the diversity and complementarity of IoT data.

[0004] In addition, some traditional methods use a centralized data collection and processing architecture to aggregate data to a central node for fusion, which leads to problems such as high communication overhead, high latency, and poor scalability. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a multi-source data information fusion method and system based on the Internet of Things protocol, which can realize efficient fusion processing of multi-source heterogeneous data and improve the performance and reliability of Internet of Things applications.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] In a first aspect, an embodiment of the present invention provides a multi-source data information fusion method based on an Internet of Things protocol, comprising the following steps:

[0008] Step 1, obtaining raw data from multiple data sources based on different IoT protocols, and preprocessing the raw data to obtain preprocessed multi-source heterogeneous data, wherein the IoT protocols include at least NB-IoT protocol, LoRa protocol and ZigBee protocol;

[0009] Step 2: construct a multi-source data mapping model based on the semantic ontology, map the pre-processed multi-source heterogeneous data to corresponding nodes in the semantic ontology, and generate fused data at the semantic level;

[0010] Step 3, constructing a data fusion model based on deep learning, inputting the fusion data at the semantic level into the data fusion model, and generating fusion features through multi-level feature extraction and feature fusion;

[0011] Step 4: Input the fusion features into the decision model, combine them with the needs of IoT applications, and output the final result after fusion of multi-source heterogeneous data.

[0012] As a possible implementation of this embodiment, step 1, obtaining raw data from multiple data sources based on different IoT protocols, and preprocessing the raw data to obtain preprocessed multi-source heterogeneous data, includes the following steps:

[0013] Collect data from multiple data sources based on different IoT protocols to obtain multi-source heterogeneous raw data;

[0014] The multi-source heterogeneous raw data are preprocessed, and the raw data under different IoT protocols are converted into preprocessed multi-source heterogeneous data in a unified format through data conversion.

[0015] As a possible implementation of this embodiment, the step 2, building a multi-source data mapping model based on the semantic ontology, mapping the preprocessed multi-source heterogeneous data to corresponding nodes in the semantic ontology, and generating fused data at the semantic level, includes the following steps:

[0016] Construct a semantic ontology for IoT applications. The semantic ontology is described in OWL language and defines multiple concepts such as data source, sensor, physical quantity and spatiotemporal characteristics, as well as the semantic relationships between concepts.

[0017] Perform attribute extraction on the preprocessed multi-source heterogeneous data to obtain key attribute information reflecting the semantic characteristics of the data and form the attribute vector of the preprocessed multi-source heterogeneous data;

[0018] Calculating the semantic similarity between the preprocessed multi-source heterogeneous data attribute vector and the concept in the semantic ontology, wherein the semantic similarity includes the semantic distance of the attribute and the matching degree of the attribute value;

[0019] According to the calculation result of the semantic similarity, the preprocessed multi-source heterogeneous data is mapped to the concept node with the highest semantic similarity in the semantic ontology, so as to fuse the preprocessed multi-source heterogeneous data of different sources and formats at the semantic level;

[0020] Through the formal representation of semantic ontology and using SPARQL query language, data query, reasoning and analysis are carried out based on the mapped semantic ontology, and multi-source data are integrated at the semantic level to obtain semantically integrated data.

[0021] As a possible implementation of this embodiment, according to the calculation result of the semantic similarity, the preprocessed multi-source heterogeneous data is mapped to the concept node with the highest semantic similarity in the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and different formats are integrated at the semantic level, including the following steps:

[0022] For each preprocessed multi-source heterogeneous data, extract the first attribute information thereof, and construct a first attribute vector based on the first attribute information;

[0023] For each concept in the semantic ontology, extract its second attribute information and form a second attribute vector corresponding to the second attribute information;

[0024] Calculating semantic similarity between the first attribute vector and the second attribute vector;

[0025] Select the ontology concept with the highest semantic similarity as the mapping node of the preprocessed multi-source heterogeneous data;

[0026] The preprocessed multi-source heterogeneous data are mapped to the corresponding mapping nodes of the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and formats are integrated at the semantic level.

[0027] As a possible implementation of this embodiment, step 3, constructing a data fusion model based on deep learning, inputting the fusion data at the semantic level into the data fusion model, and generating fusion features through multi-level feature extraction and feature fusion, includes the following steps:

[0028] Extract key attributes related to the target application from the semantic ontology, convert the attribute values ​​into numerical vectors, normalize the feature vectors, and obtain the input features of the deep learning model;

[0029] Use convolutional neural networks to extract spatial features from input features, and extract high-level spatial features of data by alternately stacking multiple convolutional layers and pooling layers;

[0030] Use the long short-term memory network to extract the time series features of the input features, and extract the high-level time series features of the data through the stacking and attention mechanism of multiple layers of long short-term memory networks;

[0031] The spatial features extracted by the convolutional neural network and the temporal features extracted by the long short-term memory network are concatenated and input into the fully connected layer for further feature extraction and fusion.

[0032] Classify or regress the fused features through the Softmax layer and output the prediction results;

[0033] Based on the loss function, the back propagation algorithm is used to train the model end-to-end, optimize the model parameters, and obtain the trained data fusion model;

[0034] The fused data at the semantic level is input into the trained data fusion model to generate fusion features.

[0035] As a possible implementation of this embodiment, the trained data fusion model is:

[0036]

[0037] in, represents the activation value of the jth neuron in the Lth layer, represents the weighted input of the jth neuron in the Lth layer, represents the connection weight from the jth neuron to the ith neuron in the Lth layer, represents the error term of the j-th neuron in the L-th layer, Represents the activation value of the i-th neuron in the L-th layer.

[0038] As a possible implementation of this embodiment, before step 5, the multi-source data information fusion method based on the Internet of Things protocol further includes the following steps:

[0039] A decision model is constructed, wherein the decision model includes any one of logistic regression, support vector machine, decision tree, random forest and neural network.

[0040] In a second aspect, an embodiment of the present invention provides a multi-source data information fusion system based on the Internet of Things protocol, including:

[0041] A data acquisition module is used to obtain raw data from multiple data sources based on different IoT protocols, and preprocess the raw data to obtain preprocessed multi-source heterogeneous data, wherein the IoT protocols include at least NB-IoT protocol, LoRa protocol and ZigBee protocol;

[0042] A mapping model building module is used to build a multi-source data mapping model based on the semantic ontology, map the pre-processed multi-source heterogeneous data to the corresponding nodes in the semantic ontology, and generate fused data at the semantic level;

[0043] A fusion model construction module is used to construct a data fusion model based on deep learning, input the fusion data at the semantic level into the data fusion model, and generate fusion features through multi-level feature extraction and feature fusion;

[0044] The multi-source data fusion module is used to input the fusion features into the decision model, and output the final result after the fusion of multi-source heterogeneous data in combination with the needs of the Internet of Things application.

[0045] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform any steps of the multi-source data information fusion method based on the Internet of Things protocol as described above.

[0046] In a fourth aspect, an embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned multi-source data information fusion methods based on the Internet of Things protocol are executed.

[0047] The beneficial effects of the technical solution of the embodiment of the present invention are as follows:

[0048] A multi-source data information fusion method based on the Internet of Things protocol of the technical solution of the embodiment of the present invention comprises the following steps: step 1, obtaining the original data of multiple data sources based on different Internet of Things protocols, and preprocessing the original data to obtain the preprocessed multi-source heterogeneous data, wherein the Internet of Things protocol at least includes NB-IoT protocol, LoRa protocol and ZigBee protocol; step 2, constructing a multi-source data mapping model based on semantic ontology, mapping the preprocessed multi-source heterogeneous data to the corresponding nodes in the semantic ontology, and generating fused data at the semantic level; step 3, constructing a data fusion model based on deep learning, inputting the fused data at the semantic level into the data fusion model, and generating fused features through multi-level feature extraction and feature fusion; step 4, inputting the fused features into the decision model, and outputting the final result after the fusion of multi-source heterogeneous data in combination with the needs of Internet of Things applications. The present invention realizes efficient fusion processing of multi-source heterogeneous data through steps such as data preprocessing, semantic ontology mapping, deep learning data fusion and decision model, and improves the performance and reliability of Internet of Things applications.

[0049] A multi-source data information fusion system based on the Internet of Things protocol of the technical solution of an embodiment of the present invention has the same beneficial effects as a multi-source data information fusion method based on the Internet of Things protocol of the technical solution of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of a multi-source data information fusion method based on the Internet of Things protocol according to an exemplary embodiment;

[0051] Figure 2 It is a structural diagram of a multi-source data information fusion system based on the Internet of Things protocol according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] In order to more clearly illustrate the technical features of the solution of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0053] like Figure 1 As shown, a multi-source data information fusion method based on the Internet of Things protocol provided by an embodiment of the present invention includes the following steps:

[0054] Step 1, obtaining raw data from multiple data sources based on different IoT protocols, and preprocessing the raw data to obtain preprocessed multi-source heterogeneous data, wherein the IoT protocols include at least NB-IoT protocol, LoRa protocol and ZigBee protocol;

[0055] Step 2: construct a multi-source data mapping model based on the semantic ontology, map the pre-processed multi-source heterogeneous data to corresponding nodes in the semantic ontology, and generate fused data at the semantic level;

[0056] Step 3, constructing a data fusion model based on deep learning, inputting the fusion data at the semantic level into the data fusion model, and generating fusion features through multi-level feature extraction and feature fusion;

[0057] Step 4: Input the fusion features into the decision model, combine them with the needs of IoT applications, and output the final result after fusion of multi-source heterogeneous data.

[0058] This embodiment achieves efficient fusion processing of multi-source heterogeneous data through steps such as data preprocessing, semantic ontology mapping, deep learning data fusion, and decision-making models, thereby improving the performance and reliability of IoT applications.

[0059] As a possible implementation of this embodiment, step 1, obtaining raw data from multiple data sources based on different IoT protocols, and preprocessing the raw data to obtain preprocessed multi-source heterogeneous data, includes the following steps:

[0060] Collect data from multiple data sources based on different IoT protocols to obtain multi-source heterogeneous raw data;

[0061] The multi-source heterogeneous raw data are preprocessed, and the raw data under different IoT protocols are converted into preprocessed multi-source heterogeneous data in a unified format through data conversion.

[0062] As a possible implementation of this embodiment, the step 2, building a multi-source data mapping model based on the semantic ontology, mapping the preprocessed multi-source heterogeneous data to corresponding nodes in the semantic ontology, and generating fused data at the semantic level, includes the following steps:

[0063] Construct a semantic ontology for IoT applications. The semantic ontology is described in OWL language and defines multiple concepts such as data source, sensor, physical quantity and spatiotemporal characteristics, as well as the semantic relationships between concepts.

[0064] Perform attribute extraction on the preprocessed multi-source heterogeneous data to obtain key attribute information reflecting the semantic characteristics of the data and form the attribute vector of the preprocessed multi-source heterogeneous data;

[0065] Calculating the semantic similarity between the preprocessed multi-source heterogeneous data attribute vector and the concept in the semantic ontology, wherein the semantic similarity includes the semantic distance of the attribute and the matching degree of the attribute value;

[0066] According to the calculation result of the semantic similarity, the preprocessed multi-source heterogeneous data is mapped to the concept node with the highest semantic similarity in the semantic ontology, so as to fuse the preprocessed multi-source heterogeneous data of different sources and formats at the semantic level;

[0067] Through the formal representation of semantic ontology and using SPARQL query language, data query, reasoning and analysis are carried out based on the mapped semantic ontology, and multi-source data are integrated at the semantic level to obtain semantically integrated data.

[0068] As a possible implementation of this embodiment, according to the calculation result of the semantic similarity, the preprocessed multi-source heterogeneous data is mapped to the concept node with the highest semantic similarity in the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and different formats are integrated at the semantic level, including the following steps:

[0069] For each preprocessed multi-source heterogeneous data, extract the first attribute information thereof, and construct a first attribute vector based on the first attribute information;

[0070] For each concept in the semantic ontology, extract its second attribute information and form a second attribute vector corresponding to the second attribute information;

[0071] Calculating semantic similarity between the first attribute vector and the second attribute vector;

[0072] Select the ontology concept with the highest semantic similarity as the mapping node of the preprocessed multi-source heterogeneous data;

[0073] The preprocessed multi-source heterogeneous data are mapped to the corresponding mapping nodes of the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and formats are integrated at the semantic level.

[0074] As a possible implementation of this embodiment, step 3, constructing a data fusion model based on deep learning, inputting the fusion data at the semantic level into the data fusion model, and generating fusion features through multi-level feature extraction and feature fusion, includes the following steps:

[0075] Extract key attributes related to the target application from the semantic ontology, convert the attribute values ​​into numerical vectors, normalize the feature vectors, and obtain the input features of the deep learning model;

[0076] Use convolutional neural networks to extract spatial features from input features, and extract high-level spatial features of data by alternately stacking multiple convolutional layers and pooling layers;

[0077] Use the long short-term memory network to extract the time series features of the input features, and extract the high-level time series features of the data through the stacking and attention mechanism of multiple layers of long short-term memory networks;

[0078] The spatial features extracted by the convolutional neural network and the temporal features extracted by the long short-term memory network are concatenated and input into the fully connected layer for further feature extraction and fusion.

[0079] Classify or regress the fused features through the Softmax layer and output the prediction results;

[0080] Based on the loss function, the back propagation algorithm is used to train the model end-to-end, optimize the model parameters, and obtain the trained data fusion model;

[0081] The fused data at the semantic level is input into the trained data fusion model to generate fusion features.

[0082] As a possible implementation of this embodiment, the trained data fusion model is:

[0083]

[0084] in, represents the activation value of the jth neuron in the Lth layer, represents the weighted input of the jth neuron in the Lth layer, represents the connection weight from the jth neuron to the ith neuron in the Lth layer, represents the error term of the j-th neuron in the L-th layer, Represents the activation value of the i-th neuron in the L-th layer.

[0085] As a possible implementation of this embodiment, before step 5, the multi-source data information fusion method based on the Internet of Things protocol further includes the following steps:

[0086] A decision model is constructed, wherein the decision model includes any one of logistic regression, support vector machine, decision tree, random forest and neural network.

[0087] like Figure 2 As shown, an embodiment of the present invention provides a multi-source data information fusion system based on the Internet of Things protocol, including:

[0088] A data acquisition module is used to obtain raw data from multiple data sources based on different IoT protocols, and preprocess the raw data to obtain preprocessed multi-source heterogeneous data, wherein the IoT protocols include at least NB-IoT protocol, LoRa protocol and ZigBee protocol;

[0089] A mapping model building module is used to build a multi-source data mapping model based on the semantic ontology, map the pre-processed multi-source heterogeneous data to the corresponding nodes in the semantic ontology, and generate fused data at the semantic level;

[0090] A fusion model construction module is used to construct a data fusion model based on deep learning, input the fusion data at the semantic level into the data fusion model, and generate fusion features through multi-level feature extraction and feature fusion;

[0091] The multi-source data fusion module is used to input the fusion features into the decision model, and output the final result after the fusion of multi-source heterogeneous data in combination with the needs of the Internet of Things application.

[0092] The specific process of using the system described in the present invention to perform multi-source data information fusion based on the Internet of Things protocol is as follows.

[0093] 1. Obtain raw data from multiple data sources based on different IoT protocols.

[0094] The IoT protocol includes at least NB-IoT protocol, LoRa protocol and ZigBee protocol. Data is collected from multiple data sources based on different IoT protocols to obtain multi-source heterogeneous original data.

[0095] 2. Preprocess multi-source heterogeneous raw data.

[0096] Noise data in multi-source heterogeneous raw data is removed through data cleaning, and raw data under different IoT protocols are converted into pre-processed multi-source heterogeneous data in a unified format through data conversion.

[0097] 3. Construct a multi-source data mapping model based on semantic ontology.

[0098] The preprocessed multi-source heterogeneous data are mapped to corresponding nodes in the semantic ontology using a multi-source data mapping model to generate fused data at the semantic level.

[0099] The specific process of building a multi-source data mapping model is as follows:

[0100] 3.1. Construct a semantic ontology for IoT applications. The semantic ontology is described in OWL language and defines core concepts such as data source, sensor, physical quantity and spatiotemporal characteristics, as well as the semantic relationships between concepts.

[0101] 3.2. Perform attribute extraction on the preprocessed multi-source heterogeneous data to obtain key attribute information reflecting the semantic characteristics of the data and form the attribute vector of the preprocessed multi-source heterogeneous data;

[0102] 3.3. Calculate the semantic similarity between the preprocessed multi-source heterogeneous data attribute vector and the concept in the semantic ontology, wherein the calculation of the semantic similarity takes into account the semantic distance of the attribute and the matching degree of the attribute value;

[0103] 3.4. According to the calculation result of the semantic similarity, the preprocessed multi-source heterogeneous data is mapped to the concept node with the highest semantic similarity in the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and formats are integrated at the semantic level;

[0104] 3.5. Through the formal representation of semantic ontology and using SPARQL query language, data query, reasoning and analysis are performed based on the mapped semantic ontology to fuse multi-source data at the semantic level and obtain semantically fused data.

[0105] Step 3.4 specifically includes the following steps:

[0106] For each preprocessed multi-source heterogeneous data, extract the first attribute information thereof, and construct a first attribute vector based on the first attribute information;

[0107] For each concept in the semantic ontology, extract its second attribute information and form a second attribute vector corresponding to the second attribute information;

[0108] Calculating semantic similarity between the first attribute vector and the second attribute vector;

[0109] Select the ontology concept with the highest semantic similarity as the mapping node of the preprocessed multi-source heterogeneous data;

[0110] The preprocessed multi-source heterogeneous data are mapped to the corresponding mapping nodes of the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and formats are integrated at the semantic level.

[0111] In order to achieve effective fusion of preprocessed multi-source heterogeneous data from different sources and formats at the semantic level, this paper proposes a multi-source data mapping model based on semantic ontology. The model uses semantic ontology to describe the semantic association between data sources, and realizes semantic layer fusion of data by mapping preprocessed multi-source heterogeneous data to corresponding nodes of semantic ontology.

[0112] First, according to the characteristics of multi-source IoT data, a semantic ontology for IoT applications is constructed. This paper uses OWL (Web Ontology Language) to describe the semantic ontology, defines core concepts such as data source, sensor, physical quantity, spatiotemporal characteristics, and semantic relationships between concepts, such as "hasDataSource", "measuredBy", "hasLocation", etc. Through the semantic ontology, the semantic connections between different data sources can be formally represented.

[0113] On the basis of constructing semantic ontology, a mapping algorithm from multi-source data to semantic ontology is designed. For each preprocessed multi-source heterogeneous data, its key attribute information is extracted, such as data source type, sensor type, measured physical quantity, timestamp, geographic location, etc. Then, using the semantic ontology reasoning mechanism, the attribute information is matched with the concepts in the ontology to find the best mapping node of the preprocessed multi-source heterogeneous data in the semantic ontology. In this way, the preprocessed multi-source heterogeneous data from different sources and formats are mapped to the corresponding nodes of the semantic ontology, realizing the semantic association of the data.

[0114] In order to improve the accuracy and efficiency of the mapping algorithm, the present invention introduces a mapping optimization strategy based on semantic similarity. By calculating the semantic similarity between the preprocessed multi-source heterogeneous data attributes and the ontology concepts, the node with the highest similarity is selected as the final mapping node. The calculation of semantic similarity takes into account factors such as the semantic distance of concepts and the degree of attribute matching, which effectively improves the accuracy of data mapping.

[0115] After data mapping is implemented, the nodes in the semantic ontology already contain pre-processed multi-source heterogeneous data from different data sources. Through the formal representation of the semantic ontology, data from different data sources are integrated at the semantic level. This semantic-based fusion method overcomes the challenge of data heterogeneity and provides a unified semantic representation for subsequent data analysis and application.

[0116] The present invention verifies the effectiveness of the proposed multi-source data mapping model through experiments. In the experiment, different types of IoT data sources, such as temperature sensors, humidity sensors, GPS positioning devices, etc., were selected, and corresponding semantic ontologies were constructed. By comparing the semantic consistency of data before and after mapping, it is proved that the model can effectively map multi-source heterogeneous data into a unified semantic space and realize the semantic fusion of data. After experimental verification, the present invention has good performance and application prospects in dealing with the problem of multi-source IoT data fusion.

[0117] In order to achieve semantic fusion of multi-source heterogeneous data, it is first necessary to build a semantic ontology for IoT applications. Semantic ontology is a formal representation of domain knowledge, which defines the core concepts, attributes and semantic relationships between concepts in the domain. The present invention uses OWL (Web Ontology Language) to describe the semantic ontology. Through the rich semantic expression ability of OWL, the semantic characteristics of IoT data can be accurately characterized.

[0118] The construction process of semantic ontology is as follows:

[0119] (1) Determine core concepts: According to the characteristics of IoT application fields, determine the core concepts in the semantic ontology, such as data source, sensor, physical quantity, spatiotemporal characteristics, etc. These concepts reflect the basic semantic elements of IoT data.

[0120] (2) Define the concept hierarchy: Based on the core concepts, construct the concept hierarchy. Through the is-a relationship, the concepts are organized into a tree structure to express the inheritance and generalization relationship between concepts. For example, the sensor concept can be further divided into sub-concepts such as temperature sensor and humidity sensor.

[0121] (3) Define concept attributes: Define the relevant attributes for each concept to describe the characteristics of the concept. Attributes can be divided into data attributes and object attributes. Data attributes represent the data characteristics of a concept, such as the measurement accuracy of a sensor; object attributes represent the association between concepts, such as the "hasDataSource" relationship between a data source and a sensor.

[0122] (4) Define attribute constraints: Constrain concept attributes and limit the value range and cardinality of the attributes. Attribute constraints can ensure the rationality and consistency of the semantic ontology. For example, it can be stipulated that the measurement accuracy of the sensor must be a positive number and each sensor can only belong to one data source.

[0123] (5) Adding semantic relations: Based on concepts and attributes, define semantic relations between concepts. Common semantic relations include equivalentClass, disjointWith, unionOf, etc. Through semantic relations, semantic characteristics such as equivalence, mutual exclusion, and combination between concepts can be characterized.

[0124] Through the above steps, a semantic ontology for IoT applications is constructed, which provides a basis for semantic mapping of multi-source data.

[0125] After constructing the semantic ontology, it is necessary to map the preprocessed multi-source heterogeneous data to the corresponding nodes in the semantic ontology to achieve the semantic fusion of data. The present invention proposes a mapping algorithm based on semantic similarity, which finds the best mapping node by calculating the semantic similarity between the preprocessed multi-source heterogeneous data and the ontology concepts.

[0126] The steps for semantic mapping of multi-source data are as follows:

[0127] (1) Attribute extraction: For each preprocessed multi-source heterogeneous data, extract its key attribute information, such as data source type, sensor type, measured physical quantity, timestamp, geographic location, etc. These attributes reflect the semantic characteristics of the data.

[0128] (2) Semantic similarity calculation: Calculate the semantic similarity between the attributes of preprocessed multi-source heterogeneous data and the concepts in the semantic ontology. The calculation of semantic similarity takes into account factors such as the semantic distance of the attributes and the matching degree of the attribute values. Semantic similarity can be calculated using methods based on information content or attributes.

[0129] (3) Mapping node selection: According to the calculation results of semantic similarity, the ontology concept with the highest similarity is selected as the mapping node of the preprocessed multi-source heterogeneous data. In this way, the preprocessed multi-source heterogeneous data of different sources and formats are mapped to the corresponding nodes in the semantic ontology.

[0130] (4) Mapping optimization: In order to improve the accuracy and efficiency of mapping, some optimization strategies can be adopted. For example, threshold filtering can be introduced to eliminate mapping results with semantic similarity below the threshold; weighted calculation can be used to assign different weights to different attributes to highlight the influence of key attributes; and ontology reasoning mechanism can be used to further verify and adjust the mapping results according to the semantic relationship in the ontology.

[0131] (5) Semantic fusion: After completing data mapping, the nodes in the semantic ontology already contain pre-processed multi-source heterogeneous data from different data sources. Through the formal representation of the semantic ontology, data from different data sources are integrated at the semantic level. The SPARQL query language can be used to query, reason, and analyze data based on the semantic ontology, thereby realizing the semantic-level fusion application of multi-source data.

[0132] In order to realize semantic mapping of multi-source data, the present invention proposes a mapping algorithm based on semantic similarity. The core idea of ​​the mapping algorithm is to find the best mapping node by calculating the semantic similarity between the pre-processed multi-source heterogeneous data attributes and the ontology concepts.

[0133] The implementation steps of the mapping algorithm are as follows:

[0134] (1) For each preprocessed multi-source heterogeneous data, extract its key attribute information to form an attribute vector A = {a1, a2, ..., ai ..., an}, where ai represents the value of the i-th attribute.

[0135] (2) For each concept in the semantic ontology, extract its key attribute information to form an attribute vector B = {b1, b2, ..., bj ..., bm}, where bj represents the value of the jth attribute.

[0136] (3) Calculate the semantic similarity Sim(A,B) between attribute vectors A and B. The formula for calculating semantic similarity is as follows:

[0137] Sim(A,B)=∑(i=1to n)wi×sim(ai,bi) / ∑(i=1to n)wi;

[0138] Among them, wi represents the weight of the i-th attribute, and sim(ai,bi) represents the similarity between the attribute values ​​ai and bi, which can be calculated using methods such as string matching and semantic distance.

[0139] (4) Select the ontology concept with the highest semantic similarity as the mapping node of the preprocessed multi-source heterogeneous data.

[0140] (5) Optimize and adjust the mapping results, such as introducing threshold filtering, using ontology reasoning and other strategies to improve the accuracy of the mapping.

[0141] (6) Map the preprocessed multi-source heterogeneous data to the corresponding nodes of the semantic ontology to achieve semantic fusion of data.

[0142] The time complexity of the algorithm is O(n×m), where n is the number of attributes of the preprocessed multi-source heterogeneous data and m is the number of attributes of the ontology concept. The efficiency of the algorithm can be improved by optimizing strategies such as attribute indexing and parallel computing.

[0143] 4. Build a data fusion model based on deep learning.

[0144] The fused data at the semantic level is input into the data fusion model, and fused features are generated through multi-level feature extraction and feature fusion.

[0145] The specific implementation process is as follows:

[0146] 4.1. Extract key attributes related to the target application from the semantic ontology, convert the attribute values ​​into numerical vectors, normalize the feature vectors, and obtain the input features of the deep learning model;

[0147] 4.2. Use convolutional neural networks to extract spatial features from input features. By alternately stacking multiple convolutional layers and pooling layers, high-level spatial features of the data are extracted.

[0148] 4.3. Use the long short-term memory network to extract the time series features of the input features. Through the stacking of multiple layers of long short-term memory networks and the attention mechanism, the high-level time series features of the data are extracted;

[0149] 4.4. Concatenate the spatial features extracted by the convolutional neural network and the temporal features extracted by the long short-term memory network, and input them into the fully connected layer for further feature extraction and fusion;

[0150] 4.5. Classify or regress the fused features through the Softmax layer and output the prediction results;

[0151] 4.6. Based on the loss function, the back propagation algorithm is used to train the model end-to-end and optimize the model parameters to obtain the trained data fusion model:

[0152]

[0153] in, represents the activation value of the jth neuron in the Lth layer, represents the weighted input of the jth neuron in the Lth layer, represents the connection weight from the jth neuron to the ith neuron in the Lth layer, represents the error term of the j-th neuron in the L-th layer, represents the activation value of the i-th neuron in the L-th layer;

[0154] 4.7. Input the fused data at the semantic level into the trained data fusion model to generate fusion features.

[0155] After mapping multi-source heterogeneous data to semantic ontology to achieve semantic fusion, it is necessary to further use deep learning technology to build a data fusion model, and generate higher-level fusion features through multi-level feature extraction and feature fusion to support intelligent decision-making and application. This paper proposes a deep data fusion model based on convolutional neural network (CNN) and long short-term memory network (LSTM), which uses CNN to extract spatial features of data, uses LSTM to extract temporal features of data, and generates the final fusion features through feature fusion.

[0156] The input of the data fusion model comes from the fused data in the semantic ontology. Through semantic mapping, the original data from different sources and formats have been fused in the semantic ontology to form a unified semantic representation. In order to adapt to the input of the deep learning model, the fused data in the semantic ontology needs to be feature vectorized. The specific steps are as follows:

[0157] (1) Attribute selection: Select key attributes related to the target application from the semantic ontology, such as sensor type, measured physical quantity, timestamp, geographic location, etc. These attributes will serve as input features of the data fusion model.

[0158] (2) Feature vectorization: Feature vectorization is performed on the selected attributes to convert the attribute values ​​into numerical vectors. For categorical attributes, such as sensor types, one-hot encoding can be used to convert them into binary vectors; for numerical attributes, such as measurement values, their numerical values ​​can be used directly; for timestamp attributes, they can be converted into time interval or time period features.

[0159] (3) Data normalization: Normalize the feature vectors to unify the value ranges of different attributes to the same scale, such as the interval [0,1]. Normalization can use methods such as maximum and minimum value normalization and Z-score normalization.

[0160] Through the above steps, the fused data in the semantic ontology is converted into a feature vector adapted to the deep learning model as the input of the data fusion model.

[0161] IoT data usually has obvious spatial correlation, such as the geographical location relationship between sensors. In order to extract the spatial features of the data, the present invention uses a convolutional neural network (CNN) to extract features from the input data. CNN can effectively capture the spatial local features of the data through local connections and weight sharing.

[0162] The steps of spatial feature extraction based on CNN are as follows:

[0163] (1) Input layer: The fused data after feature vectorization is input into the input layer of CNN. The spatial dimension of the data corresponds to the number of input channels of CNN.

[0164] (2) Convolution layer: Through the convolution operation, multiple convolution kernels are used to slide on the input data to extract the local features of the data. The size and number of convolution kernels can be set according to the characteristics of the data. The convolution operation can use different step sizes and padding methods to control the size of the feature map.

[0165] (3) Activation function: After the convolution layer, a nonlinear activation function, such as ReLU (Rectified Linear Unit), is introduced to increase the nonlinear expression ability of the network. The activation function performs nonlinear transformation on the feature map to improve the representation ability of the feature.

[0166] (4) Pooling layer: Through the pooling operation, the feature map output by the convolutional layer is downsampled to reduce the size of the feature map while retaining important feature information. Commonly used pooling operations include max pooling and average pooling.

[0167] (5) Multi-layer convolution: By alternately stacking multiple convolutional layers and pooling layers, high-level spatial features of the data are gradually extracted. As the network depth increases, the features extracted by the convolutional layer become more abstract and advanced.

[0168] Through CNN-based spatial feature extraction, a high-dimensional feature vector representing the spatial correlation of the data is obtained, laying the foundation for subsequent feature fusion.

[0169] IoT data is usually time series data with obvious time correlation. In order to extract the time series features of the data, the present invention uses a long short-term memory network (LSTM) to extract features from the input data. LSTM is a variant of a recurrent neural network (RNN) that can effectively capture the long-term and short-term dependencies of data by introducing a gating mechanism.

[0170] The steps of LSTM-based time series feature extraction are as follows:

[0171] (1) Input layer: The fused data after feature vectorization is input into the input layer of LSTM in chronological order, and each time step corresponds to a feature vector.

[0172] (2) LSTM layer: The input data is processed through the LSTM unit to extract the temporal features of the data. The LSTM unit contains an input gate, a forget gate, an output gate, and a memory unit, and controls the flow and update of information through a gating mechanism. At each time step, the LSTM unit updates the memory unit and hidden state based on the current input and the state of the previous time step to capture the temporal dependency of the data.

[0173] (3) Multi-layer LSTM: By stacking multiple layers of LSTM, the high-level temporal features of the data are gradually extracted. The output of each layer of LSTM is used as the input of the next layer of LSTM. Through hierarchical feature extraction, the long-term and short-term dependencies of the data are captured.

[0174] (4) Attention mechanism: After the LSTM layer, the attention mechanism is introduced to assign different weights to features at different time steps to highlight important temporal information. The attention mechanism adaptively adjusts the importance of features at different time steps by learning weights, thereby improving the representation ability of temporal features.

[0175] Through LSTM-based time series feature extraction, a high-dimensional feature vector representing the time correlation of the data is obtained, which provides a representation of time series features for subsequent feature fusion.

[0176] After obtaining the spatial and temporal features of the data, feature fusion is required to generate the final fused features. The purpose of feature fusion is to comprehensively utilize feature information of different dimensions to obtain a more comprehensive and discriminative data representation.

[0177] The steps of feature fusion and model output are as follows:

[0178] (1) Feature concatenation: The spatial features extracted by CNN and the temporal features extracted by LSTM are concatenated to form a high-dimensional feature vector. The concatenation operation can be performed by simple vector connection or more complex feature interaction methods, such as feature multiplication or feature outer product.

[0179] (2) Fully connected layer: The concatenated high-dimensional feature vector is input into the fully connected layer to further extract and fuse features through nonlinear transformation and weight learning.

[0180] 5. Build a decision-making model.

[0181] The decision model includes any one of logistic regression, support vector machine, decision tree, random forest, and neural network.

[0182] In order to use fusion features to support intelligent decision-making in the Internet of Things, it is necessary to build a corresponding decision model. The steps to build a decision model are as follows:

[0183] 5.1. Decision problem definition: clarify the decision problem of the IoT application and determine the decision objectives and constraints. The decision problem can be a classification problem, regression problem, or optimization problem, etc., and needs to be defined according to the application requirements.

[0184] 5.2. Feature selection and processing: Select fusion features related to the decision problem and further process the features, such as normalization and standardization. Feature selection and processing can improve the performance and generalization ability of the decision model.

[0185] 5.3 Model selection and design: According to the type and characteristics of the decision problem, select appropriate machine learning or deep learning models, such as support vector machines, decision trees, neural networks, etc. Design the model architecture and hyperparameters to adapt to specific decision-making tasks.

[0186] 5.4. Model training and optimization: Use training data to train the decision model, adjust the model parameters through optimization algorithms (such as gradient descent), and minimize the decision error. During the training process, use cross-validation, regularization and other techniques to avoid model overfitting.

[0187] 5.5. Model evaluation and tuning: Evaluate the performance of the decision model on the test data and calculate the evaluation indicators (such as accuracy, precision, recall, etc.). Tune the model according to the evaluation results, such as adjusting feature selection, optimizing model architecture, etc., to improve the performance of the decision model.

[0188] 5.6. Generation of decision results: Input the fusion features into the trained decision model to generate the final decision results. The decision results can be prediction labels, prediction values ​​or optimization solutions, etc., and are output according to the needs of IoT applications.

[0189] 5.7. Explanation of decision results: Explain and analyze the decision results, and clarify the decision basis and influencing factors. Explanation of decision results can enhance the credibility and transparency of the decision, and facilitate users to understand and accept the decision results.

[0190] Through the above steps, a decision model based on fusion features is constructed, which combines the fusion features with the specific needs of IoT applications and outputs the final result after fusion of multi-source heterogeneous data. The construction and application of the decision model realizes the complete process from data to decision-making and supports the intelligent application of IoT.

[0191] In order to more intuitively illustrate the application of fusion features and the construction of decision-making models, a case analysis of a smart home scenario is given below.

[0192] In smart homes, a variety of sensors (such as temperature, humidity, light, sound, etc.) and devices (such as smart door locks, smart sockets, smart cameras, etc.) are deployed. The multi-source heterogeneous data generated by these sensors and devices need to be integrated and analyzed to support various applications of smart homes, such as environmental monitoring, security management, energy optimization, etc.

[0193] (1) Data fusion: semantic mapping and feature extraction are performed on multi-source heterogeneous data in smart homes, and the data fusion method based on deep learning proposed in this paper is used to generate high-quality fusion features. The fusion features integrate the data from different sensors and devices to represent the overall status and characteristics of the smart home.

[0194] (2) Anomaly detection: Select fusion features related to anomaly detection, such as temperature, humidity, door and window status, etc., and build an anomaly detection model. When the fusion features show abnormal patterns, the model can detect potential safety hazards in real time, such as fire, illegal intrusion, etc., and issue an alarm in time.

[0195] (3) Energy optimization: Select fusion features related to energy consumption, such as appliance usage status and lighting control, to build an energy optimization model. By analyzing user behavior patterns and environmental conditions, the model can automatically adjust the working status of appliances and lighting to achieve efficient energy utilization while meeting user needs.

[0196] (4) Context awareness: Select fusion features related to user activities, such as sound, video, location, etc., to build a context awareness model. By identifying the user's daily activities and behavioral habits, the model can automatically adjust the settings of the smart home, such as adjusting the indoor temperature, playing music, turning on the TV, etc., to provide personalized services.

[0197] 6. Generate fusion data required for Internet of Things applications based on the fusion features.

[0198] The fused features are input into the decision model, and combined with the needs of IoT applications, the final result after fusion of multi-source heterogeneous data is output.

[0199] After obtaining high-quality fusion features, they need to be applied to specific tasks and decisions in the Internet of Things. Next, we will explain in detail how to generate the fusion data required for Internet of Things applications based on fusion features, input the fusion features into the decision model, and output the final result after multi-source heterogeneous data fusion in combination with the specific needs of Internet of Things applications.

[0200] Fusion features are high-level representations obtained by extracting and fusing features from multi-source heterogeneous data through deep learning models. These fusion features contain the essential characteristics and key information of the data and can be directly applied to various tasks of the Internet of Things, such as state monitoring, anomaly detection, event recognition, etc.

[0201] The steps to generate the fusion data required for IoT applications based on the fusion features are as follows:

[0202] 6.1. Selection of fusion features: According to the needs of IoT applications, select fusion features related to the task. Different applications may focus on different data features, and it is necessary to select fusion features in a targeted manner according to the characteristics of the application.

[0203] 6.2. Interpretation of fused features: Interpret and analyze the selected fused features to understand the physical meaning of each feature and the data attributes it represents. Fusion features are usually abstract high-level representations that need to be interpreted in conjunction with specific IoT applications.

[0204] 6.3. Generation of fusion data: Generate the fusion data required for IoT applications based on the selected fusion features. Fusion data is the specific embodiment of fusion features, which can be state quantities, event labels, anomaly scores, etc., and is generated according to application requirements.

[0205] 6.4 Visualization of fused data: Visualize the generated fused data to present the characteristics and trends of the data in an intuitive way. Visualization can be in the form of charts, dashboards, etc., to facilitate users to understand and interpret the fused data.

[0206] Through the above steps, the fusion features are applied to the specific tasks of the Internet of Things to generate the fusion data required for the application. The fusion data can be directly used for tasks such as status monitoring and anomaly detection, or it can be used as the input of the decision model to further support intelligent decision-making.

[0207] An electronic device provided by an embodiment of the present invention includes a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform any step of the multi-source data information fusion method based on the Internet of Things protocol as described above.

[0208] Specifically, the above-mentioned memory and processor can be general-purpose memory and processor, which are not specifically limited here. When the processor runs the computer program stored in the memory, it can execute the above-mentioned multi-source data information fusion method based on the Internet of Things protocol.

[0209] Those skilled in the art will appreciate that the structure of the electronic device does not limit the electronic device and may include more or fewer components than shown in the figure, or combine or split certain components, or arrange the components differently.

[0210] In some embodiments, the electronic device may also include a touch screen that can be used to display a graphical user interface (e.g., a startup interface of an application) and receive user operations on the graphical user interface (e.g., startup operations on an application). The specific touch screen may include a display panel and a touch panel. The display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. The touch panel may collect the user's contact or non-contact operations on or near it, and generate pre-set operation instructions, for example, the user uses any suitable object such as a finger, stylus, or accessories on or near the touch panel. In addition, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation and posture, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into information that the processor can process, and then sends it to the processor, and can receive and execute commands from the processor. In addition, the touch panel can be implemented by various types such as resistive, capacitive, infrared and surface acoustic wave, and any technology developed in the future can also be used to implement the touch panel. Further, the touch panel can cover the display panel, and the user can operate on or near the touch panel covered on the display panel according to the graphical user interface displayed on the display panel. After the touch panel detects the operation on or near it, it is transmitted to the processor to determine the user input, and then the processor provides corresponding visual output on the display panel in response to the user input. In addition, the touch panel and the display panel can be implemented as two independent components or integrated.

[0211] Corresponding to the method for starting the above-mentioned application, an embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is run by a processor, the steps of any of the above-mentioned multi-source data information fusion methods based on the Internet of Things protocol are executed.

[0212] The application startup device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0213] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0214] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0215] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0216] In addition, each functional module in the embodiments provided in the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0217] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0218] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0219] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A multi-source data information fusion method based on the Internet of Things protocol, characterized in that: The steps include: Step 1, obtaining raw data from multiple data sources based on different IoT protocols, and preprocessing the raw data to obtain preprocessed multi-source heterogeneous data, wherein the IoT protocols include at least NB-IoT protocol, LoRa protocol and ZigBee protocol; Step 2: construct a multi-source data mapping model based on the semantic ontology, map the pre-processed multi-source heterogeneous data to corresponding nodes in the semantic ontology, and generate fused data at the semantic level; Step 3, constructing a data fusion model based on deep learning, inputting the fusion data at the semantic level into the data fusion model, and generating fusion features through multi-level feature extraction and feature fusion; Step 4: Input the fusion features into the decision model, combine them with the needs of IoT applications, and output the final result after fusion of multi-source heterogeneous data.

2. The multi-source data information fusion method based on the Internet of Things protocol according to claim 1 is characterized in that: The step 1, obtaining raw data from multiple data sources based on different IoT protocols, and preprocessing the raw data to obtain preprocessed multi-source heterogeneous data, includes the following steps: Collect data from multiple data sources based on different IoT protocols to obtain multi-source heterogeneous raw data; The multi-source heterogeneous raw data are preprocessed, and the raw data under different IoT protocols are converted into preprocessed multi-source heterogeneous data in a unified format through data conversion.

3. The multi-source data information fusion method based on the Internet of Things protocol according to claim 1 is characterized in that: The step 2, constructing a multi-source data mapping model based on the semantic ontology, mapping the pre-processed multi-source heterogeneous data to corresponding nodes in the semantic ontology, and generating fused data at the semantic level, includes the following steps: Construct a semantic ontology for IoT applications. The semantic ontology is described in OWL language and defines multiple concepts such as data source, sensor, physical quantity and spatiotemporal characteristics, as well as the semantic relationships between concepts. Perform attribute extraction on the preprocessed multi-source heterogeneous data to obtain key attribute information reflecting the semantic characteristics of the data and form the attribute vector of the preprocessed multi-source heterogeneous data; Calculating the semantic similarity between the preprocessed multi-source heterogeneous data attribute vector and the concept in the semantic ontology, wherein the semantic similarity includes the semantic distance of the attribute and the matching degree of the attribute value; According to the calculation result of the semantic similarity, the preprocessed multi-source heterogeneous data is mapped to the concept node with the highest semantic similarity in the semantic ontology, so as to fuse the preprocessed multi-source heterogeneous data of different sources and formats at the semantic level; Through the formal representation of semantic ontology and using SPARQL query language, data query, reasoning and analysis are carried out based on the mapped semantic ontology, and multi-source data are integrated at the semantic level to obtain semantically integrated data.

4. The multi-source data information fusion method based on the Internet of Things protocol according to claim 3 is characterized in that: According to the calculation result of the semantic similarity, the preprocessed multi-source heterogeneous data is mapped to the concept node with the highest semantic similarity in the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and formats are integrated at the semantic level, including the following steps: For each preprocessed multi-source heterogeneous data, extract the first attribute information thereof, and construct a first attribute vector based on the first attribute information; For each concept in the semantic ontology, extract its second attribute information and form a second attribute vector corresponding to the second attribute information; Calculating semantic similarity between the first attribute vector and the second attribute vector; Select the ontology concept with the highest semantic similarity as the mapping node of the preprocessed multi-source heterogeneous data; The preprocessed multi-source heterogeneous data are mapped to the corresponding mapping nodes of the semantic ontology, and the preprocessed multi-source heterogeneous data of different sources and formats are integrated at the semantic level.

5. The multi-source data information fusion method based on the Internet of Things protocol according to claim 1 is characterized in that: The step 3, constructing a data fusion model based on deep learning, inputting the fusion data at the semantic level into the data fusion model, and generating fusion features through multi-level feature extraction and feature fusion, includes the following steps: Extract key attributes related to the target application from the semantic ontology, convert the attribute values ​​into numerical vectors, normalize the feature vectors, and obtain the input features of the deep learning model; Use convolutional neural networks to extract spatial features from input features, and extract high-level spatial features of data by alternately stacking multiple convolutional layers and pooling layers; Use the long short-term memory network to extract the time series features of the input features, and extract the high-level time series features of the data through the stacking and attention mechanism of multiple layers of long short-term memory networks; The spatial features extracted by the convolutional neural network and the temporal features extracted by the long short-term memory network are concatenated and input into the fully connected layer for further feature extraction and fusion. Classify or regress the fused features through the Softmax layer and output the prediction results; Based on the loss function, the back propagation algorithm is used to train the model end-to-end, optimize the model parameters, and obtain the trained data fusion model; The fused data at the semantic level is input into the trained data fusion model to generate fusion features.

6. The multi-source data information fusion method based on the Internet of Things protocol according to claim 5 is characterized in that: The trained data fusion model is: in, represents the activation value of the jth neuron in the Lth layer, represents the weighted input of the jth neuron in the Lth layer, represents the connection weight from the jth neuron to the ith neuron in the Lth layer, represents the error term of the j-th neuron in the L-th layer, Represents the activation value of the i-th neuron in the L-th layer.

7. The multi-source data information fusion method based on the Internet of Things protocol according to any one of claims 1 to 6, characterized in that: Before step 5, the method further includes the following steps: A decision model is constructed, wherein the decision model includes any one of logistic regression, support vector machine, decision tree, random forest and neural network.

8. A multi-source data information fusion system based on the Internet of Things protocol, characterized in that: include: A data acquisition module is used to obtain raw data from multiple data sources based on different IoT protocols, and preprocess the raw data to obtain preprocessed multi-source heterogeneous data, wherein the IoT protocols include at least NB-IoT protocol, LoRa protocol and ZigBee protocol; A mapping model building module is used to build a multi-source data mapping model based on the semantic ontology, map the pre-processed multi-source heterogeneous data to the corresponding nodes in the semantic ontology, and generate fused data at the semantic level; A fusion model construction module is used to construct a data fusion model based on deep learning, input the fusion data at the semantic level into the data fusion model, and generate fusion features through multi-level feature extraction and feature fusion; The multi-source data fusion module is used to input the fusion features into the decision model, and output the final result after the fusion of multi-source heterogeneous data in combination with the needs of the Internet of Things application.

9. An electronic device, characterized in that: It includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the multi-source data information fusion method based on the Internet of Things protocol as described in any one of claims 1-8.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the multi-source data information fusion method based on the Internet of Things protocol as described in any one of claims 1 to 8.

Images