A multi-level extraction method for IoT perception information and related equipment

Abstract

The present application relates to the field of spatiotemporal data mining technology, and provides a multi-level extraction method for IoT perception information and related equipment. The method of the present application includes: constructing an IoT perception data ontology model for describing the attributes of sensor data in a target IoT; performing data enhancement on IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and performing shallow mapping on the enhanced perception data according to the IoT perception data ontology model to obtain attribute triples corresponding to each enhanced sensor data; performing deep feature extraction based on all attribute triples to obtain spatiotemporal feature triples for each enhanced sensor data; performing spatiotemporal pattern mining on the enhanced perception data to obtain multiple spatiotemporal relationship triples; and constructing a knowledge graph of the target IoT based on all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples. The method of the present application can improve the extraction quality of IoT perception information.

Description

Technical Field

[0001] The present application relates to the field of spatiotemporal data mining technology, and in particular to a multi-level extraction method for IoT perception information and related equipment. Background Art

[0002] Knowledge graph is a structured semantic knowledge base that stores the relationship between entities in the form of a graph, making the organization and retrieval of information more intuitive and efficient. Knowledge graph plays an important role in data integration and analysis, information retrieval and recommendation, knowledge question answering and reasoning, and other scenarios.

[0003] IoT sensing data plays a vital role in building knowledge graphs because it provides real-time, dynamic entity information and relationship data for knowledge graphs, enabling knowledge graphs to reflect changes in the real world. For example, in the field of smart cities, traffic flow data collected by sensors can help knowledge graphs identify traffic patterns and congestion, thereby optimizing traffic management; in the agricultural field, soil moisture and climate data can be used to build crop growth models to help farmers make more scientific irrigation decisions; in the medical and health field, physiological data collected by wearable devices can be combined with patients' medical records to build personalized health management knowledge graphs to assist doctors in accurate diagnosis and treatment; in industrial manufacturing, the operating data of machinery and equipment can be used to monitor production processes and identify potential faults, thereby improving production efficiency and safety. The integration and analysis of these IoT sensing data not only enriches the content of knowledge graphs, but also enhances their application value in various fields.

[0004] With the rapid development of the Internet of Things, a large number of sensors have been deployed, generating massive, multi-source, and heterogeneous perception data. These data contain rich spatiotemporal feature information and are valuable data sources for dynamically building and continuously updating knowledge graphs. However, due to the uneven quality and complex and diverse formats of IoT data, data processing and information extraction have brought huge challenges. The non-standardization of data, the complexity of spatiotemporal features, and the difficulty in capturing spatiotemporal associations have all become factors that hinder the construction of knowledge graphs. In order to meet these challenges, it is urgent to establish an information extraction method system for the diversity of IoT data. By introducing data quality monitoring, format parsing, mapping specifications and other links, systematic structured analysis and conversion of multi-source heterogeneous data can effectively extract the complex spatiotemporal information contained therein to support the dynamic incremental construction of knowledge graphs. Only by solving the problem of data utilization can IoT perception data truly become a booster for the construction of knowledge graphs, realize real-time updating and dynamic evolution of graphs, and provide timely, accurate, and comprehensive knowledge services for various applications.

[0005] At present, the research work related to the extraction of information contained in IoT perception data mainly focuses on improving the efficiency of data utilization, extracting meaningful insights, and supporting the decision-making process. Researchers have explored the key tasks of extracting features from IoT data. The research involves traditional statistical techniques and machine learning-based feature extraction methods, and evaluates them based on their ability to capture relevant information, reduce dimensions, and enhance the performance of downstream IoT analysis tasks. Overall, there are many bottleneck problems in the current IoT perception data extraction technology, mainly including interoperability problems caused by the diversity of devices and different standards, the fragmentation of communication protocols that increases the complexity of interoperability, and data format and semantic differences that hinder effective communication and data exchange, resulting in poor quality of extraction results and insufficient information. It can be seen that there is currently a problem of low information quality in IoT perception information extraction. Summary of the invention

[0006] The present application provides a multi-level extraction method and related equipment for IoT perception information, which can solve the problem of low information quality in IoT perception information extraction.

[0007] In a first aspect, an embodiment of the present application provides a multi-level extraction method for IoT perception information, the multi-level extraction method comprising:

[0008] Acquire IoT perception data of a target IoT, and construct an IoT perception data ontology model for describing the attributes of sensor data in the target IoT; the IoT perception data includes multiple sensor data and related information of each sensor data;

[0009] Performing data enhancement on IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and shallowly mapping the enhanced perception data according to the IoT perception data ontology model to obtain attribute triples corresponding to each enhanced sensor data; the attribute triples include enhanced sensor data, sensors corresponding to the enhanced sensor data, and attributes of the enhanced sensor data;

[0010] Based on all attribute triplets, deep feature extraction is performed to obtain the spatiotemporal feature triplets of each enhanced sensor data; the spatiotemporal feature triplets include the spatial information of the enhanced sensor data, the change information of the enhanced sensor data, and the time information of the enhanced sensor data;

[0011] The enhanced perception data is mined for spatiotemporal patterns to obtain multiple spatiotemporal relationship triplets; the spatiotemporal relationship triplets are used to describe the spatiotemporal associations of all enhanced sensor data;

[0012] The knowledge graph of the target IoT is constructed based on all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples.

[0013] Optionally, the IoT perception data ontology model includes a concept layer and an attribute layer;

[0014] The conceptual layer includes multiple entities related to IoT sensing data, and the multiple entities include sensor data;

[0015] The attribute layer includes the attributes of each entity.

[0016] Optionally, data enhancement is performed on the IoT sensing data to obtain enhanced sensing data including multiple enhanced sensor data, including:

[0017] Identify missing data for IoT sensing data and identify multiple missing data from all sensor data;

[0018] For each missing data, if the missing data ratio of the missing data is greater than the missing ratio threshold, the missing data will be discarded; if the missing data ratio of the missing data is less than or equal to the missing ratio threshold, the missing data will be filled to obtain complete data;

[0019] Perform anomaly correction on each complete data to obtain enhanced sensor data corresponding to each complete data;

[0020] Performing anomaly correction on each other sensor data except the missing data in all sensor data to obtain enhanced sensor data corresponding to each other sensor data;

[0021] All enhanced sensor data are integrated to obtain enhanced perception data.

[0022] Optionally, shallow mapping is performed on the enhanced perception data according to the IoT perception data ontology model to obtain attribute triples corresponding to each enhanced sensor data, including:

[0023] For each enhanced sensor data in the enhanced perception data, according to the attribute layer of the IoT perception data ontology model, the attributes of the enhanced sensor data are identified from the relevant information corresponding to the enhanced sensor data, and the enhanced sensor data, the sensor corresponding to the enhanced sensor data, and the attributes of the enhanced sensor data are integrated to obtain an attribute triplet.

[0024] Optionally, deep feature extraction is performed based on all attribute triplets to obtain a spatiotemporal feature triplet of each enhanced sensor data, including:

[0025] For each enhanced sensor data, perform the following steps:

[0026] Perform deep feature extraction on the enhanced sensor data to obtain the change information of the enhanced sensor data;

[0027] Obtaining spatial information of a sensor corresponding to the enhanced sensor data in an attribute triplet of the enhanced sensor data;

[0028] Obtaining time information of the enhanced sensor data from relevant information corresponding to the enhanced sensor data;

[0029] The change information, spatial information and temporal information of the enhanced sensor data are integrated to obtain the spatiotemporal feature triplet.

[0030] Optionally, the enhanced perception data is mined for spatiotemporal patterns to obtain multiple spatiotemporal relationship triplets, including:

[0031] Construct a spatiotemporal pattern library including multiple spatiotemporal association patterns; the spatiotemporal association pattern is used to describe the association relationship between two spatial locations in time and space;

[0032] For each two enhanced sensor data in the enhanced perception data, perform the following steps:

[0033] According to the spatial information corresponding to the two enhanced sensor data, a target spatiotemporal correlation pattern between the spatial information corresponding to the two enhanced sensor data is determined from the spatiotemporal pattern library;

[0034] The spatial information and target spatiotemporal association patterns corresponding to the two enhanced sensor data are converted into spatiotemporal relationship triples.

[0035] Optionally, the IoT perception data ontology model also includes a relationship layer and an instance layer;

[0036] The relational layer includes various entity relationships between all entities in the conceptual layer;

[0037] The instance layer is used to describe the specific instance of each entity in the concept layer.

[0038] In a second aspect, an embodiment of the present application provides a multi-level extraction device for IoT perception information, including:

[0039] An acquisition module is used to acquire IoT sensing data of a target IoT and construct an IoT sensing data ontology model for describing the attributes of sensor data in the target IoT; the IoT sensing data includes multiple sensor data and related information of each sensor data;

[0040] A shallow mapping module performs data enhancement on IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and performs shallow mapping on the enhanced perception data according to the IoT perception data ontology model to obtain an attribute triple corresponding to each enhanced sensor data; the attribute triple includes enhanced sensor data, a sensor corresponding to the enhanced sensor data, and an attribute of the enhanced sensor data;

[0041] The deep feature extraction module performs deep feature extraction based on all attribute triplets to obtain the spatiotemporal feature triplets of each enhanced sensor data; the spatiotemporal feature triplets include the spatial information of the enhanced sensor data, the change information of the enhanced sensor data, and the time information of the enhanced sensor data;

[0042] The spatiotemporal pattern mining module mines the spatiotemporal patterns of the enhanced perception data to obtain multiple spatiotemporal relationship triplets; the spatiotemporal relationship triplets are used to describe the spatiotemporal association relationship of all enhanced sensor data;

[0043] The building module builds the knowledge graph of the target IoT based on all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples.

[0044] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned multi-level extraction method of IoT perception information when executing the above-mentioned computer program.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned multi-level extraction method of IoT perception information.

[0046] The above solution of the present application has the following beneficial effects:

[0047] In an embodiment of the present application, the IoT perception data of the target IoT is obtained, and an IoT perception data ontology model for describing the attributes of sensor data in the target IoT is constructed, and then the IoT perception data is data enhanced to obtain enhanced perception data including multiple enhanced sensor data, and the enhanced perception data is shallowly mapped according to the IoT perception data ontology model to obtain the attribute triple corresponding to each enhanced sensor data, and then deep feature extraction is performed based on all attribute triples to obtain the spatiotemporal feature triple of each enhanced sensor data, and then the enhanced perception data is spatiotemporal pattern mining is performed to obtain multiple spatiotemporal relationship triples, and finally the knowledge graph of the target IoT is constructed based on all attribute triples, all spatiotemporal feature triples and all spatiotemporal relationship triples. Among them, data enhancement of IoT perception data can improve the quality and accuracy of IoT perception data, and shallow mapping, deep feature extraction and spatiotemporal pattern mining based on high-quality data can improve the information quality obtained by information extraction, and at the same time, three triples containing different information are obtained, which effectively improves the information richness of information extraction.

[0048] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A flowchart of a multi-level extraction method of IoT sensing information provided by an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a knowledge graph provided for an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the structure of a multi-level extraction device for IoT sensing information provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0060] In response to the problem of low information quality in the existing extraction of IoT perception information, an embodiment of the present application provides a multi-level extraction method for IoT perception information. The multi-level extraction method obtains the IoT perception data of the target IoT and constructs an IoT perception data ontology model for describing the attributes of sensor data in the target IoT. The IoT perception data is then enhanced to obtain enhanced perception data including multiple enhanced sensor data. The enhanced perception data is shallowly mapped according to the IoT perception data ontology model to obtain attribute triples corresponding to each enhanced sensor data. Deep feature extraction is then performed based on all attribute triples to obtain spatiotemporal feature triples of each enhanced sensor data. Spatiotemporal pattern mining is then performed on the enhanced perception data to obtain multiple spatiotemporal relationship triples. Finally, a knowledge graph of the target IoT is constructed based on all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples. Among them, data enhancement of IoT perception data can improve the quality and accuracy of IoT perception data. Three types of processing, shallow mapping, deep feature extraction, and spatiotemporal pattern mining, based on high-quality data can improve the quality of information extracted. At the same time, three triplets containing different information are obtained, which effectively improves the information richness of information extraction.

[0061] Next, the multi-level extraction method of IoT perception information provided in this application is exemplified.

[0062] like Figure 1 As shown, the multi-level extraction method of IoT perception information provided by this application includes the following steps:

[0063] Step 11, obtaining the IoT perception data of the target IoT, and constructing an IoT perception data ontology model for describing the attributes of the sensor data in the target IoT.

[0064] The above-mentioned target Internet of Things may be an Internet of Things related to landslide geological disasters in a certain area, wherein the sensors are used to detect environmental indicators related to landslide geological disasters, such as rainfall, humidity, etc. The above-mentioned Internet of Things perception data includes multiple sensor data and related information of each sensor data (such as the corresponding sensor, the geographical location of the sensor, the time of obtaining the sensor data, etc.).

[0065] In some embodiments of the present application, the sensor data is the value of a certain indicator, including the values ​​of the indicator at multiple timestamps. For example, if the indicator is temperature, the sensor data includes temperature values ​​at multiple historical timestamps.

[0066] It should be noted that the construction process of the IoT perception data ontology model includes 7 steps:

[0067] Requirements analysis: Determine the sensor application areas and the entities, attributes, and relationships that the ontology model needs to cover.

[0068] Concept definition: Based on demand analysis, define the entity categories in the concept layer.

[0069] Attribute definition: Define detailed attributes for each entity.

[0070] Relationship definition: Identify the relationship between entities and define the relationship type.

[0071] Instance creation: Create a specific instance of the entity based on the actual application scenario.

[0072] Model verification: Verify the validity of the ontology model through actual data and application scenarios.

[0073] Model iteration: Adjust and optimize the model based on the verification results.

[0074] The IoT perception data ontology model includes a concept layer and an attribute layer; the concept layer includes multiple entities related to IoT perception data (multiple entities include sensors, data, and events); the attribute layer includes the attributes of each entity (the attributes of sensors include type, location, and accuracy; the attributes of sensor data include value, timestamp, and unit; the attributes of events include description and severity). It also includes a relationship layer and an instance layer; the relationship layer includes multiple entity relationships between all entities in the concept layer (such as the entity relationship between sensors and sensor data is perception, the entity relationship between sensor data and events is triggering, the entity relationship between sensor data or events is association, and also includes "before" used to describe the order relationship between timestamps); the instance layer is used to describe the specific instance of each entity in the concept layer, such as the instance of the sensor: temperature sensor, the instance of sensor data: "At 14:00 on November 25, 2024, the temperature is -2℃", and the event instance: a landslide event in a certain area on January 22.

[0075] Events are used to describe specific situations triggered by data changes, such as "temperature exceeds threshold". The units in the attributes of the above sensor data are measurement units, such as degrees Celsius, percentage, etc. The description in the attributes of the event refers to the detailed description of the event, and the severity refers to the urgency of the event.

[0076] Exemplarily, the target Internet of Things is an Internet of Things related to landslide geological disasters in a certain area. Sensors are set at multiple geographical locations in the area to detect multiple environmental indicators (such as rainfall, etc.). Taking the rainfall sensor as an example, the sensor data includes the rainfall detected by the rainfall sensor at multiple times within a period of time. The relevant information includes the geographical location of the rainfall sensor, the detection time of each rainfall amount, the identification of the rainfall sensor (such as the number), etc.

[0077] It should be noted that after obtaining the IoT perception data, in order to solve the problem of incompatibility between data formats and field structures between different devices and systems, this application needs to perform data mapping and conversion on the IoT perception data to achieve data association processing of different file formats and different data table structures to ensure the consistency of data formats. Specifically:

[0078] First, define a unified data model and semantic description framework to achieve interoperability of data and knowledge meanings between different devices and applications, including determining common data structures, field names and data types, and the associations between them, and create these pre-defined information as dictionaries. Then, use the IoT perception data ontology to define the associations between IoT devices, services, and entities, improve data comprehensibility, and extract valuable insights from the data (data content worth extracting from the data, such as the monitoring data attribute values ​​that are constantly generated in the data table. The size parameters of the monitoring equipment are secondary information and do not need to be considered, not insights. The extracted insights are determined based on the ontology and need to be observed and judged before the IoT perception data is extracted.) for reasoning and predictive analysis. Then, based on the conversion program set by the data reading preset, the conversion program integrates the valuable insights initially determined according to the dictionary to achieve the conversion of data of different formats and structures into a unified format.

[0079] Step 12: perform data enhancement on the IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and perform shallow mapping on the enhanced perception data according to the IoT perception data ontology model to obtain attribute triples corresponding to each enhanced sensor data.

[0080] The attribute triplet includes enhanced sensor data, a sensor corresponding to the enhanced sensor data, and an attribute of the enhanced sensor data.

[0081] In some embodiments of the present application, the steps of performing data enhancement on IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and shallowly mapping the enhanced perception data according to the IoT perception data ontology model to obtain the attribute triplet corresponding to each enhanced sensor data include:

[0082] The first step is to identify missing data in IoT sensing data and identify multiple missing data from all sensor data.

[0083] Specifically, if there are missing parts in the sensor data and the related information of the sensor data, the sensor data is regarded as missing data.

[0084] In the second step, for each missing data, if the missing data ratio of the missing data is greater than the missing ratio threshold, the missing data will be discarded; if the missing data ratio of the missing data is less than or equal to the missing ratio threshold, the missing data will be filled to obtain complete data.

[0085] Specifically, the complete data can be obtained by calculating the mean or median of the existing data in the missing data and filling the missing data with the value.

[0086] It should be noted that the missing ratio threshold is set according to the actual conditions of the area where the sensor is located. If the missing ratio of the missing data is greater than the missing ratio threshold, it can be reasonably assumed that the sensor corresponding to the missing data is damaged, interfered with, or other abnormal conditions, and the missing data is discarded.

[0087] Exemplarily, when the missing data ratio of missing data is less than or equal to the missing data ratio threshold, the missing data can be discarded and supplemented in a more detailed manner. For example, when the format of the IoT sensing data is a table, the data rows with missing conditions can be analyzed. If the missing data in the data row exceeds a preset ratio (such as 35%), the data row will be deleted. If the missing data in the data row does not exceed the preset ratio, the missing data in the data row can be filled and supplemented by calculating the average or median.

[0088] In the third step, anomaly correction is performed on each complete data to obtain the enhanced sensor data corresponding to each complete data.

[0089] In the fourth step, anomaly correction is performed on each other sensor data except the missing data in all sensor data to obtain enhanced sensor data corresponding to each other sensor data.

[0090] For example, for complete data or other sensor data, erroneous or inaccurate data is identified and corrected mainly through data verification and outlier detection. By verifying the data, it is checked whether the data conforms to the expected format and range (for example, the data of the temperature sensor should not be lower than absolute zero), and whether the data is abnormal. In addition, prior knowledge can be used for statistical constraints, and statistical methods (such as box plots, Z scores, etc.) can be used to identify and process outliers. For inaccurate data, corrections are made based on business logic and statistical analysis results.

[0091] The fifth step is to integrate all enhanced sensor data to obtain enhanced perception data.

[0092] In the sixth step, the enhanced perception data is shallowly mapped according to the IoT perception data ontology model to obtain the attribute triples corresponding to each enhanced sensor data.

[0093] Specifically, for each enhanced sensor data in the enhanced perception data, according to the attribute layer of the IoT perception data ontology model, the attributes of the enhanced sensor data are identified from the relevant information corresponding to the enhanced sensor data, and the enhanced sensor data, the sensor corresponding to the enhanced sensor data, and the attributes of the enhanced sensor data are integrated to obtain an attribute triplet.

[0094] Exemplarily, the attribute defined in the attribute layer of the IoT perception data ontology model is identified from the relevant information, and the attribute is used as the attribute of the enhanced sensor data. For example, the relevant information of the enhanced sensor data contains the timestamp when the data is detected, which corresponds to the attribute of the data in the attribute layer of the IoT perception data ontology model - timestamp. Then the timestamp is used as the attribute of the enhanced sensor data, and the attribute triplet (sensor identifier, enhanced sensor data value, timestamp) is finally obtained.

[0095] It should be noted that if the contents of the enhanced sensor data and related information are complex, such as the related information describes multiple attributes of the data (such as describing a timestamp and a unit at the same time), and the enhanced sensor data includes data with multiple timestamps, then the number of attribute triplets of the generated enhanced sensor data is multiple.

[0096] Step 13: Perform deep feature extraction based on all attribute triplets to obtain spatiotemporal feature triplets of each enhanced sensor data.

[0097] The above-mentioned spatiotemporal feature triplet includes the spatial information of the enhanced sensor data, the change information of the enhanced sensor data, and the time information of the enhanced sensor data.

[0098] In some embodiments of the present application, the step of performing deep feature extraction based on all attribute triplets to obtain a spatiotemporal feature triplet of each enhanced sensor data includes:

[0099] For each enhanced sensor data, perform the following steps:

[0100] In the first step, deep feature extraction is performed on the enhanced sensor data to obtain the change information of the enhanced sensor data.

[0101] Specifically, deep features to be calculated and extracted in IoT perception data (such as the mean, standard deviation, trend slope, etc. of sensor readings) are constructed from the perspectives of physical mechanism and statistical analysis, and then the enhanced sensor data is calculated to obtain the above-mentioned deep features, and the deep features are used as the change information of the enhanced sensor data.

[0102] Exemplarily, the enhanced sensor data includes data with 5 timestamps, and the 5 data are averaged to obtain the deep feature as the average.

[0103] The second step is to obtain the spatial information of the sensor corresponding to the enhanced sensor data in the attribute triplet of the enhanced sensor data.

[0104] Exemplarily, the geographic location of the sensor may be acquired from relevant information corresponding to the enhanced sensor data, and the geographic location may be used as the spatial information of the sensor.

[0105] The third step is to obtain the time information of the enhanced sensor data from the relevant information corresponding to the enhanced sensor data.

[0106] Exemplarily, the relevant information includes timestamps corresponding to all data in the enhanced sensor data, and a time window between the earliest timestamp and the latest timestamp can be used as time information.

[0107] The fourth step is to integrate the change information, spatial information, and temporal information of the enhanced sensor data to obtain the spatiotemporal feature triplet.

[0108] For example, the deep feature "12 mm" is the total rainfall within one hour at site 1 where the first rainfall sensor is located. After alignment to the site entity, a spatiotemporal feature triplet (site 1, hourly rainfall, 12 mm) is generated.

[0109] It is worth mentioning that extracting spatiotemporal feature triplets can extract deeper feature information from the data, such as trend type, cycle type, anomaly type, etc. These features are crucial to understanding the complex phenomena behind the data.

[0110] Step 14: perform spatiotemporal pattern mining on the enhanced perception data to obtain multiple spatiotemporal relationship triplets.

[0111] The above-mentioned spatiotemporal relationship triples are used to describe the spatiotemporal association relationship of all enhanced sensor data.

[0112] In some embodiments of the present application, the step of performing spatiotemporal pattern mining on the enhanced perception data to obtain a plurality of spatiotemporal relationship triplets includes:

[0113] The first step is to build a spatiotemporal pattern library including various spatiotemporal correlation patterns.

[0114] The above-mentioned spatiotemporal correlation pattern is used to describe the temporal and spatial correlation relationship (such as similarity, heterogeneity, correlation, etc.) between the geographical locations described by two spatial information.

[0115] Exemplarily, the spatiotemporal association pattern includes a spatiotemporal correlation pattern, a spatiotemporal heterogeneity pattern, a spatiotemporal similarity pattern, etc. For example, a spatiotemporal similarity pattern expresses that data such as environmental indicator values ​​of two geographical locations have similar relationships in time and space.

[0116] In the second step, for each two enhanced sensor data in the enhanced perception data, perform the following steps:

[0117] According to the spatial information corresponding to the two enhanced sensor data, a target spatiotemporal correlation pattern between the spatial information corresponding to the two enhanced sensor data is determined from the spatiotemporal pattern library.

[0118] The spatial information and target spatiotemporal association patterns corresponding to the two enhanced sensor data are converted into spatiotemporal relationship triples.

[0119] For example, data mining algorithms such as sequence pattern mining and frequent item set mining can be used to determine the target spatiotemporal correlation pattern between the two corresponding spatial information from the spatiotemporal pattern library based on the two enhanced sensor data and the corresponding related information. It helps to analyze the correlation between different spatial locations in time and space, and identify the sequence, symbiotic relationship or causal chain between geographic spatiotemporal events. Finally, the mined spatiotemporal pattern is converted into a spatiotemporal relationship triple, which contains not only basic entities and attributes, but also complex spatiotemporal relationships and rules, such as (Site A spatial autocorrelation, before, Site B spatial autocorrelation).

[0120] Step 15: construct a knowledge graph of the target Internet of Things based on all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples.

[0121] Exemplarily, the head entity and the tail entity in all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples are taken as nodes in the knowledge graph, and the relationship in the triple is the attribute of the edge between two nodes.

[0122] It should be noted that after constructing the knowledge graph, the correlation between entities in the target IoT can be analyzed based on the knowledge graph, and management and event warning of the area covered by the target IoT can be performed based on the correlation. For example, for the knowledge graph of landslide geological disasters in a certain area, a series of indicators related to the occurrence of landslide events can be analyzed based on the knowledge graph, and the relationship between the changes in these indicators and the landslide events can be analyzed. By monitoring the changes in these indicators, the occurrence of landslide events can be predicted and warned, and countermeasures can be extracted to reduce disaster losses.

[0123] It is worth mentioning that data enhancement of IoT perception data can improve the quality and accuracy of IoT perception data. Three types of processing, shallow mapping, deep feature extraction, and spatiotemporal pattern mining, based on high-quality data can improve the quality of information extracted. At the same time, three triplets containing different information are obtained, which effectively improves the information richness of information extraction.

[0124] In addition, the advantages of the method of the present application are: (1) The method can take into account the differences in data formats and data table structures caused by different devices and systems, thereby realizing the automatic reading of data from multiple devices or systems. (2) The method can fully take into account the errors, missing or inaccurate information in the sensor data, and effectively improve the quality of the imported sensor data. (3) The method can take into account the multi-level characteristics of the information contained in the IoT sensing data, and while obtaining the original attribute values, it can effectively extract the deep features and spatiotemporal patterns contained in the data. It has important application prospects in business fields such as disaster warning, traffic supervision, and weather forecasting.

[0125] The method of the present application is illustrated below with reference to a specific example.

[0126] This paper conducts a case study on the task of extracting spatiotemporal information from multi-type IoT sensing (displacement, rainfall) data related to landslide geological disasters in a certain province. Among them, a total of 5644 displacement sensors were deployed, with a data range of 30 days, and 3384 rainfall sensors were deployed, with a data range of 3 days.

[0127] The concept layer, attribute layer, relationship layer and instance layer of the IoT perception data ontology model of geological disasters are constructed by adopting steps such as demand analysis, concept definition, attribute definition, relationship definition, instance creation, model verification and model iteration. This example takes the landslide geological disaster in a certain province as the research object, and conducts an experiment on extracting spatiotemporal information contained in IoT perception data for two types of sensors, rainfall and displacement. A total of 23 entities and 32 relationship categories are defined. Then, by constructing data feature mapping, the extraction content is formed as shown in Table 1.

[0128] Table 1

[0129]

[0130] The IoT perception data is enhanced by using technical means such as missing data identification and processing, and erroneous or inaccurate information processing.

[0131] After shallow attribute mapping, deep feature extraction, and spatiotemporal pattern mining, a total of 31 disaster event and sensor data features were extracted. Disaster events were associated with sensor identifiers through township-level administrative divisions, and finally a knowledge graph centered on disaster events was formed. To prove the effectiveness of this method, taking displacement sensors as an example, this study selected four typical features: daily maximum displacement, weekly maximum displacement, weekly average displacement, and weekly maximum displacement change, for visualization analysis of the extraction results. A part of the knowledge graph constructed based on the extracted triples is shown in the figure below. Figure 2 The entities in the figure include station 1, infrasound, rain gauge, thermometer, speedometer, cumulative displacement, acceleration, maximum hourly displacement, speed, 280mm, 2mm / h 2 , 20mm, 10mm / h, Figure 2 The middle circle represents the node corresponding to the entity, and the solid line represents the existence of correlation between entities.

[0132] It can be seen that by adopting the multi-level extraction method of spatiotemporal information contained in IoT perception data proposed in this application, a total of 325,874 triples were extracted, which can provide important spatiotemporal entity and spatiotemporal relationship support for the construction of spatiotemporal knowledge graphs.

[0133] The following is an exemplary description of the multi-level extraction device of IoT perception information provided in this application.

[0134] like Figure 3 As shown, the embodiment of the present application provides a multi-level extraction device for IoT perception information, and the multi-level extraction device 300 for IoT perception information includes:

[0135] The acquisition module 301 acquires the IoT sensing data of the target IoT and constructs an IoT sensing data ontology model for describing the attributes of the sensor data in the target IoT; the IoT sensing data includes multiple sensor data and related information of each sensor data;

[0136] The shallow mapping module 302 performs data enhancement on the IoT sensing data to obtain enhanced sensing data including multiple enhanced sensor data, and performs shallow mapping on the enhanced sensing data according to the IoT sensing data ontology model to obtain an attribute triple corresponding to each enhanced sensor data; the attribute triple includes the enhanced sensor data, the sensor corresponding to the enhanced sensor data, and the attribute of the enhanced sensor data;

[0137] The deep feature extraction module 303 performs deep feature extraction based on all attribute triples to obtain a spatiotemporal feature triple for each enhanced sensor data; the spatiotemporal feature triple includes spatial information of the enhanced sensor data, change information of the enhanced sensor data, and time information of the enhanced sensor data;

[0138] The spatiotemporal pattern mining module 304 performs spatiotemporal pattern mining on the enhanced sensing data to obtain a plurality of spatiotemporal relationship triplets; the spatiotemporal relationship triplets are used to describe the spatiotemporal association relationship of all enhanced sensor data;

[0139] The construction module 305 constructs the knowledge graph of the target Internet of Things based on all attribute triples, all spatiotemporal feature triples and all spatiotemporal relationship triples.

[0140] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0141] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0142] like Figure 4 As shown, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 4 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0143] Specifically, when the processor D100 executes the computer program D102, it acquires the IoT perception data of the target IoT and constructs an IoT perception data ontology model for describing the attributes of the sensor data in the target IoT, then performs data enhancement on the IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and performs shallow mapping on the enhanced perception data according to the IoT perception data ontology model to obtain the attribute triple corresponding to each enhanced sensor data, and then performs deep feature extraction based on all attribute triples to obtain the spatiotemporal feature triple of each enhanced sensor data, and then performs spatiotemporal pattern mining on the enhanced perception data to obtain multiple spatiotemporal relationship triples, and finally constructs the knowledge graph of the target IoT based on all attribute triples, all spatiotemporal feature triples and all spatiotemporal relationship triples. Among them, data enhancement on IoT perception data can improve the quality and accuracy of IoT perception data, and shallow mapping, deep feature extraction and spatiotemporal pattern mining based on high-quality data can improve the information quality obtained by information extraction, and at the same time, three triples containing different information are obtained, which effectively improves the information richness of information extraction.

[0144] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0145] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0146] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0147] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0148] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium that can carry computer program code to the multi-level extraction method device / terminal device of the Internet of Things perception information. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0149] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0150] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0151] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multi-level extraction method for IoT perception information, characterized in that: include: Acquire IoT perception data of a target IoT, and construct an IoT perception data ontology model for describing the attributes of sensor data in the target IoT; the IoT perception data includes a plurality of sensor data and related information of each sensor data; Performing data enhancement on the IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and shallowly mapping the enhanced perception data according to the IoT perception data ontology model to obtain an attribute triplet corresponding to each enhanced sensor data; the attribute triplet includes enhanced sensor data, a sensor corresponding to the enhanced sensor data, and an attribute of the enhanced sensor data; Perform deep feature extraction based on all attribute triplets to obtain a spatiotemporal feature triplet of each enhanced sensor data; the spatiotemporal feature triplet includes spatial information of the enhanced sensor data, change information of the enhanced sensor data, and time information of the enhanced sensor data; Perform spatiotemporal pattern mining on enhanced perception data to obtain multiple spatiotemporal relationship triplets; The spatiotemporal relationship triplet is used to describe the spatiotemporal association relationship of all enhanced sensor data; A knowledge graph of the target Internet of Things is constructed based on all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples.

2. The multi-level extraction method according to claim 1, characterized in that: The IoT perception data ontology model includes a concept layer and an attribute layer; The concept layer includes a plurality of entities related to IoT sensing data, and the plurality of entities include sensor data; The attribute layer includes the attributes of each of the entities.

3. The multi-level extraction method according to claim 1, characterized in that: The step of performing data enhancement on the IoT sensing data to obtain enhanced sensing data including a plurality of enhanced sensor data includes: Performing missing data identification on the IoT sensing data, and identifying a plurality of missing data from all sensor data; For each missing data, if the missing data ratio of the missing data is greater than the missing ratio threshold, the missing data is discarded; if the missing data ratio of the missing data is less than or equal to the missing ratio threshold, the missing data is filled with data to obtain complete data; Performing anomaly correction on each complete data to obtain enhanced sensor data corresponding to each complete data; Performing anomaly correction on each other sensor data except the missing data in all the sensor data to obtain enhanced sensor data corresponding to each other sensor data; All enhanced sensor data are integrated to obtain enhanced perception data.

4. The multi-level extraction method according to claim 2, characterized in that: The shallow mapping of the enhanced perception data according to the IoT perception data ontology model to obtain the attribute triplet corresponding to each enhanced sensor data includes: For each enhanced sensor data in the enhanced perception data, according to the attribute layer of the IoT perception data ontology model, the attributes of the enhanced sensor data are identified from the relevant information corresponding to the enhanced sensor data, and the enhanced sensor data, the sensor corresponding to the enhanced sensor data, and the attributes of the enhanced sensor data are integrated to obtain an attribute triplet.

5. The multi-level extraction method according to claim 4, characterized in that: The deep feature extraction is performed based on all attribute triples to obtain a spatiotemporal feature triple of each enhanced sensor data, including: For each enhanced sensor data, perform the following steps: Performing deep feature extraction on the enhanced sensor data to obtain change information of the enhanced sensor data; Acquire spatial information of a sensor corresponding to the enhanced sensor data in the attribute triplet of the enhanced sensor data; Acquire time information of the enhanced sensor data from relevant information corresponding to the enhanced sensor data; The change information, spatial information, and time information of the enhanced sensor data are integrated to obtain a spatiotemporal feature triplet.

6. The multi-level extraction method according to claim 5, characterized in that: The spatial-temporal pattern mining of the enhanced perception data is performed to obtain multiple spatial-temporal relationship triplets, including: Constructing a spatiotemporal pattern library including a plurality of spatiotemporal association patterns; the spatiotemporal association patterns are used to describe the association relationship between two spatial positions in time and space; For each two enhanced sensor data in the enhanced perception data, perform the following steps: According to the spatial information corresponding to the two enhanced sensor data, determining a target spatiotemporal correlation pattern between the spatial information corresponding to the two enhanced sensor data from the spatiotemporal pattern library; The spatial information corresponding to the two enhanced sensor data and the target spatiotemporal association pattern are converted into a spatiotemporal relationship triplet.

7. The multi-level extraction method according to claim 2, characterized in that: The IoT perception data ontology model also includes a relationship layer and an instance layer; The relationship layer includes multiple entity relationships between all entities in the concept layer; The instance layer is used to describe a specific instance of each entity in the concept layer.

8. A multi-level extraction device for IoT perception information, characterized in that: include: An acquisition module is used to acquire IoT sensing data of a target IoT and construct an IoT sensing data ontology model for describing the attributes of sensor data in the target IoT; the IoT sensing data includes multiple sensor data and related information of each sensor data; A shallow mapping module performs data enhancement on the IoT perception data to obtain enhanced perception data including multiple enhanced sensor data, and performs shallow mapping on the enhanced perception data according to the IoT perception data ontology model to obtain an attribute triplet corresponding to each enhanced sensor data; the attribute triplet includes enhanced sensor data, a sensor corresponding to the enhanced sensor data, and an attribute of the enhanced sensor data; A deep feature extraction module performs deep feature extraction based on all attribute triplets to obtain a spatiotemporal feature triplet of each enhanced sensor data; the spatiotemporal feature triplet includes spatial information of the enhanced sensor data, change information of the enhanced sensor data, and time information of the enhanced sensor data; The spatiotemporal pattern mining module performs spatiotemporal pattern mining on the enhanced perception data to obtain multiple spatiotemporal relationship triplets; The spatiotemporal relationship triplet is used to describe the spatiotemporal association relationship of all enhanced sensor data; A construction module is used to construct a knowledge graph of the target Internet of Things based on all attribute triples, all spatiotemporal feature triples, and all spatiotemporal relationship triples.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the multi-level extraction method of Internet of Things perception information as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multi-level extraction method of Internet of Things perception information as described in any one of claims 1 to 7 is implemented.

Images