Internet of Things cluster data analysis system based on big data and artificial intelligence

Abstract

The invention discloses an Internet of Things cluster data analysis system based on big data and artificial intelligence, relates to the technical field of artificial intelligence, and solves the problems that equipment in an Internet of Things cluster is numerous and jumbled, and data streams of all Internet of Things equipment cannot be connected and collected, so that the data streams cannot be collected. And the data security of all Internet of Things devices in the data analysis process cannot be ensured. The Internet of Things cluster data analysis system comprises a data acquisition module, a data processing module, a data analysis module, a data query module and a data security module, the method comprises the following steps: collecting different types of information data in Internet of Things service through a data collection module according to a preset information collection rule, obtaining a cluster information data table, analyzing the cluster information data table through an information reasoning model according to information data characteristics, obtaining an abnormal data set, and constructing a reasoning link graph; performing real-time monitoring and protection on each function module of the Internet of Things cluster data analysis system through a mimicry defense model of the data security module; the connection stability of different types of Internet of Things devices is improved, and the data stream security of the Internet of Things devices is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an Internet of Things cluster data analysis system based on big data and artificial intelligence. Background Art

[0002] With the continuous development and application of IoT technology, more and more devices and sensors are connected to the Internet, transmitting a large amount of data to data centers for storage and processing. These data include not only the operation data of the equipment, the environmental data collected by the sensors, but also the user's behavior data and business data. How to effectively use this data, extract valuable information, and help enterprises or institutions make scientific decisions has become a new challenge that the IoT industry needs to meet.

[0003] The IoT cluster data analysis system based on big data and artificial intelligence has emerged. By collecting and analyzing data and using big data and artificial intelligence algorithms to analyze, predict and make decisions on the massive data generated between IoT devices and sensors, the system can help companies and institutions better understand data, gain insights into consumption and market trends, and optimize important decisions such as operations and production.

[0004] In the IoT cluster, sensors continuously collect environmental data, such as temperature, humidity, vibration, etc. Abnormal data may indicate potential security threats, such as fire, intrusion, etc. By constructing an inference link graph to analyze the relationship between abnormal data points and other data points, the threat source can be quickly located and its impact range can be assessed;

[0005] In the Industrial Internet of Things, the operating data of machines is monitored in real time, and abnormal data may indicate equipment failure. By constructing an inference link graph, the abnormal data can be compared with the historical failure modes of the equipment to predict potential failure points, thereby performing timely maintenance.

[0006] In the medical Internet of Things, patients' vital signs data are continuously monitored, and any abnormalities may indicate changes in health status. By constructing a reasoning link graph, it helps doctors analyze the relationship between abnormal data and diseases, providing support for diagnosis and treatment.

[0007] In the logistics IoT, the status of goods during transportation is tracked in real time, and abnormal data may indicate a supply chain disruption. By constructing an inference link graph, the association between abnormal events and other links in the supply chain can be revealed, helping managers respond quickly and optimize processes.

[0008] Patent No. CN2024102484173 discloses an enterprise big data analysis system based on artificial intelligence. The system includes a data acquisition module, a data cleaning module, a data analysis module, a prediction module and an early warning module. The data acquisition module is used to collect various types of data inside and outside the enterprise, including sales data, market data, and customer data. The data cleaning module is used to clean and organize the collected data to ensure the accuracy and integrity of the data. The data analysis module is used to analyze the cleaned data and discover the associations and patterns between the data. The above invention can analyze and predict big data more intelligently by introducing artificial intelligence technology, providing enterprises with more accurate and timely decision-making support. At the same time, the introduction of the early warning module can help enterprises detect data anomalies in a timely manner, avoid possible risks and losses, and improve the operational efficiency and competitiveness of enterprises.

[0009] Patent No. CN2023113527163 discloses a business big data analysis system based on artificial intelligence, which relates to the field of business big data analysis technology. The object data to be analyzed is collected through a data collection terminal, and is temporarily stored after being obtained according to a set capacity. At the same time, the data collection terminal is also used to encrypt the collected object data of the set capacity. After encryption, subsequent data will be used for big data analysis. During the analysis, the encrypted content will be verified to ensure that the corresponding analysis data has not been processed or added after collection, thereby avoiding interference from some data in the big data to a certain extent, making the analysis results more referenceable. The above application is simple, effective and easy to use.

[0010] However, the above patents have great defects in data collection and analysis. Due to the large number of devices in the IoT cluster, the existing data analysis system cannot guarantee that the data streams of all IoT devices can be connected and collected when collecting and processing massive data, and cannot guarantee the data security of all IoT devices during the data analysis process. Summary of the invention

[0011] The purpose of the present invention is to provide an Internet of Things cluster data analysis system based on big data and artificial intelligence, which can collect different types of information data in Internet of Things services according to information collection rules through artificial intelligence technology, obtain cluster information data tables, use information reasoning models to analyze cluster information data tables according to information data characteristics, obtain abnormal data sets, and construct reasoning link graphs; use mimetic defense models to perform real-time monitoring and protection on various functional modules of the Internet of Things cluster data analysis system; so as to improve the connection stability of different types of Internet of Things devices and ensure the data flow security of Internet of Things devices.

[0012] The present invention utilizes the following technical solutions:

[0013] An Internet of Things cluster data analysis system based on big data and artificial intelligence includes a data acquisition module, a data processing module, a data analysis module, a data query module and a data security module; wherein,

[0014] The data collection module is used to collect different types of information data in the Internet of Things service according to the preset information collection rules and obtain the cluster information data table;

[0015] The data processing module is used to perform information segmentation, information cleaning, information correction, information encryption and information association on the cluster information data table according to the data format, and obtain a desensitized data association table;

[0016] The data analysis module is used to analyze the desensitized data association table according to the information data characteristics using information extraction tools or information reasoning models, and to construct a reasoning link map;

[0017] The data query module is used to perform a rotation query on the inference link map according to the keywords of the search query content, link the relevant information of the search query content, and sort and display them according to the relevance;

[0018] The data security module is used to provide security protection for the operating status of each functional module of the IoT cluster data analysis system.

[0019] Preferably, the information collection rules are formulated according to the service characteristics of the Internet of Things; the service characteristics of the Internet of Things include device diversity, massive data, real-time data, heterogeneous integration and physical coupling; the information collection rules first set the compatibility rules, permission rules and capability rules of the Internet of Things devices; then set the screening rules, compression rules and storage rules of the data stream transmitted by the Internet of Things devices; then set the real-time processing rules, priority rules and delay tolerance rules of the data stream; then preset the standardization rules, conversion rules and integration rules of the data stream; finally set the environmental monitoring rules, physical security rules and privacy protection rules for the deployment space of the Internet of Things devices.

[0020] Preferably, the data acquisition module uses the first set of communication protocols to connect to the current IoT device according to the information collection rules; and calculates the compatibility of each IoT device, and compares the obtained compatibility with the preset compatibility value:

[0021] If the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capability of the IoT device is evaluated according to the hardware configuration of the IoT device, thereby constructing an initial parallel transmission sequence table;

[0022] If the compatibility is less than the preset compatibility value, the communication protocol is changed to connect with the current IoT device again, and the compatibility is calculated again until the compatibility is greater than or equal to the preset compatibility value; when the compatibility of all communication protocols is less than the preset compatibility value, the communication protocol with the highest compatibility is selected to connect with the current IoT device, and the IoT device is authenticated and authorized for transmission; at the same time, the transmission capacity of the IoT device is evaluated according to the hardware configuration of the IoT device to add it to the initial parallel transmission sequence list;

[0023] Subsequently, the data stream of each IoT device in the initial parallel transmission sequence table is parsed to obtain a number of data elements.

[0024] Preferably, the data acquisition module sequentially matches the data element with all the data elements in the preset element list to obtain the key data element, and then screens and aggregates all the key data elements in the data stream, and calculates the data occupied space; and compares the acquired data occupied space with the set data space threshold:

[0025] If the data occupied space is greater than or equal to the data space threshold, the data occupied space is compressed to the data space threshold, and then the original data occupied space in the initial parallel transmission sequence table is updated to the corresponding data space threshold, and the data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0026] If the data occupied space is less than the data space threshold, the original data occupied space in the initial parallel transmission sequence table is retained, and a data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0027] Subsequently, the preferred parallel transmission sequence table is sorted from strong to weak within the delay tolerance for the timeliness of all key data elements within the data retention period, and a priority from low to high is set for each key data element, and a time-series parallel transmission table is obtained according to the key data elements after the sequence sorting;

[0028] Finally, format conversion is performed on all key data elements of different data formats sorted by priority in the time-series parallel transmission table;

[0029] According to the above method, the key data elements of different IoT devices are formatted, and then the key data elements of different IoT devices sorted by priority are integrated to obtain the data stream after format conversion, and finally the cluster information data table of IoT devices is obtained and transmitted to the data processing module for preprocessing.

[0030] Preferably, the data processing module performs data format recovery on the data stream after format conversion in the received cluster information data table to obtain the data stream in the original data format; and divides the recovered cluster information data table into several single-category information data tables according to the data format;

[0031] The data processing module first uses a data segmentation algorithm to divide all single-category information data tables into several information data blocks according to the transmission time; then uses a data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses an anomaly correction algorithm to detect anomalies and correct information on the concise information data blocks to obtain corrected information data blocks; then uses a data clustering algorithm to cluster and separate the information data in the corrected information data blocks according to the data sensitivity level to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks and high-sensitivity information separation blocks; then uses a chaotic encryption algorithm to perform sensitivity-level secondary encryption on the public-level information separation blocks and the internal-level information separation blocks, and uses a chaotic encryption algorithm and a quantum encryption algorithm to perform nested-level encryption on the sensitive-level information separation blocks and the high-sensitivity information separation blocks to obtain confidential mixed data blocks; finally, uses a distributed association algorithm according to the serial code of the Internet of Things device to aggregate and normalize all confidential mixed data blocks to obtain a desensitized data association table.

[0032] Preferably, the data analysis module uses the information dimensionality reduction layer of the information reasoning model and the dimension compression algorithm to perform dimensionality reduction on the desensitized data association table several times to obtain the dimensionality reduction data association table:

[0033] T=σ(WX s +b)+Switch(Q E (WX s ,Y),Q F (Q E (WX s ,Y))) (1)

[0035] Where T represents the dimension reduction data association table, σ() represents the activation function, W represents the weight value, X represents the information data sample in the desensitized data association table, s represents the data sensitivity level, s=1,2,3,4, b represents the deviation vector, Switch() represents the selection function, Q E represents the chaotic encryption algorithm, Y represents the true label, Q F represents the quantum encryption algorithm;

[0036] The data parsing layer of the information reasoning model uses the information parsing algorithm to parse the dimension-reduced data association table according to the composition of IoT devices, and obtains the hardware configuration data set, software operation data set, and fusion device data set:

[0037]

[0038] Among them, γ represents the fault tolerance rate, i represents the hardware serial number, n represents the number of hardware, and α represents each hardware data. represents the hardware configuration data set, j represents the software serial number, m represents the number of software, β represents each software data, Represents a collection of software running data. represents the Kronene product, and ∑σ represents the fused device data set.

[0039] Preferably, the reasoning and discrimination layer of the information reasoning model uses a dynamic discrimination algorithm to determine the abnormal data of the hardware configuration data set, the software operation data set and the fusion device data set according to the information data characteristics, and obtains the abnormal data set:

[0040]

[0041] Among them, NG represents an abnormal data set, iForest() represents a tree-based function, KNN() represents a neighbor distance function, LOF() represents a local outlier function, and SVM() represents a classifier;

[0042] Then correct the abnormal data set:

[0043]

[0044] Among them, NG ′ represents the corrected abnormal data set, ε represents the correction factor, l represents the number of outlier regressions, and δ represents the total number of outlier regressions;

[0045] The graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into a reasoning link graph:

[0046]

[0047] Among them, G represents the reasoning link graph, KG() represents the knowledge graph function, It represents multiple vector addition, and Infomap() represents the community information inference function.

[0048] Preferably, the data query module includes a keyword extraction unit, a content matching unit and a graph display unit; the keyword extraction unit uses the TextRank algorithm to extract keywords for retrieving query content, and searches for similar keywords from the inference link graph; the content matching unit uses a collaborative filtering algorithm to extend the link content in the inference link graph based on similar keywords; the graph display unit uses a touch screen to interactively display the link content searched in the inference link graph.

[0049] Preferably, the data security module uses a mimetic defense model to perform real-time monitoring and protection on each functional module of the IoT cluster data analysis system; the mimetic defense model includes a static defense layer, a dynamic defense layer and a fusion scheduling layer; the static defense layer uses an IDPS algorithm to monitor the operating status of each functional module in real time; the dynamic defense layer uses a GAN algorithm to simulate external intrusion events and / or abnormal data sets of each functional module, generates corresponding dynamic defense strategies, and uses an alarm or short message to notify enterprise managers; when an external intrusion event occurs, the fusion scheduling layer schedules the static defense layer to fully defend the severely invaded functional modules according to the degree of intrusion of each functional module, and intercepts samples of external intrusion events and uses the dynamic defense layer to evolve the defense strategy, thereby completing the all-round protection of the IoT cluster data analysis system; the functional modules include a data acquisition module, a data processing module, a data analysis module and a data query module.

[0050] The present invention collects different types of information data in the Internet of Things service according to preset information collection rules through the data acquisition module, obtains the cluster information data table, analyzes the cluster information data table according to the information data characteristics through the information reasoning model, obtains the abnormal data set, and constructs the reasoning link map; through the mimetic defense model of the data security module, the various functional modules of the Internet of Things cluster data analysis system are monitored and protected in real time; the connection stability of different types of Internet of Things devices is improved, and the data flow security of Internet of Things devices is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 This is the flow chart of the IoT cluster data analysis system;

[0053] Figure 2 Schematic diagram of the information reasoning model. DETAILED DESCRIPTION

[0054] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0055] like Figure 1-Figure 2 As shown, the Internet of Things cluster data analysis system based on big data and artificial intelligence described in the present invention includes a data acquisition module, a data processing module, a data analysis module, a data query module and a data security module; wherein,

[0056] The data collection module is used to collect different types of information data in the Internet of Things service according to the preset information collection rules and obtain the cluster information data table;

[0057] The data processing module is used to perform information segmentation, information cleaning, information correction, information encryption and information association on the cluster information data table according to the data format, and obtain a desensitized data association table;

[0058] The data analysis module is used to analyze the desensitized data association table according to the information data characteristics using information extraction tools or information reasoning models, and to construct a reasoning link map;

[0059] In this embodiment, information data characteristics include accuracy, confidentiality, integrity, timeliness, availability and security;

[0060] Accuracy: Information data should be accurate to avoid wrong decisions due to erroneous data;

[0061] Completeness: Data should be complete and no key information should be omitted;

[0062] Timeliness: Data should be time-sensitive and reflect the current situation in a timely manner;

[0063] Availability: Data and information are easily accessible or available when needed;

[0064] Confidentiality: The disclosure of patents, technical know-how, etc. should be prevented and avoided;

[0065] Safety: Ensure that there is no danger during the data and information processing process; Data query module, used to perform rotation query on the inference link map according to the keywords of the search query content, link the relevant information of the search query content, and sort and display them according to the relevance;

[0066] The data security module is used to provide security protection for the operating status of each functional module of the IoT cluster data analysis system.

[0067] In the present invention, information collection rules are formulated according to the service characteristics of the Internet of Things; the service characteristics of the Internet of Things include device diversity, massive data, real-time data, heterogeneous integration and physical coupling; the information collection rules first set the compatibility rules, authority rules and capability rules of the Internet of Things devices; then set the screening rules, compression rules and storage rules of the data stream transmitted by the Internet of Things devices; then set the real-time processing rules, priority rules and delay tolerance rules of the data stream; then preset the standardization rules, conversion rules and integration rules of the data stream; finally set the environmental monitoring rules, physical security rules and privacy protection rules for the deployment space of the Internet of Things devices.

[0068] In this embodiment, device diversity includes compatibility rules, permission rules, and capability rules; compatibility rules: ensure that information collection rules apply to all types of IoT devices, whether they are sensors, actuators, or other types of devices; permission rules: formulate rules to verify the identity and permissions of devices to prevent unauthorized devices from joining the network; capability rules: adjust the frequency and amount of data collection according to the processing power, storage capacity, and energy efficiency of the device;

[0069] The massiveness of data includes screening rules, compression rules and storage rules; screening rules: only collect data that is critical to achieving service goals and avoid indiscriminate data floods; compression rules: compress data when possible to reduce storage and transmission requirements; storage rules: formulate data storage strategies, including data retention periods and data archiving methods;

[0070] Data real-time includes real-time processing rules, priority rules and delay tolerance rules; real-time processing rules: formulate data collection and processing rules for services that require real-time response to ensure the real-time availability of data; priority rules: formulate priorities based on the importance of data to ensure that key data can be processed first; delay tolerance rules: for non-critical data, a certain transmission and processing delay can be tolerated;

[0071] Heterogeneous integration includes standardization rules, conversion rules and integration rules; Standardization rules: formulate standardized rules for data formats and protocols so that data between different devices and services can interoperate; Conversion rules: provide clear rules and tools for conversion between different data formats; Integration rules: ensure that data from different sources can be effectively integrated to provide a comprehensive perspective;

[0072] Physical coupling includes environmental monitoring rules, physical security rules and privacy protection rules; Environmental monitoring rules: When collecting environmental data, ensure compliance with relevant environmental protection laws and regulations; Physical security rules: Establish rules to protect IoT devices from physical damage and ensure the continuity of data collection; Privacy protection rules: Especially when it comes to monitoring personal space or behavior, ensure that the collected data does not infringe on personal privacy.

[0073] In the present invention, the data acquisition module uses the first set of communication protocols to connect with the current IoT device according to the information collection rules; and calculates the compatibility of each IoT device, and compares the obtained compatibility with the compatible preset value:

[0074] If the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capability of the IoT device is evaluated according to the hardware configuration of the IoT device, thereby constructing an initial parallel transmission sequence table;

[0075] If the compatibility is less than the preset compatibility value, the communication protocol is changed to connect with the current IoT device again, and the compatibility calculation is performed again until the compatibility is greater than or equal to the preset compatibility value; when the compatibility of all communication protocols is less than the preset compatibility value, the communication protocol with the highest compatibility is selected to connect with the current IoT device, and the IoT device is authenticated and authorized for transmission; at the same time, the transmission capacity of the IoT device is evaluated according to the hardware configuration of the IoT device to add it to the initial parallel transmission sequence list;

[0076] Subsequently, the data stream of each IoT device in the initial parallel transmission sequence list is parsed to obtain a number of data elements;

[0077] The data acquisition module matches the data elements with all the data elements in the preset element list in turn to obtain the key data elements, and then screens and aggregates all the key data elements in the data stream, and calculates the space occupied by the data; the acquired data space occupied is compared with the set data space threshold:

[0078] If the data occupied space is greater than or equal to the data space threshold, the data occupied space is compressed to the data space threshold, and then the original data occupied space in the initial parallel transmission sequence table is updated to the corresponding data space threshold, and the data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0079] If the data occupied space is less than the data space threshold, the original data occupied space in the initial parallel transmission sequence table is retained, and a data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0080] Subsequently, the preferred parallel transmission sequence table is sorted from strong to weak within the delay tolerance for the timeliness of all key data elements within the data retention period, and a priority from low to high is set for each key data element, and a time-series parallel transmission table is obtained according to the key data elements after the sequence sorting;

[0081] Finally, format conversion is performed on all key data elements of different data formats sorted by priority in the time-series parallel transmission table;

[0082] According to the above method, the key data elements of different IoT devices are formatted, and then the key data elements of different IoT devices sorted by priority are integrated to obtain the data stream after format conversion, and finally the cluster information data table of IoT devices is obtained and transmitted to the data processing module for preprocessing.

[0083] In this embodiment, the data format includes integer type, real number type, Boolean type, character type, pointer type, array type, record type, set type and file type;

[0084] In the present invention, the data processing module performs data format recovery on the data stream after format conversion in the received cluster information data table to obtain the data stream in the original data format; and divides the recovered cluster information data table into a plurality of single-category information data tables according to the data format;

[0085] The data processing module first uses a data block algorithm to divide all single-category information data tables into several information data blocks according to the transmission time; then uses a data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses an anomaly correction algorithm to detect anomalies and correct information on concise information data blocks to obtain corrected information data blocks; then uses a data clustering algorithm to cluster and separate the information data in the corrected information data blocks according to the data sensitivity level to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks, and high-sensitivity information separation blocks; then uses a chaotic encryption algorithm to perform sensitivity-level secondary encryption on the public-level information separation blocks and the internal-level information separation blocks, and uses a chaotic encryption algorithm and a quantum encryption algorithm to perform nested-level encryption on the sensitive-level information separation blocks and the high-sensitivity information separation blocks to obtain confidential mixed data blocks; finally, uses a distributed association algorithm according to the serial code of the IoT device to aggregate and normalize all confidential mixed data blocks to obtain a desensitized data association table;

[0086] In this embodiment, public level (level 1): this type of data is open to the public and does not contain any sensitive information, such as publicly released press releases, company annual reports, and public research papers.

[0087] Internal level (Level 2): ​​This data is shared within the organization but does not contain sensitive personal or financial information. For example, employee contact lists, internal training materials, non-sensitive business reports, etc.

[0088] Sensitive (Level 3): Contains personally identifiable information (PII) or other sensitive information that requires additional protection measures. For example, employee personal information, customer data, financial records, medical records, etc.

[0089] High-sensitivity data (Level 4): This type of data is vital to the organization, and leakage may lead to serious consequences, requiring the highest level of protection, such as intellectual property, trade secrets, critical infrastructure design, and senior management decision-making information.

[0090] In this embodiment, nested encryption is a multi-layer encryption method, also known as layer-by-layer encryption. Nested encryption encrypts plain text multiple times, and the result of each encryption becomes the plain text of the next encryption.

[0091] In the present invention, the data analysis module uses the information dimensionality reduction layer of the information reasoning model and the dimension compression algorithm to perform several dimensionality reductions on the desensitized data association table to obtain the dimensionality reduction data association table:

[0092] T=σ(WX s +b)+Switch(Q E (WX s ,Y),Q F (Q E (WX s ,Y))) (1)

[0094] Where T represents the dimension reduction data association table, σ() represents the activation function, W represents the weight value, X represents the information data sample in the desensitized data association table, s represents the data sensitivity level, s=1,2,3,4, b represents the deviation vector, Switch() represents the selection function, Q E represents the chaotic encryption algorithm, Y represents the true label, Q F represents the quantum encryption algorithm;

[0095] The data parsing layer of the information reasoning model uses the information parsing algorithm to parse the dimension-reduced data association table according to the composition of IoT devices, and obtains the hardware configuration data set, software operation data set, and fusion device data set:

[0096]

[0097] Among them, γ represents the fault tolerance rate, i represents the hardware serial number, n represents the number of hardware, and α represents each hardware data. represents the hardware configuration data set, j represents the software serial number, m represents the number of software, β represents each software data, Represents a collection of software running data. represents the Krone product, ∑σ represents the fusion device data set;

[0098] The reasoning and discrimination layer of the information reasoning model uses a dynamic discrimination algorithm to determine the abnormal data of the hardware configuration data set, the software operation data set, and the fusion device data set according to the characteristics of the information data, and obtain the abnormal data set:

[0099]

[0100] Among them, NG represents an abnormal data set, iForest() represents a tree-based function, KNN() represents a neighbor distance function, LOF() represents a local outlier function, and SVM() represents a classifier;

[0101] Then correct the abnormal data set:

[0102]

[0103] Among them, NG ′ represents the corrected abnormal data set, ε represents the correction factor, l represents the number of outlier regressions, and δ represents the total number of outlier regressions;

[0104] The graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into a reasoning link graph:

[0105]

[0106] Among them, G represents the reasoning link graph, KG() represents the knowledge graph function, It represents multiple vector addition, and Infomap() represents the community information inference function.

[0107] In the present invention, the data query module includes a keyword extraction unit, a content matching unit and a graph display unit; the keyword extraction unit uses the TextRank algorithm to extract keywords for searching the query content, and searches for similar keywords from the inference link graph; the content matching unit uses the collaborative filtering algorithm to extend the link content in the inference link graph according to similar keywords; the graph display unit uses a touch screen to interactively display the link content searched in the inference link graph;

[0108] In the present invention, the data security module uses a mimetic defense model to perform real-time monitoring and protection on each functional module of the Internet of Things cluster data analysis system; the mimetic defense model includes a static defense layer, a dynamic defense layer and a fusion scheduling layer; the static defense layer uses an IDPS algorithm to monitor the operating status of each functional module in real time; the dynamic defense layer uses a GAN algorithm to simulate external intrusion events and / or abnormal data sets of each functional module, generates corresponding dynamic defense strategies, and uses an alarm or short message to notify enterprise managers; when an external intrusion event occurs, the fusion scheduling layer schedules the static defense layer to fully defend the severely invaded functional modules according to the degree of intrusion of each functional module, and intercepts samples of external intrusion events to evolve defense strategies using the dynamic defense layer, thereby completing the all-round protection of the Internet of Things cluster data analysis system; the functional modules include a data acquisition module, a data processing module, a data analysis module and a data query module.

[0109] Example:

[0110] Information collection rules are formulated according to the characteristics of IoT services; IoT service characteristics include device diversity, massive data, real-time data, heterogeneous integration and physical coupling; information collection rules first set the compatibility rules, permission rules and capability rules of IoT devices; then set the screening rules, compression rules and storage rules of the data stream transmitted by IoT devices; then set the real-time processing rules, priority rules and delay tolerance rules of the data stream; then preset the standardization rules, conversion rules and integration rules of the data stream; finally set environmental monitoring rules, physical security rules and privacy protection rules for the deployment space of IoT devices.

[0111] The data acquisition module calculates the compatibility of each IoT device according to the information collection rules, and compares the obtained compatibility with the preset compatibility value: if the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capacity of the IoT device is evaluated according to the hardware configuration of the IoT device, and then a parallel transmission sequence table is constructed; if the compatibility is less than the preset compatibility value, the communication protocol is changed to connect with the current IoT device again, and then the compatibility is verified and calculated until the compatibility is greater than or equal to the preset compatibility value, and the transmission capacity of the IoT device is evaluated according to the hardware configuration of the IoT device to add it to the parallel transmission sequence table; the data stream of each IoT device in the parallel transmission sequence table is parsed to obtain a number of data elements;

[0112] The data elements are matched with all the data elements in the preset element list in turn, and then all the key data elements in the data stream are filtered and aggregated, and the data occupied space is calculated; the acquired data occupied space is compared with the set data space threshold: if the data occupied space is greater than or equal to the data space threshold, the data occupied space is compressed to the data space threshold, and then the parallel transmission sequence table is updated to obtain the preferred parallel transmission table, and the data retention period is set; if the data occupied space is less than the data space threshold, the data retention period is set; within the data retention period, the preferred parallel transmission table is sorted from strong to weak in terms of the timeliness of all key data elements within the delay tolerance, and a priority is set for each key data element to obtain a time-series parallel transmission table; the priority key data elements of all different data formats in the time-series parallel transmission table are converted in format, and the priority key data elements of different IoT devices are integrated to obtain the cluster information data table of IoT devices, and then transmitted to the data processing module for preprocessing;

[0113] The data processing module recovers the data format of the data stream to be converted in the received cluster information data table to obtain the data stream in the original data format; and divides the recovered cluster information data table into several single-category information data tables according to the data format; the data processing module first uses the data block algorithm to divide all single-category information data tables into several information data blocks according to the transmission time; then uses the data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses the anomaly correction algorithm to detect anomalies and correct information on the concise information data blocks to obtain corrected information data blocks; then according to the data sensitivity level, the data processing module recovers the data stream in the original data format; and then divides the data stream into several single-category information data tables according to the transmission time; and then uses the data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses the anomaly correction algorithm to detect anomalies and correct information on the concise information data blocks to obtain corrected information data blocks; Using a data clustering algorithm, the information data in the correction information data block is clustered and separated to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks and high-sensitivity information separation blocks; then, a chaotic encryption algorithm is used to perform sensitivity-level secondary encryption on the public-level information separation blocks and the internal-level information separation blocks, and a chaotic encryption algorithm and a quantum encryption algorithm are used to perform nested-level encryption on the sensitive-level information separation blocks and the high-sensitivity information separation blocks to obtain confidential mixed data blocks; finally, a distributed association algorithm is used according to the serial code of the Internet of Things device to aggregate and normalize all confidential mixed data blocks to obtain a desensitized data association table.

[0114] The data analysis module uses the information dimension reduction layer of the information reasoning model to perform several dimension reductions on the desensitized data association table using the dimensionality compression algorithm to obtain the reduced dimension data association table; the data parsing layer of the information reasoning model uses the information parsing algorithm to perform data parsing on the reduced dimension data association table according to the composition of the Internet of Things devices to obtain the hardware configuration data set, software operation data set and fusion device data set; the reasoning discrimination layer of the information reasoning model uses the dynamic discrimination algorithm to determine the abnormal data of the hardware configuration data set, software operation data set and fusion device data set according to the characteristics of the information data, and obtains the abnormal data set; then the abnormal data set is corrected; the graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into an inference link graph.

[0115] The keyword extraction unit of the data query module uses the TextRank algorithm to extract keywords for the search query content and searches for similar keywords from the inference link map; the content matching unit uses the collaborative filtering algorithm to extend the link content in the inference link map based on similar keywords; the map display unit uses the touch screen to interactively display the link content searched in the inference link map.

[0116] The data security module uses the IDPS algorithm through the static defense layer of the mimetic defense model to monitor the operating status of each functional module in real time; uses the GAN algorithm through the dynamic defense layer to simulate the external intrusion events and / or abnormal data sets of each functional module, generates corresponding dynamic defense strategies, and uses alarms or short messages to notify enterprise managers; also through the fusion scheduling layer, when an external intrusion event occurs, according to the intrusion degree of each functional module, the static defense layer is scheduled to fully defend the functional modules with serious intrusion, and samples of external intrusion events are intercepted to evolve the defense strategy using the dynamic defense layer, thereby completing the all-round protection of the IoT cluster data analysis system; the functional modules include data acquisition module, data processing module, data analysis module and data query module.

Claims

1. An Internet of Things cluster data analysis system based on big data and artificial intelligence, characterized by: It includes data acquisition module, data processing module, data analysis module, data query module and data security module; among them, The data collection module is used to collect different types of information data in the Internet of Things service according to the preset information collection rules and obtain the cluster information data table; The data processing module is used to perform information segmentation, information cleaning, information correction, information encryption and information association on the cluster information data table according to the data format, and obtain a desensitized data association table; The data analysis module is used to analyze the desensitized data association table according to the information data characteristics using information extraction tools or information reasoning models, and to construct a reasoning link map; The data query module is used to perform a rotation query on the inference link map according to the keywords of the search query content, link the relevant information of the search query content, and sort and display them according to the relevance; The data security module is used to provide security protection for the operating status of each functional module of the IoT cluster data analysis system.

2. The IoT cluster data analysis system based on big data and artificial intelligence according to claim 1 is characterized by: The information collection rules are formulated according to the service characteristics of the Internet of Things; the service characteristics of the Internet of Things include device diversity, massive data, real-time data, heterogeneous integration and physical coupling; the information collection rules first set the compatibility rules, permission rules and capability rules of the Internet of Things devices; then set the screening rules, compression rules and storage rules of the data stream transmitted by the Internet of Things devices; then set the real-time processing rules, priority rules and delay tolerance rules of the data stream; then preset the standardization rules, conversion rules and integration rules of the data stream; finally set the environmental monitoring rules, physical security rules and privacy protection rules for the deployment space of the Internet of Things devices.

3. The IoT cluster data analysis system based on big data and artificial intelligence according to claim 1 is characterized by: The data acquisition module uses the first set of communication protocols to connect to the current IoT device according to the information collection rules; and calculates the compatibility of each IoT device, and compares the obtained compatibility with the preset compatibility value: If the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capability of the IoT device is evaluated according to the hardware configuration of the IoT device, thereby constructing an initial parallel transmission sequence table; If the compatibility is less than the preset compatibility value, the communication protocol is changed to connect to the current IoT device again, and the compatibility is calculated again until the compatibility is greater than or equal to the preset compatibility value; When the compatibility of all communication protocols is less than the preset compatibility value, the communication protocol with the highest compatibility is selected to connect to the current IoT device, and the IoT device is authenticated and authorized for transmission; at the same time, the transmission capability of the IoT device is evaluated according to the hardware configuration of the IoT device to add it to the initial parallel transmission sequence list; Subsequently, the data stream of each IoT device in the initial parallel transmission sequence table is parsed to obtain a number of data elements.

4. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 3 is characterized by: The data acquisition module sequentially matches the data element with all the data elements in the preset element list to obtain the key data element, and then screens and aggregates all the key data elements in the data stream, and calculates the data occupied space; compares the acquired data occupied space with the set data space threshold: If the data occupied space is greater than or equal to the data space threshold, the data occupied space is compressed to the data space threshold, and then the original data occupied space in the initial parallel transmission sequence table is updated to the corresponding data space threshold, and the data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained; If the data occupied space is less than the data space threshold, the original data occupied space in the initial parallel transmission sequence table is retained, and a data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained; Subsequently, the preferred parallel transmission sequence table is sorted from strong to weak within the delay tolerance for the timeliness of all key data elements within the data retention period, and a priority from low to high is set for each key data element, and a time-series parallel transmission table is obtained according to the key data elements after the sequence sorting; Finally, format conversion is performed on all key data elements of different data formats sorted by priority in the time-series parallel transmission table; According to the above method, the key data elements of different IoT devices are formatted, and then the key data elements of different IoT devices sorted by priority are integrated to obtain the data stream after format conversion, and finally the cluster information data table of IoT devices is obtained and transmitted to the data processing module for preprocessing.

5. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 1 is characterized by: The data processing module performs data format recovery on the data stream after format conversion in the received cluster information data table to obtain the data stream in the original data format; and divides the recovered cluster information data table into a plurality of single-category information data tables according to the data format; The data processing module first uses a data block algorithm to divide all single-category information data tables into several information data blocks according to the transmission time; then uses a data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses an anomaly correction algorithm to perform anomaly detection and information correction on the concise information data blocks to obtain corrected information data blocks; Then, according to the data sensitivity level, the data clustering algorithm is used to cluster and separate the information data in the correction information data block to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks and high-sensitivity information separation blocks; then the chaotic encryption algorithm is used to perform sensitivity-level secondary encryption on the public-level information separation blocks and the internal-level information separation blocks, and the chaotic encryption algorithm and the quantum encryption algorithm are used to perform nested-level encryption on the sensitive-level information separation blocks and the high-sensitivity information separation blocks to obtain confidential mixed data blocks; finally, according to the serial code of the Internet of Things device, the distributed association algorithm is used to aggregate and normalize all confidential mixed data blocks to obtain the desensitized data association table.

6. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 1 is characterized by: The data analysis module uses the information dimensionality reduction layer of the information reasoning model and the dimension compression algorithm to perform several dimensionality reductions on the desensitized data association table to obtain the dimensionality reduction data association table: T=σ(WX s +b)+Switch(Q E (WX s ,Y),Q F (Q E (WX s ,Y))) (1) Where T represents the dimension reduction data association table, σ() represents the activation function, W represents the weight value, X represents the information data sample in the desensitized data association table, s represents the data sensitivity level, s=1,2,3,4, b represents the deviation vector, Switch() represents the selection function, Q E represents the chaotic encryption algorithm, Y represents the true label, Q F represents the quantum encryption algorithm; The data parsing layer of the information reasoning model uses the information parsing algorithm to parse the dimension-reduced data association table according to the composition of IoT devices, and obtains the hardware configuration data set, software operation data set, and fusion device data set: Among them, γ represents the fault tolerance rate, i represents the hardware serial number, n represents the number of hardware, and α represents each hardware data. represents the hardware configuration data set, j represents the software serial number, m represents the number of software, β represents each software data, Represents a collection of software running data. represents the Kronene product, and ∑σ represents the fused device data set.

7. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 6 is characterized by: The reasoning and discrimination layer of the information reasoning model uses a dynamic discrimination algorithm to determine the abnormal data of the hardware configuration data set, the software operation data set and the fusion device data set according to the information data characteristics, and obtains the abnormal data set: Among them, NG represents an abnormal data set, iForest() represents a tree-based function, KNN() represents a neighbor distance function, LOF() represents a local outlier function, and SVM() represents a classifier; Then correct the abnormal data set: Among them, NG ′ represents the corrected abnormal data set, ε represents the correction factor, l represents the number of outlier regressions, and δ represents the total number of outlier regressions; The graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into a reasoning link graph: Among them, G represents the reasoning link graph, KG() represents the knowledge graph function, It represents multiple vector addition, and Infomap() represents the community information inference function.

8. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 1, characterized in that: The data query module includes a keyword extraction unit, a content matching unit and a graph display unit; the keyword extraction unit uses the TextRank algorithm to extract keywords for retrieving query content and searches for similar keywords from the inference link graph; the content matching unit uses the collaborative filtering algorithm to extend the link content in the inference link graph based on similar keywords; the graph display unit interactively displays the link content searched in the inference link graph.

9. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 1, characterized in that: The data security module uses a mimetic defense model to monitor and protect the functional modules of the IoT cluster data analysis system in real time; the mimetic defense model includes a static defense layer, a dynamic defense layer, and a fusion scheduling layer; the static defense layer uses the IDPS algorithm to monitor the operating status of each functional module in real time; the dynamic defense layer uses the GAN algorithm to simulate external intrusion events and / or abnormal data sets of each functional module, generate corresponding dynamic defense strategies, and notify enterprise managers using alarms or short messages; When an external intrusion event occurs, the fusion scheduling layer schedules the static defense layer to fully defend the seriously invaded functional modules according to the intrusion degree of each functional module, and intercepts samples of external intrusion events to evolve the defense strategy using the dynamic defense layer, thereby completing the all-round protection of the IoT cluster data analysis system; the functional modules include data acquisition module, data processing module, data analysis module and data query module.

Images