An IoT cluster data analysis system based on big data and artificial intelligence

Abstract

The present invention discloses an Internet of Things cluster data analysis system based on big data and artificial intelligence, which relates to the field of artificial intelligence technology and solves the problem that the devices in the Internet of Things cluster are complex and numerous, and it is impossible to ensure that the data streams of all Internet of Things devices can be connected and collected, and it is also impossible to ensure the data security of all Internet of Things devices during the data analysis process; the Internet of Things cluster data analysis system includes a data acquisition module, a data processing module, a data analysis module, a data query module and a data security module; the data acquisition module collects different types of information data in the Internet of Things service according to preset information collection rules, obtains a cluster information data table, analyzes the cluster information data table according to the information data characteristics through an information reasoning model, obtains an abnormal data set, and constructs an inference link map; the data security module uses a mimetic defense model to monitor and protect the various functional modules of the Internet of Things cluster data analysis system in real time; improves the connection stability of different types of Internet of Things devices, and ensures the data flow security of Internet of Things devices.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an Internet of Things cluster data analysis system based on big data and artificial intelligence. Background Art

[0002] With the continuous development and application of IoT technology, more and more devices and sensors are connected to the internet, transmitting massive amounts of data to data centers for storage and processing. This data includes not only device operating data and environmental data collected by sensors, but also user behavior data and business data. How to effectively utilize this data, extract valuable information, and help businesses and institutions make informed decisions has become a new challenge for the IoT industry.

[0003] IoT cluster data analysis systems based on big data and artificial intelligence have emerged. By collecting and analyzing data and leveraging big data and artificial intelligence algorithms, these systems analyze, predict, and make decisions based on the massive amounts of data generated by IoT devices and sensors. These systems can help businesses and institutions better understand data, gain insights into consumer and market trends, and optimize key operations and production decisions.

[0004] In IoT clusters, sensors continuously collect environmental data such as temperature, humidity, and vibration. Abnormal data may indicate potential security threats, such as fire or intrusion. By constructing an inference link graph and analyzing the relationship between abnormal data points and other data points, we can quickly locate the source of the threat and assess its scope.

[0005] In the Industrial Internet of Things, machine operating data is monitored in real time, and abnormal data may indicate equipment failure. By constructing an inference link graph, abnormal data can be compared with the historical failure patterns of the equipment to predict potential failure points, thereby performing timely maintenance.

[0006] In the medical internet of things, patients' vital signs are continuously monitored, and any anomalies may indicate changes in their health. By constructing an inference link graph, doctors can analyze the relationship between abnormal data and diseases, providing support for diagnosis and treatment.

[0007] In the logistics IoT, the status of goods during transportation is tracked in real time, and abnormal data may indicate a supply chain disruption. By constructing an inference link graph, we can reveal the connection between abnormal events and other links in the supply chain, helping managers to respond quickly and optimize processes.

[0008] Patent No. CN2024102484173 discloses an enterprise big data analysis system based on artificial intelligence. The system includes a data acquisition module, a data cleaning module, a data analysis module, a prediction module, and an early warning module. The data acquisition module is used to collect various types of data inside and outside the enterprise, including sales data, market data, and customer data. The data cleaning module is used to clean and organize the collected data to ensure the accuracy and completeness of the data. The data analysis module is used to analyze the cleaned data and discover the correlations and patterns between the data. The above invention, by introducing artificial intelligence technology, can more intelligently analyze and predict big data, providing enterprises with more accurate and timely decision-making support. At the same time, the introduction of the early warning module can help enterprises detect data anomalies in a timely manner, avoid possible risks and losses, and improve the operational efficiency and competitiveness of enterprises.

[0009] Patent No. CN2023113527163 discloses an artificial intelligence-based business big data analysis system, which relates to the field of business big data analysis technology. The object data to be analyzed is collected through a data collection end, and temporarily stored after acquisition according to a set capacity. At the same time, the data collection end is also used to encrypt the collected object data of the set capacity. After encryption, subsequent data will be used for big data analysis. During the analysis, the encrypted content will be verified to ensure that the corresponding analysis data has not undergone secondary processing or addition after collection, thereby avoiding interference from some data in the big data to a certain extent, making the analysis results more referenceable. The above application is simple, effective, and easy to use.

[0010] However, the above patents have major flaws in data collection and analysis. Due to the large number of devices in the IoT cluster, the existing data analysis system cannot guarantee that the data streams of all IoT devices can be connected and collected when collecting and processing massive amounts of data, and cannot guarantee the data security of all IoT devices during the data analysis process. Summary of the Invention

[0011] The purpose of the present invention is to provide an Internet of Things cluster data analysis system based on big data and artificial intelligence, which can collect different types of information data in Internet of Things services according to information collection rules through artificial intelligence technology, obtain cluster information data tables, use information reasoning models to analyze cluster information data tables according to information data characteristics, obtain abnormal data sets, and construct reasoning link graphs; use mimetic defense models to monitor and protect various functional modules of the Internet of Things cluster data analysis system in real time; so as to improve the connection stability of different types of Internet of Things devices and ensure the data flow security of Internet of Things devices.

[0012] The present invention utilizes the following technical solutions:

[0013] An Internet of Things cluster data analysis system based on big data and artificial intelligence, including a data acquisition module, a data processing module, a data analysis module, a data query module and a data security module; wherein,

[0014] The data collection module is used to collect different types of information data in the Internet of Things service according to the preset information collection rules and obtain the cluster information data table;

[0015] The data processing module is used to perform information segmentation, information cleaning, information correction, information encryption and information association on the cluster information data table according to the data format to obtain the desensitized data association table;

[0016] The data analysis module is used to analyze the desensitized data association table based on the information data characteristics using information extraction tools or information reasoning models, and to construct an inference link map;

[0017] The data query module is used to perform a rotation query on the inference link map based on the keywords of the search query content, and to link the relevant information of the search query content and display them in order of relevance;

[0018] The data security module is used to provide security protection for the operating status of each functional module of the IoT cluster data analysis system.

[0019] Preferably, information collection rules are formulated based on the service characteristics of the Internet of Things (IoT); IoT service characteristics include device diversity, massive data, real-time data, heterogeneous integration, and physical coupling; information collection rules first set compatibility rules, permission rules, and capability rules for IoT devices; then set screening rules, compression rules, and storage rules for data streams transmitted by IoT devices; then set real-time processing rules, priority rules, and delay tolerance rules for data streams; then preset standardization rules, conversion rules, and integration rules for data streams; and finally set environmental monitoring rules, physical security rules, and privacy protection rules for the deployment space of IoT devices.

[0020] Preferably, the data acquisition module connects to the current IoT device using the first set of communication protocols according to the information collection rules; and calculates the compatibility of each IoT device, and compares the obtained compatibility with the preset compatibility value:

[0021] If the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capability of the IoT device is evaluated based on the hardware configuration of the IoT device, thereby constructing an initial parallel transmission sequence table;

[0022] If the compatibility is less than the preset compatibility value, the communication protocol is changed and the current IoT device is connected again, and the compatibility calculation is performed again until the compatibility is greater than or equal to the preset compatibility value. When the compatibility of all communication protocols is less than the preset compatibility value, the communication protocol with the highest compatibility is selected to connect to the current IoT device, and the IoT device is authenticated and authorized for transmission. At the same time, the transmission capacity of the IoT device is evaluated based on its hardware configuration to add it to the initial parallel transmission sequence list.

[0023] Subsequently, the data stream of each IoT device in the initial parallel transmission sequence table is parsed to obtain a number of data elements.

[0024] Preferably, the data acquisition module sequentially matches the data element with all data elements in the preset element list to obtain key data elements, and then filters and aggregates all key data elements in the data stream, and calculates the data occupied space; and compares the obtained data occupied space with the set data space threshold:

[0025] If the data occupied space is greater than or equal to the data space threshold, the data occupied space is compressed to the data space threshold, and then the original data occupied space in the initial parallel transmission sequence table is updated to the corresponding data space threshold, and the data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0026] If the data occupied space is less than the data space threshold, the original data occupied space in the initial parallel transmission sequence table is retained, and a data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0027] Subsequently, the preferred parallel transmission sequence table is sorted from strong to weak in terms of the timeliness of all key data elements within the data retention period and within the delay tolerance, and a priority is set from low to high for each key data element, and a time-series parallel transmission table is obtained according to the key data elements after the sequence sorting;

[0028] Finally, perform format conversion on all key data elements of different data formats sorted by priority in the time-series parallel transmission table;

[0029] According to the above method, the key data elements of different IoT devices are formatted, and then the key data elements of different IoT devices sorted by priority are integrated to obtain the format-converted data stream, and finally the cluster information data table of IoT devices is obtained and transmitted to the data processing module for preprocessing.

[0030] Preferably, the data processing module performs data format recovery on the data stream after format conversion in the received cluster information data table to obtain the data stream in the original data format; and divides the recovered cluster information data table into several single-category information data tables according to the data format;

[0031] The data processing module first uses a data segmentation algorithm to divide all single-category information data tables into several information data blocks according to the transmission time; then uses a data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses an anomaly correction algorithm to detect anomalies and correct information in the concise information data blocks to obtain corrected information data blocks; then uses a data clustering algorithm to cluster and separate the information data in the corrected information data blocks according to the data sensitivity level to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks and high-sensitivity information separation blocks; then uses a chaotic encryption algorithm to perform sensitivity-level secondary encryption on the public-level information separation blocks and the internal-level information separation blocks, and uses a chaotic encryption algorithm and a quantum encryption algorithm to perform nested-level encryption on the sensitive-level information separation blocks and the high-sensitivity information separation blocks to obtain confidential mixed data blocks; finally, uses a distributed association algorithm according to the serial code of the Internet of Things device to aggregate and normalize all confidential mixed data blocks to obtain a desensitized data association table.

[0032] Preferably, the data analysis module uses the information dimensionality reduction layer of the information reasoning model to perform dimensionality reduction on the desensitized data association table several times using a dimensionality compression algorithm to obtain a dimensionality-reduced data association table:

[0033] T=σ(WX s +b)+Switch(Q E (WX s ,Y),Q F (Q E (WX s ,Y))) (1)

[0035] Among them, T represents the dimension reduction data association table, σ() represents the activation function, W represents the weight value, X represents the information data sample in the desensitized data association table, s represents the data sensitivity level, s=1,2,3,4, b represents the bias vector, Switch() represents the selection function, Q E represents the chaotic encryption algorithm, Y represents the true label, Q F Represents the quantum encryption algorithm;

[0036] The data parsing layer of the information reasoning model uses information parsing algorithms to parse the dimensionality-reduced data association table according to the composition of IoT devices, and obtains hardware configuration data sets, software operation data sets, and fusion device data sets:

[0037]

[0038] Among them, γ represents the fault tolerance rate, i represents the hardware serial number, n represents the number of hardware, and α represents the data of each hardware. represents the hardware configuration data set, j represents the software serial number, m represents the number of software, β represents each software data, Represents a collection of software running data. represents the Krone product, and ∑σ represents the fusion device data set.

[0039] Preferably, the reasoning and discrimination layer of the information reasoning model uses a dynamic discrimination algorithm to determine abnormal data of the hardware configuration data set, the software operation data set, and the fusion device data set according to the information data characteristics, and obtains the abnormal data set:

[0040]

[0041] Among them, NG represents an abnormal data set, iForest() represents a tree-based function, KNN() represents a neighbor distance function, LOF() represents a local outlier function, and SVM() represents a classifier;

[0042] Then correct the abnormal data set:

[0043]

[0044] Among them, NG ′ represents the corrected abnormal data set, ε represents the correction factor, l represents the number of outlier regressions, and δ represents the total number of outlier regressions;

[0045] The graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into a reasoning link graph:

[0046]

[0047] Among them, G represents the inference link graph, KG() represents the knowledge graph function, represents the addition of multiple vectors, and Infomap() represents the community information inference function.

[0048] Preferably, the data query module includes a keyword extraction unit, a content matching unit and a graph display unit; the keyword extraction unit uses the TextRank algorithm to extract keywords for retrieving query content and searches for similar keywords from the inference link graph; the content matching unit uses the collaborative filtering algorithm to extend the link content in the inference link graph based on similar keywords; the graph display unit uses a touch screen to interactively display the link content searched in the inference link graph.

[0049] Preferably, the data security module uses a mimetic defense model to perform real-time monitoring and protection on the functional modules of the IoT cluster data analysis system; the mimetic defense model includes a static defense layer, a dynamic defense layer and a fusion scheduling layer; the static defense layer uses the IDPS algorithm to monitor the operating status of each functional module in real time; the dynamic defense layer uses the GAN algorithm to simulate external intrusion events and / or abnormal data sets of each functional module, generates corresponding dynamic defense strategies, and uses alarms or short messages to notify enterprise managers; when an external intrusion event occurs, the fusion scheduling layer dispatches the static defense layer to fully defend the severely invaded functional modules according to the degree of intrusion of each functional module, and intercepts samples of external intrusion events and uses the dynamic defense layer to evolve the defense strategy, thereby completing the all-round protection of the IoT cluster data analysis system; the functional modules include a data acquisition module, a data processing module, a data analysis module and a data query module.

[0050] The present invention collects different types of information data in the Internet of Things service according to preset information collection rules through the data acquisition module, obtains the cluster information data table, analyzes the cluster information data table according to the information data characteristics through the information reasoning model, obtains the abnormal data set, and constructs the reasoning link map; through the mimetic defense model of the data security module, the various functional modules of the Internet of Things cluster data analysis system are monitored and protected in real time; the connection stability of different types of Internet of Things devices is improved, and the data flow security of Internet of Things devices is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0052] Figure 1 This is the flow chart of the IoT cluster data analysis system;

[0053] Figure 2 Schematic diagram of the principle of the information reasoning model. DETAILED DESCRIPTION

[0054] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0055] like Figure 1-Figure 2 As shown, the Internet of Things cluster data analysis system based on big data and artificial intelligence described in the present invention includes a data acquisition module, a data processing module, a data analysis module, a data query module and a data security module; wherein,

[0056] The data collection module is used to collect different types of information data in the Internet of Things service according to the preset information collection rules and obtain the cluster information data table;

[0057] The data processing module is used to perform information segmentation, information cleaning, information correction, information encryption and information association on the cluster information data table according to the data format to obtain the desensitized data association table;

[0058] The data analysis module is used to analyze the desensitized data association table based on the information data characteristics using information extraction tools or information reasoning models, and to construct an inference link map;

[0059] In this embodiment, information data characteristics include accuracy, confidentiality, integrity, timeliness, availability, and security;

[0060] Accuracy: Information and data should be accurate to avoid wrong decisions caused by erroneous data;

[0061] Completeness: Data should be complete and no key information should be omitted;

[0062] Timeliness: Data should be time-sensitive and reflect the current situation in a timely manner;

[0063] Availability: data and information are easily accessible or available when needed;

[0064] Confidentiality: The disclosure of patents, technical know-how, etc. should be prevented and avoided;

[0065] Security: Ensure that there are no risks during data and information processing. The data query module is used to perform rotational queries on the inference link graph based on the keywords of the search query content, link the relevant information of the search query content, and display them in order of relevance.

[0066] The data security module is used to provide security protection for the operating status of each functional module of the IoT cluster data analysis system.

[0067] In the present invention, information collection rules are formulated according to the service characteristics of the Internet of Things; the service characteristics of the Internet of Things include device diversity, massive data, real-time data, heterogeneous integration and physical coupling; the information collection rules first set the compatibility rules, permission rules and capability rules of the Internet of Things devices; then set the screening rules, compression rules and storage rules of the data stream transmitted by the Internet of Things devices; then set the real-time processing rules, priority rules and delay tolerance rules of the data stream; then preset the standardization rules, conversion rules and integration rules of the data stream; finally set the environmental monitoring rules, physical security rules and privacy protection rules for the deployment space of the Internet of Things devices.

[0068] In this embodiment, device diversity includes compatibility rules, permission rules, and capability rules. Compatibility rules ensure that information collection rules apply to all types of IoT devices, whether they are sensors, actuators, or other types of devices. Permission rules establish rules to verify the identity and permissions of devices and prevent unauthorized devices from joining the network. Capability rules adjust the frequency and amount of data collection based on the processing power, storage capacity, and energy efficiency of the devices.

[0069] Data mass includes filtering rules, compression rules, and storage rules. Filtering rules: only collect data that is critical to achieving service goals and avoid indiscriminate data floods. Compression rules: compress data whenever possible to reduce storage and transmission requirements. Storage rules: formulate data storage strategies, including data retention periods and data archiving methods.

[0070] Data real-time performance includes real-time processing rules, priority rules, and delay tolerance rules. Real-time processing rules: Develop data collection and processing rules for services that require real-time responses to ensure real-time data availability. Priority rules: Develop priorities based on data importance to ensure that critical data is processed first. Delay tolerance rules: For non-critical data, a certain amount of transmission and processing delay can be tolerated.

[0071] Heterogeneous integration includes standardization rules, conversion rules, and integration rules. Standardization rules: formulate standardized rules for data formats and protocols so that data between different devices and services can interoperate. Conversion rules: provide clear rules and tools for converting between different data formats. Integration rules: ensure that data from different sources can be effectively integrated to provide a comprehensive perspective.

[0072] Physical coupling includes environmental monitoring rules, physical security rules, and privacy protection rules; environmental monitoring rules: when collecting environmental data, ensure compliance with relevant environmental protection laws and regulations; physical security rules: formulate rules to protect IoT devices from physical damage and ensure the continuity of data collection; privacy protection rules: especially when it comes to monitoring personal space or behavior, ensure that the collected data does not infringe on personal privacy.

[0073] In the present invention, the data acquisition module uses the first set of communication protocols to connect to the current IoT device according to the information collection rules; and calculates the compatibility of each IoT device and compares the obtained compatibility with the compatibility preset value:

[0074] If the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capability of the IoT device is evaluated based on the hardware configuration of the IoT device, thereby constructing an initial parallel transmission sequence table;

[0075] If the compatibility is less than the preset compatibility value, the communication protocol is changed and the current IoT device is connected again, and the compatibility calculation is performed again until the compatibility is greater than or equal to the preset compatibility value. When the compatibility of all communication protocols is less than the preset compatibility value, the communication protocol with the highest compatibility is selected to connect to the current IoT device, and the IoT device is authenticated and authorized for transmission. At the same time, the transmission capacity of the IoT device is evaluated based on its hardware configuration to add it to the initial parallel transmission sequence list.

[0076] Subsequently, the data stream of each IoT device in the initial parallel transmission sequence list is parsed to obtain a number of data elements;

[0077] The data acquisition module matches the data elements with all the data elements in the preset element list in sequence to obtain key data elements, and then filters and aggregates all key data elements in the data stream and calculates the data occupied space; the obtained data occupied space is compared with the set data space threshold:

[0078] If the data occupied space is greater than or equal to the data space threshold, the data occupied space is compressed to the data space threshold, and then the original data occupied space in the initial parallel transmission sequence table is updated to the corresponding data space threshold, and the data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0079] If the data occupied space is less than the data space threshold, the original data occupied space in the initial parallel transmission sequence table is retained, and a data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained;

[0080] Subsequently, the preferred parallel transmission sequence table is sorted from strong to weak in terms of the timeliness of all key data elements within the data retention period and within the delay tolerance, and a priority is set from low to high for each key data element, and a time-series parallel transmission table is obtained according to the key data elements after the sequence sorting;

[0081] Finally, perform format conversion on all key data elements of different data formats sorted by priority in the time-series parallel transmission table;

[0082] According to the above method, the key data elements of different IoT devices are formatted, and then the key data elements of different IoT devices sorted by priority are integrated to obtain the format-converted data stream, and finally the cluster information data table of IoT devices is obtained and transmitted to the data processing module for preprocessing.

[0083] In this embodiment, the data formats include integer type, real number type, Boolean type, character type, pointer type, array type, record type, set type and file type;

[0084] In the present invention, the data processing module performs data format recovery on the data stream after format conversion in the received cluster information data table to obtain the data stream in the original data format; and divides the recovered cluster information data table into several single-category information data tables according to the data format;

[0085] The data processing module first uses a data segmentation algorithm to divide all single-category information data tables into several information data blocks according to the transmission time; then uses a data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses an anomaly correction algorithm to detect anomalies and correct information in the concise information data blocks to obtain corrected information data blocks; then uses a data clustering algorithm to cluster and separate the information data in the corrected information data blocks according to the data sensitivity level to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks and high-sensitivity information separation blocks; then uses a chaotic encryption algorithm to perform sensitivity-level secondary encryption on the public-level information separation blocks and the internal-level information separation blocks, and uses a chaotic encryption algorithm and a quantum encryption algorithm to perform nested-level encryption on the sensitive-level information separation blocks and the high-sensitivity information separation blocks to obtain confidential mixed data blocks; finally, uses a distributed association algorithm based on the serial code of the IoT device to aggregate and normalize all confidential mixed data blocks to obtain a desensitized data association table;

[0086] In this embodiment, public level (level 1): This type of data is open to the public and does not contain any sensitive information, such as publicly released press releases, company annual reports, and published research papers.

[0087] Internal (Level 2): ​​This data is shared within the organization but does not contain sensitive personal or financial information. Examples include employee contact lists, internal training materials, and non-sensitive business reports.

[0088] Sensitive (Level 3): Contains personally identifiable information (PII) or other sensitive information that requires additional protection measures, such as employee personal information, customer data, financial records, and medical records.

[0089] High Sensitivity (Level 4): This type of data is critical to the organization and its disclosure could lead to serious consequences, requiring the highest level of protection. Examples include intellectual property, trade secrets, critical infrastructure designs, and senior management decision-making information.

[0090] In this embodiment, nested encryption is a multi-layer encryption method, also known as layered encryption. Nested encryption encrypts plaintext multiple times, and the result of each encryption becomes the plaintext of the next encryption.

[0091] In the present invention, the data analysis module uses the information dimensionality reduction layer of the information inference model to perform several dimensionality reductions on the desensitized data association table using the dimensionality compression algorithm to obtain the reduced dimensionality data association table:

[0092] T=σ(WX s +b)+Switch(Q E (WX s ,Y),Q F (Q E (WX s ,Y))) (1)

[0094] Among them, T represents the dimension reduction data association table, σ() represents the activation function, W represents the weight value, X represents the information data sample in the desensitized data association table, s represents the data sensitivity level, s=1,2,3,4, b represents the bias vector, Switch() represents the selection function, Q E represents the chaotic encryption algorithm, Y represents the true label, Q F Represents the quantum encryption algorithm;

[0095] The data parsing layer of the information reasoning model uses information parsing algorithms to parse the dimensionality-reduced data association table according to the composition of IoT devices, and obtains hardware configuration data sets, software operation data sets, and fusion device data sets:

[0096]

[0097] Among them, γ represents the fault tolerance rate, i represents the hardware serial number, n represents the number of hardware, and α represents the data of each hardware. represents the hardware configuration data set, j represents the software serial number, m represents the number of software, β represents each software data, Represents a collection of software running data. represents the Krone product, ∑σ represents the fusion device data set;

[0098] The reasoning and discrimination layer of the information reasoning model uses a dynamic discrimination algorithm to determine the abnormal data of the hardware configuration data set, software operation data set, and fusion device data set according to the characteristics of the information data, and obtain the abnormal data set:

[0099]

[0100] Among them, NG represents an abnormal data set, iForest() represents a tree-based function, KNN() represents a neighbor distance function, LOF() represents a local outlier function, and SVM() represents a classifier;

[0101] Then correct the abnormal data set:

[0102]

[0103] Among them, NG ′ represents the corrected abnormal data set, ε represents the correction factor, l represents the number of outlier regressions, and δ represents the total number of outlier regressions;

[0104] The graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into a reasoning link graph:

[0105]

[0106] Among them, G represents the inference link graph, KG() represents the knowledge graph function, represents the addition of multiple vectors, and Infomap() represents the community information inference function.

[0107] In the present invention, the data query module includes a keyword extraction unit, a content matching unit and a graph display unit; the keyword extraction unit uses the TextRank algorithm to extract keywords for searching the query content and searches for similar keywords from the inference link graph; the content matching unit uses the collaborative filtering algorithm to extend the link content from the inference link graph based on similar keywords; the graph display unit uses a touch screen to interactively display the link content searched in the inference link graph;

[0108] In the present invention, the data security module uses a mimetic defense model to perform real-time monitoring and protection on the functional modules of the Internet of Things cluster data analysis system; the mimetic defense model includes a static defense layer, a dynamic defense layer and a fusion scheduling layer; the static defense layer uses the IDPS algorithm to monitor the operating status of each functional module in real time; the dynamic defense layer uses the GAN algorithm to simulate the external intrusion events and / or abnormal data sets of each functional module, generate corresponding dynamic defense strategies, and use alarms or short messages to notify enterprise managers; when an external intrusion event occurs, the fusion scheduling layer dispatches the static defense layer to fully defend the functional modules with serious intrusions according to the degree of intrusion of each functional module, and intercepts samples of external intrusion events and uses the dynamic defense layer to evolve the defense strategy, thereby completing the all-round protection of the Internet of Things cluster data analysis system; the functional modules include a data acquisition module, a data processing module, a data analysis module and a data query module.

[0109] Example:

[0110] Information collection rules are formulated based on the characteristics of IoT services; IoT service characteristics include device diversity, massive data, real-time data, heterogeneous integration, and physical coupling. Information collection rules first set the compatibility rules, permission rules, and capability rules for IoT devices. Then, they set the screening rules, compression rules, and storage rules for the data streams transmitted by IoT devices. Then, they set the real-time processing rules, priority rules, and delay tolerance rules for the data streams. Subsequently, they preset the standardization rules, conversion rules, and integration rules for the data streams. Finally, they set environmental monitoring rules, physical security rules, and privacy protection rules for the deployment space of IoT devices.

[0111] The data acquisition module calculates the compatibility of each IoT device according to the information collection rules and compares the obtained compatibility with the preset compatibility value. If the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capability of the IoT device is evaluated based on the hardware configuration of the IoT device, thereby constructing a parallel transmission sequence table. If the compatibility is less than the preset compatibility value, the communication protocol is changed and the current IoT device is reconnected to verify and calculate the compatibility until the compatibility is greater than or equal to the preset compatibility value. At the same time, the transmission capability of the IoT device is evaluated based on the hardware configuration of the IoT device to add it to the parallel transmission sequence table. The data stream of each IoT device in the parallel transmission sequence table is parsed to obtain a number of data elements.

[0112] Match the data elements with all the data elements in the preset element list in turn, and then filter and aggregate all the key data elements in the data stream, and calculate the data occupied space; compare the obtained data occupied space with the set data space threshold: if the data occupied space is greater than or equal to the data space threshold, compress the data occupied space to the data space threshold, and then update the parallel transmission sequence table to obtain the preferred parallel transmission table, and set the data retention period; if the data occupied space is less than the data space threshold, set the data retention period; within the data retention period, sort the timeliness of all key data elements in the preferred parallel transmission table from strong to weak within the delay tolerance, and set a priority for each key data element to obtain a time-series parallel transmission table; perform format conversion on all priority key data elements of different data formats in the time-series parallel transmission table, and integrate the priority key data elements of different IoT devices to obtain the cluster information data table of IoT devices, and then transmit it to the data processing module for preprocessing;

[0113] The data processing module recovers the data format of the data stream to be converted in the received cluster information data table and obtains the data stream in the original data format; and divides the recovered cluster information data table into several single-category information data tables according to the data format; the data processing module first uses the data block algorithm to divide all single-category information data tables into several information data blocks according to the transmission time; then uses the data cleaning algorithm to denoise and deduplicate the information data blocks to obtain concise information data blocks; then uses the anomaly correction algorithm to detect anomalies and correct information on the concise information data blocks to obtain corrected information data blocks; and then according to the data sensitivity level, the data processing module recovers the data stream in the original data format; and then divides the data stream into several single-category information data tables according to the transmission time; and then uses the data cleaning algorithm to detect anomalies and correct information on the concise information data blocks to obtain corrected information data blocks. Using the data clustering algorithm, the information data in the correction information data block is clustered and separated to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks and high-sensitivity information separation blocks; then the public-level information separation blocks and internal-level information separation blocks are sub-encrypted at the sensitivity level using the chaotic encryption algorithm, and the sensitive-level information separation blocks and high-sensitivity information separation blocks are nested-encrypted using the chaotic encryption algorithm and the quantum encryption algorithm to obtain confidential mixed data blocks; finally, according to the serial code of the Internet of Things device, a distributed association algorithm is used to aggregate and normalize all confidential mixed data blocks to obtain a desensitized data association table.

[0114] The data analysis module uses the information dimensionality reduction layer of the information reasoning model to perform several dimensionality reductions on the desensitized data association table using the dimensionality compression algorithm to obtain the reduced dimensionality data association table; the data parsing layer of the information reasoning model uses the information parsing algorithm to perform data parsing on the reduced dimensionality data association table according to the composition of the Internet of Things devices to obtain the hardware configuration data set, software operation data set and fusion device data set; the reasoning discrimination layer of the information reasoning model uses the dynamic discrimination algorithm to determine the abnormal data of the hardware configuration data set, software operation data set and fusion device data set according to the characteristics of the information data, and obtain the abnormal data set; then the abnormal data set is corrected; the graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into an inference link graph.

[0115] The keyword extraction unit of the data query module uses the TextRank algorithm to extract keywords for the search query content and searches for similar keywords from the inference link graph; the content matching unit uses the collaborative filtering algorithm to extend the link content in the inference link graph based on similar keywords; the graph display unit uses the touch screen to interactively display the link content searched in the inference link graph.

[0116] The data security module uses the IDPS algorithm through the static defense layer of the mimetic defense model to monitor the operating status of each functional module in real time; uses the GAN algorithm through the dynamic defense layer to simulate the external intrusion events and / or abnormal data sets of each functional module, generates corresponding dynamic defense strategies, and uses alarms or short messages to notify enterprise managers; also through the fusion scheduling layer, when an external intrusion event occurs, according to the intrusion degree of each functional module, schedules the static defense layer to fully defend the functional modules with serious intrusion, and intercepts samples of external intrusion events to use the dynamic defense layer to evolve the defense strategy, thereby completing the all-round protection of the IoT cluster data analysis system; the functional modules include data acquisition module, data processing module, data analysis module and data query module.

Claims

1. An IoT cluster data analysis system based on big data and artificial intelligence, characterized by: It includes data acquisition module, data processing module, data analysis module, data query module and data security module; among them, The data collection module is used to collect different types of information data in the Internet of Things service according to the preset information collection rules and obtain the cluster information data table; The data processing module is used to perform information segmentation, information cleaning, information correction, information encryption and information association on the cluster information data table according to the data format to obtain the desensitized data association table; The data analysis module is used to analyze the desensitized data association table based on the information data characteristics using information extraction tools or information reasoning models, and to construct an inference link map; The data query module is used to perform a rotation query on the inference link map based on the keywords of the search query content, and to link the relevant information of the search query content and display them in order of relevance; Data security module, used to provide security protection for the operating status of each functional module of the IoT cluster data analysis system; The data analysis module uses the dimensionality reduction layer of the information inference model to perform several dimensionality reductions on the desensitized data association table using the dimensionality compression algorithm to obtain the reduced dimensionality data association table: T=σ(WX s +b)+Switch(Q E (WX s ,Y),Q F (Q E (WX s ,Y))) (1) Among them, T represents the dimension reduction data association table, σ() represents the activation function, W represents the weight value, X represents the information data sample in the desensitized data association table, s represents the data sensitivity level, s=1, 2, 3, 4, b represents the bias vector, Switch() represents the selection function, Q E represents the chaotic encryption algorithm, Y represents the true label, Q F Represents the quantum encryption algorithm; The data parsing layer of the information reasoning model uses information parsing algorithms to parse the dimensionality-reduced data association table according to the composition of IoT devices, and obtains hardware configuration data sets, software operation data sets, and fusion device data sets: Among them, γ represents the fault tolerance rate, i represents the hardware serial number, n represents the number of hardware, and α represents the data of each hardware. represents the hardware configuration data set, j represents the software serial number, m represents the number of software, β represents each software data, Represents a collection of software running data. represents the Krone product, ∑σ represents the fusion device data set; The reasoning and discrimination layer of the information reasoning model uses a dynamic discrimination algorithm to determine abnormal data of the hardware configuration data set, the software operation data set, and the fusion device data set according to the information data characteristics, and obtains the abnormal data set: Among them, NG represents an abnormal data set, iForest() represents a tree-based function, KNN() represents a neighbor distance function, LOF() represents a local outlier function, and SVM() represents a classifier; Then correct the abnormal data set: Where NG′ represents the corrected abnormal data set, ε represents the correction factor, l represents the number of outlier regressions, and δ represents the total number of outlier regressions; The graph construction layer of the information reasoning model uses the knowledge graph construction function to construct the hardware configuration data set, software operation data set, fusion device data set and abnormal data set into a reasoning link graph: Among them, G represents the inference link graph, KG() represents the knowledge graph function, represents the addition of multiple vectors, and Infomap() represents the community information inference function.

2. The IoT cluster data analysis system based on big data and artificial intelligence according to claim 1, characterized in that: The information collection rules are formulated based on the service characteristics of the Internet of Things (IoT); IoT service characteristics include device diversity, massive data, real-time data, heterogeneous integration, and physical coupling. The information collection rules first set the compatibility rules, permission rules, and capability rules of IoT devices; then set the screening rules, compression rules, and storage rules for the data streams transmitted by IoT devices; then set the real-time processing rules, priority rules, and delay tolerance rules for the data streams; then preset the standardization rules, conversion rules, and integration rules for the data streams; and finally set environmental monitoring rules, physical security rules, and privacy protection rules for the deployment space of IoT devices.

3. The IoT cluster data analysis system based on big data and artificial intelligence according to claim 1, characterized in that: The data acquisition module connects to the current IoT device using the first set of communication protocols according to the information collection rules; calculates the compatibility of each IoT device and compares the obtained compatibility with the preset compatibility value: If the compatibility is greater than or equal to the preset compatibility value, the IoT device is authenticated and authorized for transmission, and the transmission capability of the IoT device is evaluated based on the hardware configuration of the IoT device, thereby constructing an initial parallel transmission sequence table; If the compatibility is less than the preset compatibility value, the communication protocol is changed to connect to the current IoT device again, and the compatibility calculation is performed again until the compatibility is greater than or equal to the preset compatibility value; When the compatibility of all communication protocols is less than the preset compatibility value, the communication protocol with the highest compatibility is selected to connect to the current IoT device, and the IoT device is authenticated and authorized for transmission. At the same time, the transmission capability of the IoT device is evaluated based on its hardware configuration to add it to the initial parallel transmission sequence list. Subsequently, the data stream of each IoT device in the initial parallel transmission sequence table is parsed to obtain a number of data elements.

4. The IoT cluster data analysis system based on big data and artificial intelligence according to claim 3 is characterized by: The data acquisition module sequentially matches the data element with all data elements in the preset element list to obtain key data elements, and then filters and aggregates all key data elements in the data stream, and calculates the data occupied space; the obtained data occupied space is compared with the set data space threshold: If the data occupied space is greater than or equal to the data space threshold, the data occupied space is compressed to the data space threshold, and then the original data occupied space in the initial parallel transmission sequence table is updated to the corresponding data space threshold, and the data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained; If the data occupied space is less than the data space threshold, the original data occupied space in the initial parallel transmission sequence table is retained, and a data retention period is set, and finally an updated preferred parallel transmission sequence table is obtained; Subsequently, the preferred parallel transmission sequence table is sorted from strong to weak in terms of the timeliness of all key data elements within the data retention period and within the delay tolerance, and a priority is set from low to high for each key data element, and a time-series parallel transmission table is obtained according to the key data elements after the sequence sorting; Finally, perform format conversion on all key data elements of different data formats sorted by priority in the time-series parallel transmission table; According to the above method, the key data elements of different IoT devices are formatted, and then the key data elements of different IoT devices sorted by priority are integrated to obtain the format-converted data stream, and finally the cluster information data table of IoT devices is obtained and transmitted to the data processing module for preprocessing.

5. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 1, characterized in that: The data processing module performs data format recovery on the data stream after format conversion in the received cluster information data table to obtain the data stream in the original data format; and divides the recovered cluster information data table into several single-category information data tables according to the data format; The data processing module first uses a data segmentation algorithm to divide all single-category information data tables into several information data blocks based on transmission time; then uses a data cleaning algorithm to remove noise and duplicates the information data blocks to obtain concise information data blocks; then uses an anomaly correction algorithm to detect anomalies and correct information in the concise information data blocks to obtain corrected information data blocks; Then, according to the data sensitivity level, the data clustering algorithm is used to cluster and separate the information data in the corrected information data block to obtain public-level information separation blocks, internal-level information separation blocks, sensitive-level information separation blocks and high-sensitivity information separation blocks; then the chaotic encryption algorithm is used to perform sensitivity-level secondary encryption on the public-level information separation blocks and the internal-level information separation blocks, and the chaotic encryption algorithm and the quantum encryption algorithm are used to perform nested-level encryption on the sensitive-level information separation blocks and the high-sensitivity information separation blocks to obtain confidential mixed data blocks; finally, according to the serial code of the Internet of Things device, the distributed association algorithm is used to aggregate and normalize all the confidential mixed data blocks to obtain the desensitized data association table.

6. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 1, characterized in that: The data query module includes a keyword extraction unit, a content matching unit and a graph display unit; the keyword extraction unit uses the TextRank algorithm to extract keywords for retrieving query content and searches for similar keywords from the inference link graph; the content matching unit uses the collaborative filtering algorithm to extend the link content in the inference link graph based on similar keywords; the graph display unit interactively displays the link content searched in the inference link graph.

7. The Internet of Things cluster data analysis system based on big data and artificial intelligence according to claim 1, characterized in that: The data security module uses a mimetic defense model to monitor and protect the functional modules of the IoT cluster data analysis system in real time. The mimetic defense model includes a static defense layer, a dynamic defense layer, and a fusion scheduling layer. The static defense layer uses the IDPS algorithm to monitor the operating status of each functional module in real time. The dynamic defense layer uses the GAN algorithm to simulate external intrusion events and / or abnormal data sets of each functional module, generate corresponding dynamic defense strategies, and notify enterprise managers using alarms or short messages. When an external intrusion event occurs, the fusion scheduling layer dispatches the static defense layer to fully defend the functional modules with serious intrusion according to the degree of intrusion of each functional module, and intercepts samples of external intrusion events and uses the dynamic defense layer to evolve the defense strategy, thereby completing the all-round protection of the IoT cluster data analysis system; the functional modules include data acquisition module, data processing module, data analysis module and data query module.

Images