Internet-based management and control analysis system

Abstract

The application discloses an internet-based management and control analysis system, and a running method thereof, which comprises the following steps: collecting and processing industrial internet data information; performing data cleaning and persistent processing on the collected industrial internet data information; performing running analysis, optimization management and security alarm control management on the industrial internet; the collecting and processing of the industrial internet data information comprises the following steps: starting a thread to listen to a port and collect industrial internet data information, receiving data according to the corresponding port, and writing manufacturer information; the data cleaning and persistent processing on the collected industrial internet data information comprises the following steps: classifying and cleaning the received data, classifying different types of data generated by different manufacturers and different devices, and filtering and filling data information fields. The application has the characteristics of high safety and efficient analysis management.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, specifically to an Internet-based management and analysis system. Background Technology

[0002] In recent years, the digitalization and informatization of industries and manufacturing have received increasing attention from countries around the world. Previously, industrial and manufacturing production methods primarily employed automated models based on local area networks (LANs). However, with the development of advanced internet technologies such as big data, artificial intelligence, and blockchain, these production methods are gradually shifting towards intelligent production models that are increasingly integrated with the internet. The Industrial Internet has become the foundation of industrial informatization. In the Industrial Internet control system, the "network" serves as the bridge for industrial data transmission, while "data" drives industrial intelligence. Protecting network and data security is a prerequisite for the Industrial Internet to improve efficiency, reduce costs, and create value. Currently, traditional, independently segmented protection strategies such as firewalls, intrusion detection, vulnerability scanning, and virus sandboxes are insufficient to effectively address these challenges. These systems cannot effectively share information between devices, nor can they provide a multi-faceted view of the vulnerabilities, threats, and attacks presented in the current network environment. Therefore, designing a highly secure and efficient internet-based control and analysis system is essential. Summary of the Invention

[0003] The purpose of this invention is to provide an Internet-based management and analysis system to solve the problems mentioned in the background section.

[0004] To address the aforementioned technical problems, this invention provides the following technical solution: an internet-based control and analysis method, comprising:

[0005] Collect and process industrial internet data;

[0006] The collected industrial internet data is cleaned and persisted.

[0007] To conduct operational analysis, optimization, and management of the Industrial Internet;

[0008] Implement security alarm control and management for the Industrial Internet.

[0009] According to the above technical solution, the collection and processing of industrial internet data information includes:

[0010] The thread listens to the port to collect industrial internet data, receives data according to the corresponding port, and writes it to the manufacturer information.

[0011] According to the above technical solution, the data cleaning and persistence processing of the collected industrial internet data includes:

[0012] The received data is classified and cleaned, and different types of data from different manufacturers and devices are classified, and the data information fields are filtered and populated.

[0013] The data received and cleaned by the data platform is persistently written to Elasticsearch or the message queue Kafka for use by other services.

[0014] According to the above technical solution, the operation analysis and optimization management of the industrial internet includes:

[0015] After receiving the data information that has been cleaned and persisted, it is further normalized to achieve uniformity in structure and content.

[0016] After completing the normalization process of the data information, data flow analysis and optimization are performed on it.

[0017] Debezium is used to monitor changes in the PostgresQL database and send them to the message queue Kafka for control and management.

[0018] According to the above technical solution, the security alarm control and management of the industrial internet includes:

[0019] By creating a display page, the vulnerability information of the current industrial internet will be displayed and processed, and users will be provided with interfaces to query asset vulnerabilities and query historical vulnerabilities;

[0020] Furthermore, the page provides users with an alarm list, alarm statistics, and alarm settings, allowing users to view basic alarm information and create handling tasks, effectively improving user satisfaction.

[0021] According to the above technical solution, an Internet-based control and analysis system includes:

[0022] The data acquisition and processing module is used to acquire and process industrial internet data.

[0023] The optimization management module is used for operational analysis and optimization management of the Industrial Internet.

[0024] The control and management module is used for security alarm control and management in the industrial internet.

[0025] According to the above technical solution, the acquisition and processing module includes:

[0026] The data acquisition and processing module is used for acquiring and processing industrial internet data.

[0027] The data cleaning module is used to clean and process data information.

[0028] The persistence module is used to persist industrial internet data.

[0029] According to the above technical solution, the optimization management module includes:

[0030] The normalization module is used to perform normalization processing of industrial internet data information;

[0031] The data flow module is used for data flow control analysis and processing;

[0032] The management and control module is used for the control and management of database information.

[0033] According to the above technical solution, the control and management module includes:

[0034] The vulnerability management module is used for displaying, querying, and managing vulnerability information in the industrial internet.

[0035] The alarm control module is used to control and process alarm information.

[0036] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: By setting up a data acquisition and processing module, an optimization management module, and a control management module, this invention can make data acquisition more efficient and accurate, reduce the load pressure on a single listening port, effectively improve data cleaning efficiency, and enable records processed by the matching tree to carry detailed data type fields and filter out useless fields in the records, effectively reducing the matching difficulty of each type of data, making analysis and processing more efficient and accurate, and effectively improving the security and stability of the industrial internet. Attached Figure Description

[0037] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0038] Figure 1 This is a flowchart of the Internet-based control and analysis system method provided in Embodiment 1 of the present invention;

[0039] Figure 2 This is a module configuration diagram of the Internet-based management and analysis system provided in Embodiment 2 of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1: Figure 1 This is a flowchart of the Internet-based management and analysis system method provided in Embodiment 1 of the present invention. This embodiment can be applied to a system, and the method can be executed by the Internet-based management and analysis system provided in this embodiment of the present invention. The system consists of multiple software and hardware modules, such as... Figure 1 As shown, the method specifically includes the following steps:

[0042] S101. Collect and process industrial internet data information;

[0043] For example, in this embodiment of the invention, threads are enabled to listen to ports to collect industrial internet data. Data is received according to the corresponding ports and written to the manufacturer information. Since the data collection probes or security protection devices deployed in industrial plants come from different manufacturers, including firewalls, intrusion detection devices, auditing devices, host protection devices, etc., and the devices from which the data come have significant differences and no unified standard, and the data format definitions of each device manufacturer are different, the types and forms of data are complex. Therefore, through this step, multiple threads are enabled to listen to multiple ports, and a listening port corresponding to a specific manufacturer is customized. Data received from different ports is assigned a value from a specific manufacturer. According to the different probe manufacturers, multiple threads are enabled to listen to and collect industrial internet data while receiving data according to the corresponding ports and writing to the manufacturer information. Through this process, the collection of data information can be made more efficient and accurate, the load pressure of a single listening port can be reduced, and the efficiency of subsequent data cleaning can be effectively improved.

[0044] S102. Perform data cleaning and persistence processing on the collected industrial internet data information;

[0045] For example, in this embodiment of the invention, the received data is classified and cleaned. Different types of data from different manufacturers and devices are classified, and the data information fields are filtered and filled. In this step, firstly, the manufacturer is determined according to the manufacturer field of the JSON object. Then, the JSON object is matched with the rules in the device sub-node corresponding to the manufacturer node to determine the device type to which the JSON object belongs. Next, the JSON object is matched with the rules in the version sub-node corresponding to the device node to determine the device version information to which the JSON object belongs. Further, the JSON object is matched with the rules in the data type sub-node of the corresponding device version node. After determining the data type, it is transmitted to a tree-shaped matching rule tree composed of rule nodes. This allows the records processed by the matching tree to carry detailed data type fields and filter out useless fields in the records, effectively reducing the matching difficulty of each type of data and making subsequent analysis and processing more efficient and accurate.

[0046] After receiving and cleaning the data from the data platform, the data is persistently written to Elasticsearch or the Kafka message queue for use by other services. In this step, the data write operation adopts a multi-threaded asynchronous batch approach. Atomic references and concurrency-safe containers are used to avoid multi-threading safety issues. Data in the cache is periodically extracted and written to Elasticsearch in batches. At the same time, relatively small amounts of data are written to the Kafka message queue for use by other services, effectively improving the processing security of the industrial Internet.

[0047] S103. Conduct operational analysis and optimization management of the Industrial Internet;

[0048] For example, in this embodiment of the invention, after receiving data information that has undergone data cleaning and persistence processing, it is further normalized to achieve uniformity in structure and content. In this step, different measures are taken for data normalization based on factors such as the frequency and volume of industrial internet data. For traffic data with relatively frequent reporting and volumes exceeding set thresholds, a scheduled task performs aggregation queries from Elasticsearch based on set time conditions. For asset, alarm, and vulnerability data with volumes less than the set threshold, processing is achieved by subscribing to messages in Kafka. A scheduled task periodically starts a thread to aggregate queries in Elasticsearch based on time conditions. Taking traffic data from device A as an example, aggregation queries are performed based on conditions such as source IP, source port, destination IP, destination port, and time, and the query results are converted into a system-unified traffic format.

[0049] After the data information is normalized, it undergoes data flow analysis and optimization. In this step, for threat data flow, it first determines whether the alarm type will generate a compromise alarm, then generates a unique identifier based on the alarm's multiple basic attributes, merges the events, and if the alarm exists, increments the alarm data; if it does not exist, the alarm is entered into the database and the corresponding asset risk value is calculated. For vulnerability data flow, the data is first verified; if there is no corresponding asset data for the vulnerability, the data is discarded; otherwise, an asset vulnerability association object is created and saved or updated in the database for business retrieval by the web module. This process effectively improves the security and stability of the industrial internet.

[0050] Debezium is used to monitor changes in the PostgresQL database and send them to the message queue Kafka for control and management. In this step, the database is first paginated and retrieved from Elasticsearch using search criteria, and then retrieved from the relational database using the primary key ID obtained from Elasticsearch, thereby effectively improving the database retrieval speed.

[0051] S104. Perform security alarm control and management for the Industrial Internet;

[0052] For example, in this embodiment of the invention, a display page is created to display the vulnerability information that currently exists in the industrial internet, and to provide users with interfaces for querying asset vulnerabilities and querying historical vulnerabilities.

[0053] Furthermore, the page provides users with an alarm list, alarm statistics, and alarm settings, allowing users to view basic alarm information and create handling tasks, effectively improving user satisfaction.

[0054] Example 2: Example 2 of the present invention provides an Internet-based management and analysis system. Figure 2 This is a schematic diagram of the module structure of the Internet-based management and analysis system provided in Embodiment 2. Figure 2 As shown, the system includes:

[0055] The data acquisition and processing module is used to acquire and process industrial internet data.

[0056] The optimization management module is used for operational analysis and optimization management of the Industrial Internet.

[0057] The control and management module is used for security alarm control and management in the industrial internet.

[0058] In some embodiments of the present invention, the acquisition and processing module includes:

[0059] The data acquisition and processing module is used for acquiring and processing industrial internet data.

[0060] The data cleaning module is used to clean and process data information.

[0061] The persistence module is used to persist industrial internet data.

[0062] In some embodiments of the present invention, the optimization management module includes:

[0063] The normalization module is used to perform normalization processing of industrial internet data information;

[0064] The data flow module is used for data flow control analysis and processing;

[0065] The management and control module is used for the control and management of database information.

[0066] In some embodiments of the present invention, the control and management module includes:

[0067] The vulnerability management module is used for displaying, querying, and managing vulnerability information in the industrial internet.

[0068] The alarm control module is used to control and process alarm information.

[0069] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0070] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A control and analysis method based on the Internet, characterized in that: include: S101. Collecting and processing industrial internet data; including: The thread listens to the port to collect industrial internet data, receives data according to the corresponding port, and writes it to the manufacturer information. Multiple threads are started to listen to multiple ports. The listening ports corresponding to the preset manufacturers are customized, and the data received from different ports are assigned the preset manufacturer values. Depending on the probe manufacturer, multiple threads are started to listen separately to collect industrial Internet data information and receive data according to the corresponding ports, and write the manufacturer information. S102. Perform data cleaning and persistence processing on the collected industrial internet data; including: S1021. Classify and clean the received data, classify different types of data from different manufacturers and different devices, and filter and populate the data information fields. First, after determining the manufacturer based on the manufacturer field of the JSON object, the JSON object is matched with the rules in the device sub-node corresponding to the manufacturer node to determine the device type to which the JSON object belongs. Then, the JSON object is matched with the rules in the version sub-node corresponding to the device node to determine the device version information to which the JSON object belongs. Next, the JSON object is matched with the rules in the data type sub-node of the corresponding device version node. After determining the data type, it is transmitted to a tree-shaped matching rule tree composed of rule nodes. This allows the records processed by the matching tree to carry detailed data type fields and filters out useless fields in the records to reduce the matching difficulty of each type of data. S1022. Persistently write the data received and cleaned by the data platform to Elasticsearch or message queue Kafka for use by other services; The write operations to the database adopt a multi-threaded asynchronous batch approach, using atomic references and concurrency-safe containers to avoid multi-threading safety issues, and periodically extracting data from the cache and writing it to Elasticsearch in batches, while writing relatively small amounts of data to the message queue Kafka for use by other services; S103. Conduct operational analysis and optimization management of the Industrial Internet; including: S1031. After receiving the data information that has been cleaned and persisted, further normalize it to achieve uniformity in structure and content; Based on the frequency and volume of industrial internet data, different measures are taken for data normalization. For traffic data with relatively frequent reporting and volumes exceeding set thresholds, scheduled tasks perform aggregation queries from Elasticsearch based on set time conditions. For assets, alarms, and vulnerabilities with volumes less than set thresholds, messages are subscribed to in Kafka for processing. A scheduled task is also used to periodically start a thread to aggregate queries in the corresponding index in Elasticsearch based on time conditions. Aggregation queries are performed by source IP, source port, destination IP, destination port, and time, and the query results are converted into a unified traffic format for the system. S1032. After completing the normalization process of the data information, perform data flow analysis and optimization processing on it; For threat data streams, first determine whether the type of alarm will generate a compromise alarm, then generate a unique identifier through multiple basic attributes of the alarm, merge the events, if the alarm exists, then increment the alarm data of the alarm, if it does not exist, then enter the alarm into the database and calculate the asset risk value corresponding to the alarm. For vulnerability data streams, first verify the data, that is, if there is no corresponding asset data for the vulnerability, then discard the data, otherwise create an asset vulnerability association object and save or update it to the database for business retrieval by the web module; S1033. Debezium is used to monitor changes in the PostgresQL database and send them to the message queue Kafka for control and management. First, a paginated search is performed in Elasticsearch based on search criteria. Then, the primary key ID obtained from Elasticsearch is used to retrieve data from a relational database to improve database retrieval speed. S104. Conduct security alarm control and management for the Industrial Internet.

2. The Internet-based control and analysis method according to claim 1, characterized in that: The aforementioned security alarm control and management for the Industrial Internet includes: By creating a display page, the vulnerability information of the current industrial internet will be displayed and processed, and users will be provided with interfaces to query asset vulnerabilities and query historical vulnerabilities; Furthermore, the page provides users with an alarm list, alarm statistics, and alarm settings, allowing users to view basic alarm information and create handling tasks, effectively improving user satisfaction.

Images