Network operation and maintenance data processing method, system, medium and equipment

Abstract

The invention provides a network operation and maintenance data processing method and system, a medium and equipment, and the method comprises the steps: obtaining initial operation and maintenance data, carrying out the preprocessing of the initial operation and maintenance data, and obtaining to-be-analyzed operation and maintenance data; performing screening processing on the to-be-analyzed operation and maintenance data by utilizing a pre-constructed containerized engine layer to obtain target operation and maintenance data; the target operation and maintenance data is effective data which attacks network operation and maintenance; thus, after the operation and maintenance data to be analyzed are screened by using the containerized engine layer, effective data generating attacks can be obtained, real attack alarm data are prevented from being submerged, and then the quality of network operation and maintenance can be improved; and after a part of invalid data is screened out, the resource burden of data analysis can be reduced, and the efficiency of network operation and maintenance can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a network operation and maintenance data processing method, system, medium and equipment. Background Art

[0002] With the rapid development of information technology, network environments are becoming increasingly complex, and the log and alarm information generated by third-party devices is becoming more diverse and massive.

[0003] In the currently commonly used network operation and maintenance methods, the network operation and maintenance system will receive and store all attack alarm data and other behavioral data, resulting in a large amount of operation and maintenance data stored in the system. When the network operation and maintenance system performs alarm analysis, the alarm data obtained is complex and changeable, which may cause real attack alarms to be overwhelmed, thereby causing delayed responses to real attacks, affecting the network's operation and maintenance efficiency and quality.

[0004] Therefore, how to improve the efficiency and quality of network operation and maintenance is an issue that needs to be urgently addressed. Summary of the Invention

[0005] In response to the problems existing in the prior art, the embodiments of the present invention provide a network operation and maintenance data processing method, system, medium and equipment to solve or partially solve the technical problem in the prior art that network operation and maintenance efficiency and quality cannot be ensured during network operation and maintenance.

[0006] A first aspect of the present invention provides a method for processing network operation and maintenance data, the method comprising:

[0007] Acquiring initial operation and maintenance data, and preprocessing the initial operation and maintenance data to obtain operation and maintenance data to be analyzed;

[0008] The pre-built containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that attacks network operation and maintenance.

[0009] In the above solution, before using the pre-built containerized engine layer to filter and process the operation and maintenance data to be analyzed, the method further includes:

[0010] Determine each containerization engine included in the containerization engine layer, and build an image file under a different operating environment for each containerization engine;

[0011] Obtaining the current operating environment type, and calling the engine image file corresponding to each containerized engine based on the current operating environment type;

[0012] The containerized orchestration tool is used to orchestrate the engine image files corresponding to each containerized engine to obtain a containerized engine layer; wherein,

[0013] The containerized engine layer includes an intelligence engine, an asset identification engine, an application identification engine, a correlation analysis engine, and a vulnerability scanning engine.

[0014] In the above solution, the pre-built containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain the target operation and maintenance data, including:

[0015] When the operation and maintenance data to be analyzed is behavior activity log data, the correlation analysis engine in the containerized engine layer is used to obtain historical behavior activity log data under normal system operation, and a pre-created baseline model is trained based on the historical behavior activity log data; the baseline model is a machine learning neural network;

[0016] Training the baseline model and generating device behavior baseline data using the trained baseline model;

[0017] Obtaining device context data of the operation and maintenance data to be analyzed, and determining attack event chain data from the operation and maintenance data to be analyzed based on a preset analysis strategy and the device context data;

[0018] The attack event chain data is compared with the device behavior baseline data to obtain the target operation and maintenance data.

[0019] In the above solution, the step of comparing the attack event chain data with the device behavior baseline data to obtain the target operation and maintenance data includes:

[0020] If it is determined that the attack event chain data is consistent with the device behavior baseline data, the attack event chain data is deleted from the operation and maintenance data to be analyzed, and the remaining operation and maintenance data to be analyzed is determined as the target operation and maintenance data.

[0021] In the above solution, the step of comparing the attack event chain data with the device behavior baseline data to obtain the target operation and maintenance data includes:

[0022] If it is determined that the attack event chain data does not conform to the device behavior baseline data, the attack event chain data is added as attack alarm data to the attack alarm list, and the attack alarm list is determined as the target operation and maintenance data.

[0023] In the above solution, the use of the containerized engine layer to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data includes:

[0024] When the operation and maintenance data to be analyzed is alarm data, the intelligence engine in the containerized engine layer is used to receive current alarm data and match the current alarm data with historical alarm data stored in the containerized engine layer;

[0025] If it is determined that the current alarm data can be successfully matched with the historical alarm data stored in the self-stored data, the current alarm data is persisted in the attack alarm list, and the attack alarm list is determined as the target operation and maintenance data.

[0026] In the above solution, after the containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data, the method further includes:

[0027] Storing the target operation and maintenance data in a pre-built containerized storage layer;

[0028] When service request data is received, the target operation and maintenance data is output to the service function module corresponding to the service request data through the containerized storage layer to generate service response data.

[0029] A second aspect of the present invention provides a network operation and maintenance data processing system, the system comprising:

[0030] The alarm heterogeneous processing layer is used to obtain initial operation and maintenance data, pre-process the initial operation and maintenance data, and obtain the operation and maintenance data to be analyzed;

[0031] The containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that attacks network operation and maintenance.

[0032] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0033] According to a fourth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the methods described in the first aspect are implemented.

[0034] The present invention provides a network operation and maintenance data processing method, system, medium and equipment, the method comprising: obtaining initial operation and maintenance data, pre-processing the initial operation and maintenance data to obtain operation and maintenance data to be analyzed; using a pre-built containerized engine layer to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that generates attacks on network operation and maintenance; in this way, after using the containerized engine layer to filter and process the operation and maintenance data to be analyzed, valid data that generates attacks can be obtained, thereby avoiding the real attack alarm data from being submerged, thereby improving the quality of network operation and maintenance; and after filtering out a portion of invalid data, the resource burden on data analysis can be reduced, thereby improving the efficiency of network operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0036] Figure 1 A schematic diagram of a process flow of a method for processing network operation and maintenance data according to an embodiment of the present invention is shown;

[0037] Figure 2 A schematic diagram of the structure of a network operation and maintenance data processing system according to an embodiment of the present invention is shown;

[0038] Figure 3 A schematic diagram of the device structure of a hardware operating environment involved in a network operation and maintenance method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0039] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0040] The present invention provides a network operation and maintenance data processing method, which can be applied to the scenario of network operation and maintenance of third-party equipment in the network operation and maintenance system, such as Figure 1 As shown, the method mainly includes the following steps:

[0041] S110 , obtaining initial operation and maintenance data, and preprocessing the initial operation and maintenance data to obtain operation and maintenance data to be analyzed.

[0042] With the rapid development of information technology and the increasing complexity of network environments, the vast amount of log and alarm information generated by third-party devices is becoming increasingly diverse and massive. Traditional alarm analysis systems face challenges in information processing, format standardization, and accuracy. Understandably, the operational data to be analyzed comes from different functional modules of third-party devices, using different data formats and data transmission protocols. The vast amount of log and alarm information generated by third-party devices is becoming increasingly diverse and massive. When network operations systems perform alarm analysis, the resulting attack alarm data is complex and highly variable, potentially causing real attack alarms to be buried among a large number of false alarms, leading to delayed responses to real attacks.

[0043] In currently common network operation and maintenance methods, the network operation and maintenance system receives and stores all attack alert data and other behavioral data. This results in a large amount of operation and maintenance data stored in the system, increases the burden of analyzing attack alert data, and leads to low network operation and maintenance efficiency. The present invention uses the network operation and maintenance system to analyze the behavioral activities of third-party devices, thereby reducing the amount of operation and maintenance data stored in the system, lowering network operation and maintenance costs, and improving network operation and maintenance efficiency and quality.

[0044] First, we need to obtain initial O&M data. Initial O&M data is unprocessed, collected directly from third-party devices. For example, this data can include raw alarm and log data regarding the status, performance, and faults of third-party devices. It can also include attack alarm data generated by third-party devices, or data from third-party devices that record process events related to user behavior within IT systems or applications. These data are not limited to network security devices or system monitoring equipment.

[0045] Since the initial operation and maintenance data of each third-party device has different formats and complex contents, it is necessary to pre-process the initial operation and maintenance data in order to smoothly carry out subsequent alarm analysis and processing.

[0046] When preprocessing the initial operation and maintenance data, you can first clean the initial operation and maintenance data. For missing data, you can use the mean method or interpolation to fill the missing data; for abnormal data, you can directly delete the abnormal data; then standardize the format of the cleaned data so that the cleaned data has the same data format; then deduplicate the data after format standardization and delete the duplicate records; finally, perform timestamp processing on the data and convert the data into a standardized time format (usually a timestamp) for easy storage, calculation and analysis.

[0047] After preprocessing the initial operation and maintenance data, the operation and maintenance data to be analyzed is obtained.

[0048] S111, using a pre-built containerized engine layer to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that attacks network operation and maintenance.

[0049] In order to reduce the storage volume of operation and maintenance data in the network operation and maintenance system and improve the effectiveness of the data, the present invention needs to build a containerized engine layer, and use the containerized engine layer to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that attacks network operation and maintenance.

[0050] In one embodiment, before using the pre-built containerized engine layer to filter and process the operation and maintenance data to be analyzed, the method further includes:

[0051] Determine the containerized engines included in the containerized engine layer and build image files for each containerized engine in different operating environments;

[0052] Get the current running environment type and call the engine image file corresponding to each containerized engine based on the current running environment type;

[0053] Use the containerized orchestration tool to orchestrate the engine image files corresponding to each containerized engine to obtain the containerized engine layer;

[0054] The containerized engine layer includes an intelligence engine, an asset identification engine, an application identification engine, a correlation analysis engine, and a vulnerability scanning engine.

[0055] Specifically, different types of operating environments correspond to different image files of each engine, so it is necessary to pre-produce the engine image file corresponding to each containerized engine for different types of operating environments.

[0056] When creating a corresponding engine image file for each containerized engine, the container file DockerFile can be used to write an image configuration file for each containerized engine; when it is determined that the order and integrity of the image configuration file are intact, the container build command is received and the corresponding engine image file is built according to the image configuration file.

[0057] After the engine image file is built, you can also call the image list command docker images to check whether the engine image file is generated successfully.

[0058] After the engine image file is successfully generated, if the current operating environment type calls the engine image file corresponding to each containerized engine;

[0059] The engine image file corresponding to each containerized engine is orchestrated using a containerized orchestration tool to obtain a containerized engine layer.

[0060] Among them, the containerization engine layer includes multiple containerization engines, namely: intelligence engine, asset identification engine, application identification engine, correlation analysis engine and vulnerability scanning engine; then the operation and maintenance data to be analyzed can be containerized through the constructed containerization engine layer, reducing the storage volume of operation and maintenance data in the network operation and maintenance system, thereby improving network operation and maintenance efficiency.

[0061] In one embodiment, the pre-built containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data, including:

[0062] When the operation and maintenance data to be analyzed is behavioral activity log data, the correlation analysis engine in the containerized engine layer is used to obtain the behavioral activity log data under normal system operation to train a pre-created baseline model; the baseline model is a machine learning neural network;

[0063] Train the baseline model and use the trained baseline model to generate device behavior baseline data;

[0064] Obtain device context data of the operation and maintenance data to be analyzed, and determine attack event chain data from the operation and maintenance data to be analyzed based on a preset analysis strategy and the device context data;

[0065] Compare the attack event chain data with the device behavior baseline data to obtain target operation and maintenance data.

[0066] In one embodiment, attack event chain data is compared with device behavior baseline data to obtain target operation and maintenance data, including:

[0067] If it is determined that the attack event chain data is consistent with the device behavior baseline data, the attack event chain data is deleted from the operation and maintenance data to be analyzed, and the remaining operation and maintenance data to be analyzed is determined as the target operation and maintenance data.

[0068] In one embodiment, attack event chain data is compared with device behavior baseline data to obtain target operation and maintenance data, including:

[0069] If it is determined that the attack event chain data does not conform to the device behavior baseline data, the attack event chain data is added to the attack alarm list as attack alarm data, and the attack alarm list is determined as target operation and maintenance data.

[0070] Specifically, when the type of operation and maintenance data to be analyzed is different, the containerization engine used is also different. When the operation and maintenance data to be analyzed is behavior activity logs, the containerization engine used can be a correlation analysis engine.

[0071] In order to better identify whether the behavior activity log is a normal behavior activity log, this embodiment needs to obtain historical behavior activity log data under normal operation in advance, and use the historical behavior activity log data as sample data to train a pre-created baseline model. When it is determined that the baseline model training is qualified, the baseline model is used to generate device behavior baseline data.

[0072] Device behavior baseline data can be understood as reference data used to evaluate device behavior. For example, this data may include the device's normal operating mode, normal traffic pattern, and normal performance parameters. When a device's current behavior data deviates from this baseline, it indicates a potential risk. This risk can be promptly identified and appropriate measures can be taken to improve the security of system operations.

[0073] However, when a network or system is attacked, third-party devices will generate a large number of alarm messages. However, these alarm messages are often fragmented and isolated, making it difficult to directly form a complete attack chain event. Therefore, in order to accurately determine the actual attack data and improve the quality of network operation and maintenance, this embodiment also needs to obtain the device context data of the operation and maintenance data to be analyzed. The device context data records the behavior data of the device before and after a certain point in time. Therefore, the current time point corresponding to the current behavior activity log can be used to obtain the behavior activity log for a period of time before and after the current time point; then the behavior activity log for a period of time before and after the current time point is the device context data.

[0074] Based on the preset analysis strategy and device context data, the attack event chain data is determined from the operation and maintenance data to be analyzed; the attack event chain data is then compared with the device behavior baseline data to obtain the target operation and maintenance data.

[0075] For example, in a network access scenario, if a device with an IP address of xxxx normally only connects and works with devices 1.1.1.1 and 2.2.2.2 (baseline data), this is the baseline obtained through learning. However, one day, xxxx suddenly makes a large number of connections with devices 3.3.3.3, indicating that the device is likely an anomaly. To further determine whether the device has been attacked, additional judgment is required based on device context data. For example, if the device context data determines that the device has previously only connected to devices 1.1.1.1 and 2.2.2.2 and has not connected to any other devices, then the connection between the device and device 3.3.3.3 is considered abnormal behavior, generating an alarm. The alarm data generated by the connection between the device and device 3.3.3.3 is then identified as the target operation and maintenance data.

[0076] For example, in a network attack scenario, network attacks are divided into multiple attack stages, and each attack stage has different attack methods. At this time, the alarms in different attack stages can be directly linked together according to the analysis strategy to form attack event chain data.

[0077] For example: 1. During the reconnaissance phase, a common attack method is port scanning to find available IP ports; 2. Then, a brute force attack is used to attempt to log into the device; 3. If the login is successful, the attacker attempts to elevate their privileges to access the device; 4. Then, a series of attacks are carried out, such as virus implantation, lateral virus transmission, and using the device as a mining machine.

[0078] There will be a single alarm in each of the above links. This embodiment uses the analysis strategy to assemble these attack alarms, and then analyzes the assembled attack alarm data (matches it with the intelligence data). If the match is successful, the assembled attack alarm data will be used as the attack event chain data.

[0079] It should be noted that in a network attack scenario, only the operation and maintenance data to be analyzed may be needed, and the device context data is not needed to determine the attack event chain data.

[0080] When it is determined that the attack event chain data does not conform to the device behavior baseline data, it means that the attack alarm corresponding to the attack event chain data is a real and valid attack alarm. Then the attack event chain data is added to the attack alarm list as a higher-level attack alarm data, and the attack alarm list is determined as the target operation and maintenance data.

[0081] If the attack event chain data is determined to match the device behavior baseline data, the attack alarm corresponding to the attack event chain data is a false positive. The attack event chain data is removed from the O&M data to be analyzed, and the remaining O&M data to be analyzed is designated as the target O&M data. It should be noted that the total alarm data here refers to the attack alarm data analyzed by the O&M system, not the attack alarm data generated by third-party devices.

[0082] In this embodiment, the operation and maintenance data to be analyzed is input into a correlation analysis engine, which is used to generate device behavior baseline data based on normal behavior activity log data. Based on pre-acquired device context data and a preset analysis strategy, attack event chain data is determined from the operation and maintenance data to be analyzed. Based on the device behavior baseline data, it is determined whether the attack event chain data conforms to the behavior baseline corresponding to the device behavior baseline data. If the attack event chain data conforms to the behavior baseline, the attack event chain data is filtered from the operation and maintenance data to be analyzed to obtain target operation and maintenance data. If the attack event chain data does not conform to the behavior baseline, the attack alarm data corresponding to the attack event chain data is persisted to obtain the corresponding target operation and maintenance data. The present invention generates device behavior baseline data through a correlation analysis engine. When identifying potential attack events, comparison can be performed based on these baseline data, thereby more accurately determining which events are abnormal or malicious, thereby improving the quality of network operation and maintenance.

[0083] In another implementation of this embodiment, the containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data, including:

[0084] When the operation and maintenance data to be analyzed is alarm data, the intelligence engine in the containerized engine layer is used to receive the current alarm data and match the current alarm data with the historical alarm data stored in the system.

[0085] If it is determined that the current alarm data can be successfully matched with the historical alarm data stored in its own intelligence library, the current alarm data will be persisted in the attack alarm list, and the attack alarm list target operation and maintenance data will be.

[0086] For example, if the intelligence database contains information about the IP address xxx1, and the current alarm data also contains the IP address xxx1, then the match is successful. If the intelligence database contains information about the domain name www.ccc.xx.cm, and the current alarm data also contains the domain name www.ccc.xx.cm, then the match is successful.

[0087] Specifically, when the operation and maintenance data to be analyzed is alarm data, the containerized engine used can be an intelligence engine and a correlation analysis engine.

[0088] It should be noted that current alarm data refers to attack alarm data determined by the third-party device, including both real and false alarms. This embodiment utilizes an intelligence engine to match received current alarm data with its stored historical alarm data. A successful match indicates that the current alarm data is likely associated with a known security risk and carries a high potential risk. Therefore, the current alarm data needs to be persisted in an attack alarm list, which is then linked to operational data. The intelligence engine includes an attack alarm list, which stores historical alarm data.

[0089] Conversely, if the current alarm data fails to match the stored historical alarm data, it indicates that the current alarm data may be an unknown attack threat. The correlation analysis engine will then be used to further analyze the current alarm data to determine whether it is a real attack alarm. Ultimately, the target operation and maintenance data will be obtained based on the correlation analysis results. The specific implementation of using the correlation analysis engine to analyze the current alarm data can be found in the specific implementation steps for using the correlation analysis engine to analyze behavioral activity log data described above, and will not be repeated here.

[0090] In this implementation, current alarm data is input into the intelligence engine, which then matches it with its own stored historical alarm data. If the current alarm data successfully matches the historical alarm data, the current alarm data is persisted and the corresponding target operation and maintenance data is obtained. If the current alarm data fails to match the historical alarm data, the current alarm data is input into the correlation analysis engine for correlation data analysis to obtain the corresponding target operation and maintenance data. This can significantly improve the efficiency and accuracy of network security protection and reduce false alarms.

[0091] In one embodiment, after filtering and processing the operation and maintenance data to be analyzed using the containerized engine layer to obtain target operation and maintenance data, the method further includes:

[0092] Store target operation and maintenance data in a pre-built containerized storage layer;

[0093] When service request data is received, the target operation and maintenance data is output to the service function module corresponding to the service request data through the containerized storage layer to generate service response data.

[0094] Furthermore, the containerized storage layer can include the relational persistence database MySQL ClickHouse, the index database Elasticsearch, the distributed cache database Redis, and the graph database Nebula. The persistence database is used to store long-term data and historical records, the index database supports fast data retrieval and query, the cache database provides temporary data storage and fast access, and the graph database is used to store and query data relationships and network topology information. By deploying these different database types in the containerized storage layer, appropriate storage and query strategies can be selected based on data characteristics and business needs, achieving efficient data utilization and management. Containerization also makes these database services more flexible, scalable, and easy to manage.

[0095] Furthermore, the target operation and maintenance data output by the containerized storage layer is input into the containerized service layer, and the corresponding service function modules in the containerized service layer generate service response data to provide various services. The containerized service layer can include functional modules such as the system center module, alarm center module, asset center module, response linkage module, operation center module, and monitoring center module.

[0096] The system center module is responsible for the management and configuration of the entire system; the alarm center module is used to monitor and process the alarm information generated by the system in real time; the asset center module is used to manage and track various asset information in the network; the response linkage module is used to automatically respond to and process security incidents; the operation center module is used to provide network operation and management functions; and the monitoring center module is used to monitor the operating status and performance indicators of the system in real time. In addition, when service request data is received, the service request data usually contains the type of data and / or data ID, service type and other information that needs to be accessed. After receiving the target operation and maintenance data, each functional module will process the data according to its business logic and rules and generate corresponding service response data. The processing process may include operations such as data analysis, aggregation, conversion or formatting, and then output service response data, which will be fed back to the service requester.

[0097] For example, the alarm center module will generate alarm notifications or alarm reports based on the target operation and maintenance data, and the operation center will generate network operation and maintenance reports or network operation and maintenance suggestions based on the alarm notifications or alarm high bursts.

[0098] In this embodiment, the target operation and maintenance data is stored in a pre-built containerized storage layer; when service request data is received, the target operation and maintenance data is output to the service function module corresponding to the service request data through the containerized storage layer to generate service response data. The present invention realizes efficient processing and response to service requests through the collaborative work of the containerized storage layer and the containerized service layer. This architecture not only improves the flexibility and scalability of the system, but also makes the retrieval and use of operation and maintenance data simpler and more efficient. At the same time, the decoupling between the various functional modules also makes the system easier to maintain and expand.

[0099] Based on the same inventive concept as the above embodiment, the present invention also provides a network operation and maintenance data processing system, such as Figure 2 As shown, the system includes:

[0100] The alarm heterogeneous processing layer 1 is used to obtain initial operation and maintenance data, pre-process the initial operation and maintenance data, and obtain the operation and maintenance data to be analyzed;

[0101] The containerized engine layer 2 is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that attacks network operation and maintenance.

[0102] Specifically, the Alarm Heterogeneous Processing Layer 1 needs to obtain initial O&M data, which is unprocessed, relevant O&M data collected directly from third-party devices. For example, this data may include raw alarm data and log data regarding the status, performance, and faults of third-party devices. It can also include attack alarm data generated by third-party devices, or data from third-party devices that record procedural events related to user behavior within IT systems or applications. These data are not limited to network security devices or system monitoring equipment.

[0103] Since the initial operation and maintenance data of each third-party device has different formats and complex contents, it is necessary to pre-process the initial operation and maintenance data in order to smoothly carry out subsequent alarm analysis and processing.

[0104] When preprocessing the initial operation and maintenance data, you can first clean the initial operation and maintenance data. For missing data, you can use the mean method or interpolation to fill the missing data; for abnormal data, you can directly delete the abnormal data; then standardize the format of the cleaned data so that the cleaned data has the same data format; then deduplicate the data after format standardization and delete the duplicate records; finally, perform timestamp processing on the data and convert the data into a standardized time format (usually a timestamp) for easy storage, calculation and analysis.

[0105] After preprocessing the initial operation and maintenance data, the operation and maintenance data to be analyzed is obtained.

[0106] refer to Figure 2 The containerized engine layer 2 includes: an intelligence engine, an asset identification engine, an application identification engine, a correlation analysis engine, and a vulnerability scanning engine; the operation and maintenance data to be analyzed can be containerized through the containerized engine layer 2, thereby reducing the storage amount of operation and maintenance data in the network operation and maintenance system, thereby improving network operation and maintenance efficiency.

[0107] The containerized engine layer 2 is specifically configured to: when the operation and maintenance data to be analyzed is behavior activity log data, use the correlation analysis engine in the containerized engine layer to obtain the behavior activity log data under normal system operation to train a pre-created baseline model; the baseline model is a machine learning neural network;

[0108] Train the baseline model and use the trained baseline model to generate device behavior baseline data;

[0109] Obtain device context data of the operation and maintenance data to be analyzed, and determine attack event chain data from the operation and maintenance data to be analyzed based on a preset analysis strategy and the device context data;

[0110] Compare the attack event chain data with the device behavior baseline data to obtain target operation and maintenance data.

[0111] In one embodiment, attack event chain data is compared with device behavior baseline data to obtain target operation and maintenance data, including:

[0112] If it is determined that the attack event chain data is consistent with the device behavior baseline data, the attack event chain data is deleted from the operation and maintenance data to be analyzed, and the remaining operation and maintenance data to be analyzed is determined as the target operation and maintenance data.

[0113] In one embodiment, attack event chain data is compared with device behavior baseline data to obtain target operation and maintenance data, including:

[0114] If it is determined that the attack event chain data does not conform to the device behavior baseline data, then the attack alarm data corresponding to the attack event chain data is obtained;

[0115] The attack alarm data is added to the attack alarm list, and the attack alarm list is determined as the target operation and maintenance data.

[0116] The containerized engine layer 2 is specifically used for:

[0117] When the operation and maintenance data to be analyzed is alarm data, the intelligence engine in the containerized engine layer is used to receive the current alarm data and match the current alarm data with the historical alarm data stored in the system.

[0118] If it is determined that the current alarm data can be successfully matched with the historical alarm data stored in itself, the current alarm data will be persisted in the attack alarm list, and the attack alarm list target operation and maintenance data will be added.

[0119] Further, refer to Figure 2 The system also includes a containerized storage layer 3, which can include the relational persistent database MySQL Clickhouse, the index database Elasticsearch, the distributed cache database Redis, and the graph database Nebula. The persistent database is used to store long-term data and historical records, the index database supports fast data retrieval and query, the cache database provides temporary data storage and fast access, and the graph database is used to store and query data relationships and network topology information. By deploying these different types of databases in the containerized storage layer, appropriate storage and query strategies can be selected based on data characteristics and business needs, achieving efficient data utilization and management. At the same time, containerization technology also makes these database services more flexible, scalable, and easy to manage.

[0120] refer to Figure 2 The system also includes: a containerized service layer 4, which can include functional modules such as a system center module, an alarm center module, an asset center module, a response linkage module, an operation center module and a monitoring center module.

[0121] The system center module is responsible for the management and configuration of the entire system; the alarm center module is used to monitor and process the alarm information generated by the system in real time; the asset center module is used to manage and track various asset information in the network; the response linkage module is used to automatically respond to and process security incidents; the operation center module is used to provide network operation and management functions; and the monitoring center module is used to monitor the operating status and performance indicators of the system in real time. In addition, when service request data is received, the service request data usually contains the type of data and / or data ID, service type and other information that needs to be accessed. After receiving the target operation and maintenance data, each functional module will process the data according to its business logic and rules and generate corresponding service response data. The processing process may include operations such as data analysis, aggregation, conversion or formatting, and then output service response data, which will be fed back to the service requester.

[0122] For example, the alarm center module will generate alarm notifications or alarm reports based on the target operation and maintenance data, and the operation center will generate network operation and maintenance reports or network operation and maintenance suggestions based on the alarm notifications or alarm high bursts.

[0123] In this embodiment, after using the containerized engine layer to filter and process the operation and maintenance data to be analyzed, valid data that generates an attack can be obtained, thereby avoiding the real attack alarm data from being overwhelmed, thereby improving the quality of network operation and maintenance; and after filtering out some invalid data, the resource burden on data analysis can be reduced, thereby improving the efficiency of network operation and maintenance.

[0124] The present invention provides a network operation and maintenance device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the network operation and maintenance method in the above-mentioned embodiment one.

[0125] Reference below Figure 3 , which shows a schematic diagram of the structure of a network operation and maintenance device suitable for implementing an embodiment of the present invention. The network operation and maintenance device in the embodiment of the present invention may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The network operation and maintenance device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0126] like Figure 3As shown, the network operation and maintenance equipment may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the network operation and maintenance equipment. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the network operation and maintenance device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a network operation and maintenance device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems can be implemented or provided instead.

[0127] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present invention are performed.

[0128] The network operation and maintenance device provided by the present invention, employing the network operation and maintenance method of the above-described embodiment, can solve the technical problems of network operation and maintenance. Compared with the prior art, the beneficial effects of the network operation and maintenance device provided by the present invention are the same as those of the network operation and maintenance method provided by the above-described embodiment. Other technical features of the network operation and maintenance device are the same as those disclosed in the above-described embodiment and are not further described here.

[0129] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages:

[0130] The present invention provides a network operation and maintenance data processing method, system, medium and equipment, the method comprising: obtaining initial operation and maintenance data, pre-processing the initial operation and maintenance data to obtain operation and maintenance data to be analyzed; constructing a containerized engine layer, and using the containerized engine layer to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that generates attacks on network operation and maintenance; in this way, after using the containerized engine layer to filter and process the operation and maintenance data to be analyzed, valid data that generates attacks can be obtained, thereby avoiding the real attack alarm data from being submerged, thereby improving the quality of network operation and maintenance; and after filtering out a portion of invalid data, the resource burden on data analysis can be reduced, thereby improving the efficiency of network operation and maintenance.

[0131] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general-purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing this type of system. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0132] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0133] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0134] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0135] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0136] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the gateway, proxy server, or system according to an embodiment of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0137] It should be noted that the above embodiments illustrate rather than limit the invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0138] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A network operation and maintenance data processing method, characterized in that: The method comprises: Acquiring initial operation and maintenance data, and preprocessing the initial operation and maintenance data to obtain operation and maintenance data to be analyzed; The pre-built containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that attacks network operation and maintenance.

2. The method according to claim 1, wherein Before using the pre-built containerized engine layer to filter and process the operation and maintenance data to be analyzed, the method further includes: Determine each containerization engine included in the containerization engine layer, and build an image file under a different operating environment for each containerization engine; Obtaining the current operating environment type, and calling the engine image file corresponding to each containerized engine based on the current operating environment type; The containerized orchestration tool is used to orchestrate the engine image files corresponding to each containerized engine to obtain a containerized engine layer; wherein, The containerized engine layer includes an intelligence engine, an asset identification engine, an application identification engine, a correlation analysis engine, and a vulnerability scanning engine.

3. The method according to claim 1, wherein The pre-built containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data, including: When the operation and maintenance data to be analyzed is behavior activity log data, the correlation analysis engine in the containerized engine layer is used to obtain historical behavior activity log data under normal system operation, and a pre-created baseline model is trained based on the historical behavior activity log data; the baseline model is a machine learning neural network; Training the baseline model and generating device behavior baseline data using the trained baseline model; Obtaining device context data of the operation and maintenance data to be analyzed, and determining attack event chain data from the operation and maintenance data to be analyzed based on a preset analysis strategy and the device context data; The attack event chain data is compared with the device behavior baseline data to obtain the target operation and maintenance data.

4. The method according to claim 3, wherein The comparing the attack event chain data with the device behavior baseline data to obtain the target operation and maintenance data includes: If it is determined that the attack event chain data is consistent with the device behavior baseline data, the attack event chain data is deleted from the operation and maintenance data to be analyzed, and the remaining operation and maintenance data to be analyzed is determined as the target operation and maintenance data.

5. The method according to claim 3, wherein: The comparing the attack event chain data with the device behavior baseline data to obtain the target operation and maintenance data includes: If it is determined that the attack event chain data does not conform to the device behavior baseline data, the attack event chain data is added as attack alarm data to the attack alarm list, and the attack alarm list is determined as the target operation and maintenance data.

6. The method according to claim 1, wherein The using the containerized engine layer to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data includes: When the operation and maintenance data to be analyzed is alarm data, the intelligence engine in the containerized engine layer is used to receive current alarm data and match the current alarm data with historical alarm data stored in the containerized engine layer; If it is determined that the current alarm data can be successfully matched with the historical alarm data stored in the self-stored data, the current alarm data is persisted in the attack alarm list, and the attack alarm list is determined as the target operation and maintenance data.

7. The method according to claim 1, wherein After the containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data, the method further includes: Storing the target operation and maintenance data in a pre-built containerized storage layer; When service request data is received, the target operation and maintenance data is output to the service function module corresponding to the service request data through the containerized storage layer to generate service response data.

8. A network operation and maintenance data processing system, characterized in that: The system comprises: The alarm heterogeneous processing layer is used to obtain initial operation and maintenance data, pre-process the initial operation and maintenance data, and obtain the operation and maintenance data to be analyzed; The containerized engine layer is used to filter and process the operation and maintenance data to be analyzed to obtain target operation and maintenance data; the target operation and maintenance data is valid data that attacks network operation and maintenance.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images