Data processing method and device, electronic equipment, storage medium, security detection platform and computer program product

Abstract

The invention provides a data processing method and device, electronic equipment, a storage medium, a security detection platform and a computer program product, and relates to the technical field of data processing, and the method comprises the following steps: monitoring a resident monitoring node in a processing flow of to-be-processed data based on a first monitoring parameter, the monitoring nodes are used for monitoring anomalies in the resident monitoring nodes; wherein the first monitoring parameter comprises a target monitoring index and a target troubleshooting strategy corresponding to the resident monitoring node; if the monitoring determines that the first resident monitoring node is abnormal, monitoring a trigger type partition monitoring node related to the first resident monitoring node based on a second monitoring parameter; wherein the second monitoring parameter comprises a target monitoring index and a target troubleshooting strategy corresponding to the trigger type partition monitoring node. Through the technical scheme in the embodiment of the invention, the abnormal node can be quickly recovered.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device, electronic device, storage medium, security detection platform and computer program product. Background Art

[0002] In related technologies, the streaming data governance corresponding to traditional SIEM (Security Information and Event Management) and SOC (Security Operations Center) products faces the problems of long processing links and scattered distribution of abnormal points, making it difficult to quickly and accurately locate abnormal situations and causes. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, device, electronic device, storage medium, security detection platform and computer program product, which can improve data processing efficiency.

[0004] The technical solution of this application is implemented as follows:

[0005] The embodiment of the present application further provides a data processing method, which is applied to a security detection platform, wherein the security detection platform is used to process data to be processed from different source devices, and the method includes:

[0006] Based on the first monitoring parameter, the resident monitoring node in the processing flow of the data to be processed is monitored to monitor the abnormality in the resident monitoring node; wherein the first monitoring parameter includes the target monitoring indicator and the target troubleshooting strategy corresponding to the resident monitoring node;

[0007] If monitoring determines that there is an abnormality in the first resident monitoring node, the triggered partition monitoring node related to the first resident monitoring node is monitored based on the second monitoring parameters; wherein the second monitoring parameters include the target monitoring indicators and target troubleshooting strategies corresponding to the triggered partition monitoring nodes.

[0008] In the above scheme, the method further includes:

[0009] The data to be processed is parsed based on parsing rules to determine target data corresponding to the data to be processed; wherein the parsing rules are determined based on the data log of the source device; the data to be processed originates from the source device; and the target data is used to analyze potential security issues in the data to be processed.

[0010] In the above scheme, the method further includes:

[0011] Acquire the data to be processed and the data log corresponding to the data to be processed from the source device based on a universal interface;

[0012] The data log is input into the first preset model to obtain the parsing rule; the parsing rule supports data parsing for a target data format, and the target data format is the data format of the data log.

[0013] In the above solution, the step of parsing the data to be processed based on the parsing rules to determine the target data corresponding to the data to be processed includes:

[0014] Parsing the data to be processed based on the parsing rules to generate corresponding parsing data; wherein the parsing data corresponds to the value data in the data to be processed;

[0015] The analyzed data is enriched to determine the target data.

[0016] In the above solution, the enrichment processing is performed on the parsed data to determine the target data, including:

[0017] Analyzing and processing the data to be processed by using a second preset model to determine corresponding attack category information and key enriched fields;

[0018] The target data is determined based on the attack category information, the key enriched fields and the parsed data.

[0019] In the above scheme, the method further includes:

[0020] Each of the data to be processed is compared with the target data formed by analysis to determine a comparison result; wherein the comparison result is used to indicate whether the corresponding target data is obtained by analyzing the data to be processed.

[0021] In the above scheme, the method further includes:

[0022] Performing a difference comparison between each of the to-be-processed data and the corresponding target data to determine difference information between the first to-be-processed data and the corresponding target data;

[0023] Based on the difference information, an abnormal node is determined in the resident monitoring point and the triggered partition monitoring node, and the abnormal node is restored.

[0024] In the above solution, the monitoring of the resident monitoring nodes in the processing flow of the data to be processed based on the monitoring parameters to monitor the abnormalities in the resident monitoring nodes includes:

[0025] Based on the target monitoring index, the intermediate data generated by the corresponding resident monitoring node is monitored to determine the abnormal situation and the abnormal cause in the resident monitoring node.

[0026] In the above scheme, the method further includes:

[0027] If the abnormal situation indicates that the intermediate data corresponding to the first resident monitoring node is abnormal, the first resident monitoring node is restored based on the target troubleshooting strategy corresponding to the first resident monitoring node.

[0028] In the above scheme, the method further includes:

[0029] In response to the acquired monitoring instruction, monitoring is performed on any one of the resident monitoring node and the triggered partition monitoring node indicated by the monitoring instruction.

[0030] In the above scheme, the method further includes:

[0031] In response to the acquired analysis requirement parameters, the target data is analyzed, and analysis results corresponding to the target data are output.

[0032] The embodiment of the present application further provides a data processing device, which is applied to a security detection platform, wherein the security detection platform is used to process data to be processed from different source devices, including:

[0033] A monitoring unit, configured to monitor a resident monitoring node in a processing flow of the data to be processed based on a first monitoring parameter to monitor an abnormality in the resident monitoring node; wherein the first monitoring parameter includes a target monitoring indicator and a target troubleshooting strategy corresponding to the resident monitoring node;

[0034] The monitoring unit is also used to monitor the triggered partition monitoring node related to the first resident monitoring node based on the second monitoring parameters if the monitoring determines that there is an abnormality in the first resident monitoring node; wherein the second monitoring parameters include the target monitoring indicators and target troubleshooting strategies corresponding to the triggered partition monitoring nodes.

[0035] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the computer program.

[0036] An embodiment of the present application also provides a security detection platform, including a memory, a processor and a communication interface, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the computer program; the communication interface is used to connect to one or more source devices.

[0037] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are implemented.

[0038] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps in the above method when executed by a processor.

[0039] In an embodiment of the present application, the resident monitoring nodes in the processing flow of the data to be processed are monitored based on the first monitoring parameter to monitor the abnormalities in the resident monitoring nodes; wherein the first monitoring parameter includes the target monitoring index and the target troubleshooting strategy corresponding to the resident monitoring node; if the monitoring determines that the first resident monitoring node has an abnormality, the triggered partition monitoring nodes related to the first resident monitoring node are monitored based on the second monitoring parameter; wherein the second monitoring parameter includes the target monitoring index and the target troubleshooting strategy corresponding to the triggered partition monitoring node. In this way, the resident monitoring nodes are monitored based on the first monitoring parameter, and the scheme of the related triggered partition monitoring nodes when an abnormality occurs can use fewer monitoring resources to achieve full-process monitoring of the data to be processed. This monitoring method has low overhead and comprehensive monitoring indicators. After an abnormality occurs in the processing process of the data to be processed, the abnormal nodes can be quickly determined in the resident monitoring nodes and the triggered partition monitoring nodes through the first monitoring parameter and the second monitoring parameter, thereby achieving rapid recovery of the abnormal nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0041] Figure 2 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0042] Figure 3 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0043] Figure 4 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0044] Figure 5 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0045] Figure 6 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0046] Figure 7An optional flowchart of a data processing method provided in an embodiment of the present application;

[0047] Figure 8 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0048] Fig. 9 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0049] Fig.10 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0050] Fig.11 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0051] Fig.12 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0052] Fig.13 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0053] Fig.14 A schematic diagram of a hardware entity of an electronic device provided in an embodiment of the present application;

[0054] Fig.15 A schematic diagram of a hardware entity of a security detection platform provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0056] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0057] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] In the related technologies, data access processing of SIEM and SOC vendors is divided into two categories of solutions:

[0060] Solution 1: One is to directly store the original logs until they are used and then parse them. Solution 1 theoretically supports data access from any device, but leaves the problem to the place where the data is used in the detection scenario. This is a tricky way. It simply and crudely stores all the original logs, but does not understand the fields. The stored data cannot be directly used in subsequent detection scenarios.

[0061] Solution 2: One is to store after parsing all predefined fields, and directly access them when using them. Solution 2 is to directly perform access and parsing according to predefined fields during the three-party docking, so that subsequent businesses can shield third-party log data with huge differences and only process predefined standard data in a unified manner. Our data platform adopts Solution 2, which performs data governance while accessing. The intelligent data governance base mentioned in the present invention also performs data governance while following the access process.

[0062] Traditional SIEM and SOC vendors generally rely heavily on manually customized parsing rules due to lack of openness. The typical disadvantages of this type of traditional platform are: because the system design does not consider sufficient system openness, it relies heavily on manual processing of equipment adaptation customization, and because of the lack of intelligent access methods, the access efficiency is low and the development cycle is long, resulting in relatively low data processing efficiency.

[0063] At the same time, there are no convenient tools for writing and tuning custom detection rules on the platform, and customers cannot customize their security detection capabilities. In addition, there is a lack of customizable troubleshooting tools. When data processing anomalies (such as parsing anomalies, format errors, etc.) are discovered, they cannot be quickly located and restored as needed, resulting in low maintainability of the platform. In summary, this type of traditional platform has low access efficiency and cannot quickly locate and troubleshoot faults in scenarios where new third-party devices need to be quickly adapted or versions of connected devices need to be updated. The flexibility cannot meet customers' customized detection requirements.

[0064] Among them, the shortcomings of the solutions in the related art include but are not limited to:

[0065] 1. Single data collection method: Many existing solutions lack flexibility in data collection and cannot effectively support the docking requirements of multiple devices and systems, resulting in incomplete or inaccurate data collection.

[0066] 2. Insufficient parsing performance: Traditional data parsing engines often have problems with high resource usage and low performance, making it difficult to meet the real-time requirements of large-scale data processing.

[0067] 3. Limited level of intelligence: Existing solutions have a low level of intelligence in data analysis and mining and rely on manual adaptation and adjustment, resulting in low efficiency in access customization and delivery.

[0068] 4. Weak link monitoring capabilities: Many solutions lack effective tools and mechanisms for link monitoring and troubleshooting, making it difficult to detect and resolve problems in data links in a timely manner.

[0069] According to the above content, current security operation products have problems such as fixed access capabilities, low system flexibility, difficulty in flexibly expanding multiple access methods, multiple custom parsing rules, multiple custom detection rules, etc. The platform is not intelligent enough, highly dependent on manual participation, and the long data link cannot quickly locate problems, quickly troubleshoot and restore the system. Although traditional solutions can collect, analyze and store security data from different sources, monitor and analyze security events, and help organizations identify potential security threats, they have disadvantages such as low system openness, difficulty in flexible expansion, low intelligence, and low parsing performance.

[0070] In order to solve the above technical problems, the embodiment of the present application provides a data processing method, which is applied to a security detection platform. The security detection platform is used to process the data to be processed from different source devices. The full link of the security detection platform processing the data to be processed includes the following processes: data docking, data analysis, data enrichment and storage, and data association detection. Figure 1 , is an optional flow chart of the data processing method provided in the embodiment of the present application, which will be combined with Figure 1 The steps shown are explained:

[0071] S101. Monitor the resident monitoring nodes in the processing flow of the data to be processed based on a first monitoring parameter to monitor abnormalities in the resident monitoring nodes; wherein the first monitoring parameter includes a target monitoring indicator and a target troubleshooting strategy corresponding to the resident monitoring node.

[0072] In an embodiment of the present application, the security detection platform can obtain a first monitoring parameter, which includes a target monitoring indicator and a target troubleshooting strategy corresponding to the resident monitoring node. The security detection platform monitors whether the data generated by the corresponding resident monitoring node in the data processing flow to be processed is abnormal based on each target monitoring indicator in the first monitoring parameter.

[0073] Among them, the target monitoring indicators in the first monitoring parameter include: acquisition rate, resolution rate, packet loss rate, and integrity of the target data field. Among them, different target monitoring indicators are used for different resident monitoring nodes. Exemplarily, the access node in the processing flow can correspond to the acquisition rate indicator. The target troubleshooting strategy includes: a method for restoring and updating the underlying code or program of the corresponding node. The resident monitoring nodes may include: access nodes, resolution nodes, enrichment nodes, and storage nodes. Among them, each resident monitoring node corresponds to a related triggered partition monitoring node.

[0074] In the embodiment of the present application, the security detection platform may include SIEM or SOC system or cloud WAF (Web Application Firewall) device, or XDR (Extended Detection and Response) device. Among them, SIEM: data governance module, access to various external original logs, output unified standard format logs defined in the platform; for units with network security construction requirements, they usually purchase security devices of various manufacturers and categories, and it is difficult for security operators to frequently view the security logs they monitor on each device and analyze the security events they detect; SIEM relies on access to various device logs on the same platform to build a centralized operating environment for customers, save manpower, and improve network security operation efficiency. SOC: The Security Operation Center is a department dedicated to enterprise information security, and its main responsibilities are to monitor, detect and respond to security incidents in enterprise networks and systems. SOC is usually composed of a group of professional security analysts and engineers who use various security tools and technologies to protect the information assets of the enterprise, including real-time monitoring of network traffic, analysis of security logs, detection of malware and network attacks, etc. SIEM tends to be a technical solution for data collection, analysis and storage of security data from different sources, while SOC tends to be a centralized team and facility for monitoring, detecting and responding to security incidents. As a tool for data collection and analysis, SIEM provides the necessary information support for SOC, while SOC uses this information for real-time monitoring and response. The effective combination of the two can significantly enhance the security protection capabilities of an organization. WAF is a security protection tool specifically designed to protect World Wide Web (web) applications. It prevents various network attacks such as Structured Query Language (SQL) injection, cross-site scripting, and cross-site request forgery by monitoring and filtering Hypertext Transfer Protocol (HTTP) traffic. WAF is an indispensable part of the modern Web application security architecture, which can effectively prevent various network attacks and protect the security of user data and corporate assets.

[0075] S102. If monitoring determines that there is an abnormality in the first resident monitoring node, monitor the triggered partition monitoring node related to the first resident monitoring node based on the second monitoring parameters; wherein the second monitoring parameters include the target monitoring indicators and target troubleshooting strategies corresponding to the triggered partition monitoring nodes.

[0076] In an embodiment of the present application, the security detection platform monitors all resident monitoring nodes through a first monitoring parameter, determines that the data corresponding to the first resident monitoring node is abnormal, and then monitors whether the data generated by the triggered partition monitoring node related to the first resident monitoring node is abnormal based on the target monitoring indicator in the second monitoring parameter.

[0077] The second monitoring parameter includes the target monitoring index and target troubleshooting strategy corresponding to the triggered partition monitoring node. The triggered partition monitoring node is a monitoring node adjacent to the first resident monitoring node, or a monitoring node with data association with the first resident monitoring node, or a subnode under the first resident monitoring node.

[0078] Among them, the target monitoring indicators in the second monitoring parameter also include: acquisition rate, resolution rate, packet loss rate, and integrity of the target data field. Among them, different target monitoring indicators are used for different triggered partition monitoring nodes. Exemplarily, if the access node in the processing flow is abnormal, it can be determined that the parsing node related to the access node is a triggered partition monitoring node, and the resolution rate indicator can be used to monitor the data generated by the parsing node. The target troubleshooting strategy includes: a method for restoring and updating the underlying code or program of the corresponding node.

[0079] In an embodiment of the present application, the resident monitoring nodes in the processing flow of the data to be processed are monitored based on the first monitoring parameter to monitor the abnormalities in the resident monitoring nodes; wherein the first monitoring parameter includes the target monitoring index and the target troubleshooting strategy corresponding to the resident monitoring node; the triggered partition monitoring node is a node related to the resident monitoring node with the abnormality; if the monitoring determines that the first resident monitoring node has an abnormality, the triggered partition monitoring node related to the first resident monitoring node is monitored based on the second monitoring parameter; wherein the second monitoring parameter includes the target monitoring index and the target troubleshooting strategy corresponding to the triggered partition monitoring node. In this way, the resident monitoring node is monitored based on the first monitoring parameter, and the scheme of the related triggered partition monitoring node when an abnormality occurs can use less monitoring resources to realize the full process monitoring of the data to be processed. This monitoring method has low overhead and comprehensive monitoring indicators. After an abnormality occurs in the processing process of the data to be processed, the abnormal node can be quickly determined in the resident monitoring node and the triggered partition monitoring node through the first monitoring parameter and the second monitoring parameter, so as to realize the rapid recovery of the abnormal node.

[0080] See also Figure 2 , which is an optional flow chart of the data processing method provided in the embodiment of the present application, will be described in combination with the steps:

[0081] S201. Parse the data to be processed based on parsing rules to determine target data corresponding to the data to be processed; wherein the parsing rules are determined based on the data log of the source device; the data to be processed originates from the source device; and the target data is used to analyze potential security issues in the data to be processed.

[0082] In the embodiment of the present application, after the security detection platform is connected to the source device, it can obtain the data to be processed and the data log from the source device, and perform intelligent analysis on the data log to determine the parsing rules for the data to be processed. The parsing engine is used in combination with the parsing rules to parse the data to be processed and determine the corresponding target data. The data to be processed comes from the source device; the target data is used to analyze potential security issues in the data to be processed.

[0083] In an embodiment of the present application, the parsing rules include two types of parameters: data source policy parameters and actual operation parameters. The data source policy parameters include at least one of the following: device name, device manufacturer, device data source refers to Internet Protocol (IP), transmission protocol, encoding method. The actual operation parameters include at least one of the following: regular matching parameters of the log header, plug-in parsing parameters such as KV (Key-Value) object notation (JavaScript Object Notation, JSON), field extraction mapping parameters, and parsing status label post-processing parameters. Among them, the data source policy parameters are used to control what rules the data source uses. The actual operation parameters control how the parsing engine defines the specific operations of parsing logs.

[0084] Among them, the data log is a file or file group in the source device used to record the modification operations of the data to be processed. It records the operations such as insertion, update and deletion in the source device, and contains sufficient information to support data recovery after a system failure. The data to be processed may include attribute data for the object, and for example, may include fingerprint data or native place data for a certain person. The parsing rules may include code strings for parsing the data to be processed. When the source device connected to the security detection platform changes, the parsing rules change. When the data to be processed of the source device corresponding to the security detection platform changes, the parsing rules change synchronously.

[0085] See also Figure 3 , which is an optional flow chart of the data processing method provided in the embodiment of the present application, will be described in combination with the steps:

[0086] S301. Acquire the data to be processed and the data log corresponding to the data to be processed from the source device based on a universal interface.

[0087] In an embodiment of the present application, the security detection platform can use a highly available unified application programming interface (API) to connect to the source device, and use the highly available unified API interface to obtain the source device's processed data and the data log corresponding to the processed data.

[0088] Among them, the high-availability unified application programming interface is a pre-defined function that aims to provide applications and developers with the ability to access a set of routines based on certain software or hardware without having to access the source code or understand the details of the internal working mechanism.

[0089] In the related technology, traditional solutions generally only support mainstream log transmission methods such as syslog, kafka, etc., but for cloud security equipment docking that only provides API methods to obtain data, traditional solutions cannot be flexibly expanded and require customized access methods. In the embodiment of the present application, the high openness of the highly available unified API interface is utilized in the access method to support a variety of active and passive data collection methods, and can flexibly connect to different types of devices and data sources. This open design ensures that it can adapt to diverse needs in the initial stage of data collection and ensure the comprehensiveness and timeliness of data. The highly available unified API interface docking solution can support flexible configuration of multiple data access, device linkage and other API docking methods for the data base. Providing an API service base to achieve hierarchical docking and rapid business expansion capabilities with minimal duplication of investment. The openness of the data collection and access dimensions covers the following three aspects:

[0090] High scalability: By centrally managing device information, the system can flexibly adapt to different business needs and enhance customer interaction experience. It also supports multiple API docking methods to achieve rapid business expansion.

[0091] Low cost: By reducing the connection cost of third-party API log access modules, the system allows these modules to focus on their core business, reduce duplication of investment, and optimize resource allocation.

[0092] Efficient and reusable: The system is highly reusable. It can not only provide services for the data access module, but also be used as an interface for other modules, and even as a linkage interface for the base platform, further improving the overall efficiency of the system and providing a better customer interaction experience.

[0093] S302: Input the data log into the first preset model to obtain the parsing rule.

[0094] In the embodiment of the present application, after receiving the data log of the source device, the security detection platform can use the first preset model to learn the target data format of the data log, and use the first preset model to generate corresponding parsing rules. Among them, the parsing rules support data parsing for data in the target data format, and the target data format is the format of the data log.

[0095] Among them, the first preset model may include a generative pre-trained transformer (GPT) model. In other embodiments, the first preset model may also be other types of models, which are not specifically limited in the embodiments of the present application. GPT is a natural language processing model architecture based on deep learning. It is pre-trained through large-scale text data and can understand and generate human language. Among them, the security detection platform realizes efficient data parsing with low resource usage through a high-performance streaming parsing engine. According to the data log reported by the device, the local first preset model or the cloud first preset model can be used to dynamically learn and abstractly output the parsing rules of the corresponding data log, so that the new access device does not need to customize the parsing rules, but only relies on the model to continuously learn the format paradigm of the log of the newly accessed device, generate rules and automatically replace them, and complete the access of the new device without manual intervention. Compared with the current solutions in the industry, the new device access mode has changed the closed and inefficient working mode of such products in the past, and the openness of data governance has been greatly improved. As long as the original device log or the device log instruction manual can be provided, the access task closed loop can be quickly realized, which greatly improves the access efficiency and the range of devices that the platform can connect.

[0096] In the embodiment of the present application, the security detection platform uses the automated log parsing rule auxiliary generation method of the first preset model to directly analyze the target data format of the data log, generate parsing rules for parsing the data to be processed, and load them into the high-performance log parsing engine. This can achieve fully automatic data parsing of new devices without human intervention.

[0097] In an embodiment of the present application, the data to be processed and the data log corresponding to the data to be processed are obtained from the source device based on a universal interface. The data log is input into the first preset model to obtain the parsing rule; the parsing rule supports data parsing for the target data format, and the target data format is the data format of the data log. In addition, the parsing rules in the embodiment of the present application are intelligently determined by the first preset model, which improves the generation efficiency of the parsing rules compared to the manually generated solution, thereby improving the processing efficiency of the data to be processed. In addition, in the embodiment of the application, the parsing process to be processed is monitored. When an exception occurs, the exception can be quickly located, and then the exception can be quickly restored, thereby realizing rapid storage of data and improving the processing efficiency of the data to be processed.

[0098] See also Figure 4 , is an optional flow chart of a data processing method provided in an embodiment of the present application, Figure 2 S201 shown in the figure can also be implemented through S401 to S402, which will be described in combination with the steps:

[0099] S401. Analyze the data to be processed based on the analysis rules to generate corresponding analysis data; wherein the analysis data corresponds to the value data in the data to be processed.

[0100] In the embodiment of the present application, the security detection platform can use the parsing engine in combination with the parsing rules to parse the data to be processed, extract the valuable data therein, and obtain the parsed data. The valuable data in the data to be processed can be some fields defined in the predefined TMG (Threat Management Gateway) data standard.

[0101] Exemplarily, the data to be processed may include:

[0102] "{\"timestamp\":1731910317140,\"formatVersion\":1,\"webaclId\":\"arn:vender:wafv2:us-eas t-1:229568694718:global / webacl / CreatedByCloudFront-3665c169-5f69-4cdb-93f6-e5931c1de7b1 / 91a5224a-7845-4ff4-9b8c-23281d12adbf\",\"terminatingRuleId\":\"Default_Action\",\"termin atingRuleType\":\"REGULAR\",\"action\":\"ALLOW\",\"requestId\":\"PpqER60CEKbsptnkzWftvXxn K7MtvOt991dl32zzSDWDzcyn8Vj_kQ==\"},\"ja3Fingerprint\":\"479b976148ec2a1a195ae2e15805fefa\"}\n".

[0103] The valuable data may include: \"ja3Fingerprint\":\"479b976148ec2a1a195ae2e15805fefa\"}\n". It is used to characterize ja3 fingerprints. The security detection platform can use the parsing engine in combination with the parsing rules to parse the data to be processed, extract the valuable data \"ja3Fingerprint\":\"479b976148ec2a1a195ae2e15805fefa\"}\n", and convert the corresponding valuable data into parsed data.

[0104] S402: Perform enrichment processing on the parsed data to determine the target data.

[0105] In an embodiment of the present application, the security detection platform can utilize the data type of the data to be processed and other key fields in the data to be processed to enrich the parsed data and form corresponding target data.

[0106] In the embodiment of the present application, the data to be processed is parsed based on the parsing rules to form corresponding parsing data; wherein the parsing data corresponds to the value data in the data to be processed. The parsing data is enriched to determine the target data. In this way, since the parsing rules in the embodiment of the present application are intelligently determined by the first preset model, compared with the solution of manually generating parsing rules, the generation efficiency of the preset parsing rules is improved, thereby improving the generation efficiency of the parsing data.

[0107] See also Figure 5 , is an optional flow chart of a data processing method provided in an embodiment of the present application, Figure 4 S402 shown in the figure can also be implemented through S501 to S502, which will be described in combination with the steps:

[0108] S501: Analyze and process the data to be processed using a second preset model to determine corresponding attack category information and key enrichment fields.

[0109] In the embodiment of the present application, the security detection platform can use the second preset model to analyze and process the data to be processed, and determine the attack category information corresponding to the data to be processed, as well as the key enrichment fields.

[0110] The second preset model may include a classification aggregation model. The classification aggregation model may include a K-means (K-Means Clustering Algorithm) model and a hierarchical clustering model. In other embodiments, the second preset model may also include other models with the same function, which are not specifically limited here.

[0111] Among them, the key enrichment fields may include: specific threat object information in the data to be processed, user information, etc.

[0112] S502: Determine the target data based on the attack category information, the key enriched fields and the parsed data.

[0113] In the embodiment of the present application, the attack category information, the key enriched fields, and the parsed data may be combined in a preset order to determine the target data.

[0114] In the embodiment of the present application, the security detection platform can use the classification aggregation model to deeply understand the attack type of each piece of data to be processed based on the rule name or description, and enrich the corresponding three-level threat classification, which can be accurate to a certain precise vulnerability exploitation level and specific threat entity object. All data types that have not appeared in the initial access period can be automatically aggregated into an alarm by the engine of the Extended Detection and Response (XDR) platform during operation after the above fields are enriched by the classification aggregation module, thereby realizing the aggregation and noise reduction of alarms across devices. The openness of the data governance system is further improved.

[0115] Among them, XDR integrates SIEM log access, security detection engine, security operation module, and linkage disposal in one platform; it generates security alerts based on secondary analysis of access logs, further explores potential threats based on the security capabilities of reported security equipment, and links with disposal equipment to complete the closed loop of detection and disposal actions, bringing semi-automatic operation capabilities to network security in the covered environment, saving manpower while improving the quality of network security construction. It is generally believed that XDR is a mainstream product implementation form of security operation platform and security detection platform.

[0116] In the embodiment of the present application, the second preset model is used to analyze and process the data to be processed to determine the corresponding attack category information and key enrichment fields. Based on the attack category information, the key enrichment fields and the parsed data, the target data is determined. In this way, when facing different types of data, the structure of the enriched target data can be unified through the enrichment step, which facilitates the storage and classification of the target data.

[0117] See also Figure 6 , which is an optional flow chart of the data processing method provided in the embodiment of the present application, will be described in combination with the steps:

[0118] S601, comparing each of the data to be processed with the target data obtained by parsing, and determining a comparison result; wherein the comparison result is used to indicate whether the corresponding target data is obtained by parsing the data to be processed.

[0119] In the embodiment of the present application, the security detection platform can compare each accessed data to be processed with all the target data obtained by parsing to determine the comparison result, wherein the comparison result is used to indicate whether the data to be processed is parsed to obtain the corresponding target data.

[0120] In the embodiment of the present application, the amount of the acquired data to be processed can be compared with the amount of the target data formed by the analysis to determine the comparison result. The identification information in the data to be processed can also be compared with the formed analyzed data to determine the comparison result.

[0121] Exemplarily, a special log for sniffing can be inserted into the data to be processed, and the special log for sniffing the target data formed by the parsing node out of the warehouse can be checked to see whether it has arrived accurately, etc., to determine the comparison result. Among them, currently it is done by recording before and after separately. Record a piece of data to be processed before parsing, and then record a piece of target data after the data to be processed is parsed (it may be a successful or failed parsing). When it is necessary to troubleshoot the problem of lost data, find the original log identification information of the entry data, and then compare it with the corresponding target data. If it can be found, it is not lost. If it cannot be found, it is lost.

[0122] In the embodiment of the present application, each of the data to be processed is compared with the target data formed by the analysis to determine the comparison result; wherein the comparison result is used to indicate whether the corresponding target data is obtained by the analysis of the data to be processed. In this way, when data loss or analysis failure occurs, it can be quickly determined that the data loss occurred, so that the node with the lost data can be restored to achieve complete and accurate data analysis.

[0123] See also Figure 7 , which is an optional flow chart of the data processing method provided in the embodiment of the present application, will be described in combination with the steps:

[0124] S602: Perform a difference comparison between each of the to-be-processed data and the corresponding target data to determine difference information between the first to-be-processed data and the corresponding target data.

[0125] In the embodiment of the present application, after the security detection platform forms the target data corresponding to the data to be processed, it can compare the difference between the data to be processed and the corresponding target data based on the inspection tool. After a period of comparison, the security detection platform determines the difference information between the first data to be processed and the corresponding target data. The difference information is used to characterize the difference caused by the analysis and enrichment process of the data to be processed.

[0126] S603: Determine abnormal nodes in the resident monitoring points and the triggered partition monitoring nodes based on the difference information, and recover the abnormal nodes.

[0127] In an embodiment of the present application, the security detection platform performs analysis based on the determined difference information, determines abnormal nodes in the resident monitoring points and the triggered partition monitoring nodes in the processing of the first data to be processed, and recovers the abnormal nodes using corresponding target troubleshooting strategies.

[0128] Exemplarily, if the difference information is a difference in attack category information, it can be determined that the abnormal point is an enriched node, and the enriched node can be restored using the target troubleshooting strategy corresponding to the enriched node.

[0129] In the embodiment of the present application, a difference comparison is performed between each data to be processed and the corresponding target data to determine the difference information between the first data to be processed and the corresponding target data; based on the difference information, an abnormal node is determined in the resident monitoring point and the triggered partition monitoring node, and the abnormal node is recovered. In this way, based on the difference comparison between the data to be processed and the corresponding target data, the abnormal point can be quickly determined, and then the abnormal point can be quickly recovered, thereby improving the accuracy of the analysis of the data to be processed.

[0130] See also Figure 8 , is an optional flow chart of a data processing method provided in an embodiment of the present application, Figure 1 S101 shown in the figure can also be implemented by S701, which will be described in combination with the steps:

[0131] S701: Monitor the intermediate data generated by the corresponding resident monitoring node based on the target monitoring indicator, and determine the abnormal situation and abnormal cause in the resident monitoring node.

[0132] In an embodiment of the present application, the resident monitoring node may include: an access node, a parsing node, an enrichment node, and a storage node. Each target monitoring indicator has the identification information of the corresponding resident monitoring node. After any resident monitoring node generates corresponding intermediate data, the intermediate data can be monitored using the target monitoring indicator determined by the identification information of the node. Based on the monitoring result of the intermediate data, determine whether the intermediate data is abnormal, as well as the abnormal situation and abnormal cause of the intermediate data.

[0133] Exemplarily, the intermediate data corresponding to the enriched node is the enriched target data, and the target monitoring indicator corresponding to the enriched node: the integrity of the target data field can be used to monitor the target data to determine the integrity of the target data. If the target data is incomplete, the abnormal situation of incomplete target data and the abnormal reason of insufficient enrichment are determined.

[0134] Among them, the intermediate data corresponding to the access node is the data to be processed, the intermediate data corresponding to the parsing node is the parsed data, the intermediate data corresponding to the enrichment node is the target data, and the intermediate data corresponding to the storage node is the target data after storage.

[0135] See also Fig. 9 , is an optional flow chart of a data processing method provided in an embodiment of the present application, Figure 8 S701 shown in the figure may also include S801, which will be described in combination with the steps:

[0136] S801: If the abnormal situation indicates that the intermediate data corresponding to the first resident monitoring node is abnormal, restore the abnormal target monitoring node based on the target troubleshooting strategy corresponding to the first resident monitoring node.

[0137] In the embodiment of the present application, if the security detection platform determines that the abnormal situation represents an abnormal intermediate data corresponding to any first resident node, the abnormal first resident monitoring node is restored using the corresponding target troubleshooting strategy. The restoration process for the abnormal first resident monitoring node may include: changing the configuration of the abnormal monitoring sub-process, updating the code logic, or updating the corresponding preset parsing rules, etc. The embodiment of the present application does not limit the method of restoring the monitoring sub-process.

[0138] In the embodiment of the present application, after the security detection platform recovers the abnormal first resident target monitoring node, it can use the recovered first resident monitoring node to perform processing on the data to be processed again, form new target data, and store the new parsed data. Alternatively, after the security detection platform recovers the abnormal first resident monitoring node, it can use the recovered first resident monitoring node to perform processing on the next data to be processed.

[0139] In the embodiment of the present application, the security detection platform can quickly identify and solve problems in the data processing process in a timely manner through real-time monitoring and intelligent troubleshooting mechanisms, ensuring the stability and reliability of the data flow. The intelligent monitoring module provides plug-in custom monitoring indicators and monitoring alarm mechanisms. Administrators can add or remove required monitoring sub-processes at any time according to user needs, ensuring that the basic stability of the system is not affected while the security detection platform provides highly open capabilities.

[0140] For example, the target data after parsing, enrichment and storage is viewed on the log retrieval page of the security detection platform. At this time, the following abnormal scenario is assumed: if the parsing result of the JA3 fingerprint (ja3Fingerprint) field after storage is found to be: "479b976148ec2a1a195ae2e158". It is found that the value of this field is truncated compared with the value in the data to be processed, and "05fefa" is missing. At this time, through the custom troubleshooting plug-in, in the monitoring sub-process of the conversion key-value pair process and the enrichment process, troubleshooting monitoring can be added to locate the link where the ja3Fingerprint value is truncated. The security detection platform found the fault point: the maximum length of the field output is limited in the standardized plug-in processing process. At this time, by quickly replacing the abnormal standardized plug-in, the recovery of the parsing abnormality fault can be achieved. It can be seen that the traditional solution has the disadvantages of lack of custom troubleshooting capabilities and inability to quickly restore business after a failure. Among them, in the related technology, if such an output field format is abnormal, it is impossible to locate and recover the problem at the customer site by quickly inserting a custom troubleshooting plug-in module. It is usually necessary to first capture the package from the customer site to obtain the original input data, and then have the developer play it back in the R&D environment to reproduce the problem. Then, by single-step debugging against the corresponding parsing rule logic code in the parsing engine, the field abnormal truncation problem in the parsing process can be located. After the problem is fixed, it is also impossible to achieve rapid recovery by updating the standardized plug-in.

[0141] In an embodiment of the present application, the intermediate data generated by the corresponding resident monitoring node is monitored based on the target monitoring index, and the abnormal situation and the cause of the abnormality in the resident monitoring node are determined. If the abnormal situation indicates that the intermediate data corresponding to the first resident target monitoring node is abnormal, the first resident target monitoring node is restored based on the target troubleshooting strategy corresponding to the first resident target monitoring node. Since the target monitoring index monitors the resident monitoring node in the embodiment of the present application, when an abnormality occurs, the abnormality can be quickly located, and then the abnormality can be quickly restored, thereby improving the processing efficiency of the data to be processed.

[0142] The illustrated S102 can also be implemented by S702, which will be described in combination with the steps:

[0143] S702: Based on the target monitoring index in the second monitoring parameter, monitor the intermediate data generated by the triggered partition monitoring node related to the first resident monitoring node to determine the abnormal situation and abnormal cause in the triggered partition monitoring node.

[0144] In an embodiment of the present application, each target monitoring indicator in the second monitoring parameter has the identification information of the corresponding triggered partition monitoring node. After the security detection platform determines the first resident monitoring node, the triggered partition monitoring node related to the first resident monitoring node can be determined, and the target monitoring indicator determined by the identification information of the triggered partition monitoring node can be used to monitor the intermediate data generated by the triggered partition monitoring node. Based on the monitoring results of the intermediate data, determine whether the intermediate data is abnormal, as well as the abnormal situation and abnormal cause of the intermediate data.

[0145] Among them, if the abnormal situation represents that the intermediate data corresponding to the triggered partitioned monitoring node related to the first resident monitoring node is abnormal, the abnormal target monitoring node is restored based on the target troubleshooting strategy corresponding to the triggered partitioned monitoring node.

[0146] In the embodiment of the present application, if the security detection platform determines that the abnormal situation indicates that the intermediate data corresponding to the triggered partition monitoring node related to the first resident node is abnormal, the abnormal triggered partition monitoring node is restored using the corresponding target troubleshooting strategy. The recovery process for the abnormal triggered partition monitoring node may include: changing the configuration of the abnormal monitoring sub-process, updating the code logic, or updating the corresponding preset parsing rules, etc. The embodiment of the present application does not limit the method of recovery process for the monitoring sub-process.

[0147] In an embodiment of the present application, the intermediate data generated by the relevant triggered partition monitoring nodes are monitored based on the target monitoring index described in the second monitoring parameter, and the abnormal conditions and abnormal causes in the triggered partition monitoring nodes are determined. If the abnormal condition represents that the intermediate data corresponding to the triggered partition monitoring node related to the first resident target monitoring node is abnormal, the relevant triggered partition monitoring node is restored based on the target troubleshooting strategy corresponding to the relevant triggered partition monitoring node. Since the resident monitoring nodes are monitored using target monitoring indicators in the embodiment of the present application, when an abnormality occurs, the nodes related to the abnormal first resident monitoring node can be monitored, and this process can be used to accurately determine the truly abnormal nodes, and then quickly recover the abnormality, thereby improving the processing efficiency of the data to be processed.

[0148] See also Fig.10 , which is an optional flow chart of the data processing method provided in the embodiment of the present application, will be described in combination with the steps:

[0149] S901. In response to an acquired monitoring instruction, monitor any one of the resident monitoring node and the triggered partition monitoring node indicated by the monitoring instruction.

[0150] In an embodiment of the present application, the security detection platform obtains monitoring instructions through a human-computer interaction device, wherein the monitoring instructions may include identification information of any one of the resident monitoring nodes and the triggered partition monitoring nodes, and determines the corresponding target monitoring indicator based on the identification information, and uses the determined target monitoring indicator to monitor whether the data generated by the node indicated is abnormal.

[0151] In this way, the security detection platform can trigger monitoring of different links of the entire link through commands, thereby increasing the freedom of the monitoring method.

[0152] See also Fig.11 , which is an optional flow chart of the data processing method provided in the embodiment of the present application, will be described in combination with the steps:

[0153] S902: In response to the acquired analysis requirement parameters, analyze the target data and output analysis results corresponding to the parsed data.

[0154] In an embodiment of the present application, the security detection platform can respond to the user's analysis requirements for some fields in the target data, obtain analysis requirement parameters through a human-computer interaction device, analyze the target data, output the analysis results for the target data, and display them.

[0155] The analysis requirement parameters may include: attack type analysis parameters for target data, attack target analysis parameters, and attack source analysis parameters.

[0156] In the related art, data mining and association detection capabilities are generally the detection engine capabilities of the manufacturer, which do not take into account the differentiated and highly customized security capability requirements for customers and provide sufficient system development. This results in customers being able to only use the limited development space provided by the system to write some simple filtering conditions and generate customized logs or alarms. As for newly added data standard fields or defined auxiliary customized association detection rules, they are not supported at all. The data mining and association detection module designed in the present invention fully considers the customer's customized security capability requirements, and has a high degree of support for the customer's customized operational experience to be precipitated into product capabilities. Customers can customize parsing rules, customize standardized output fields, and even write specific association detection rules based on newly added custom fields to output customized association alarms and events.

[0157] This application not only proposes the above open solution design in the three modules of data collection and access, intelligent data analysis, and link monitoring and troubleshooting, but also further improves the openness of the data governance platform by effectively combining these three modules. In the face of data combing from different manufacturers, high input openness is achieved through a unified API interface, and data is analyzed by unified standards through data analysis, so that link monitoring and troubleshooting can uniformly analyze data from different manufacturers, while ensuring openness, achieving full-link automation and unified data governance. For a detailed description of the above technical solutions, please refer to Fig.12 The steps in are as follows:

[0158] S11, data connection and collection.

[0159] In the embodiment of the present application, the security detection platform can utilize a highly available unified API interface to flexibly configure multiple data accesses.

[0160] S12. Data analysis.

[0161] In an embodiment of the present application, the data parsing engine can receive preset parsing rules generated by the rule intelligent generation module before the data parsing process, so that newly accessed data to be processed can be recognized by the parsing engine and correctly split into key-value pairs.

[0162] S13. Data enrichment and storage.

[0163] In the embodiment of the present application, the data classification aggregation module in the security detection platform will enrich and fill in the classification and key enrichment fields of the log after data analysis.

[0164] S14. Data association detection.

[0165] In the embodiment of the present application, in addition to the built-in detection engine, the security detection platform supports generating custom alarms for data enriched by standardized analysis through custom association analysis rules.

[0166] S15. Link troubleshooting.

[0167] In the embodiment of the present application, the security detection platform can also customize and add monitoring and collection indicators throughout the entire data governance process, so as to facilitate system administrators or users to quickly discover data anomalies and quickly handle and recover them.

[0168] The advantages of this application scheme in terms of high openness, intelligence, and high performance are mainly reflected as follows:

[0169] 1. Improve data collection and access efficiency: By providing a variety of active and passive data collection and docking methods, it meets the docking requirements of different devices and systems and ensures the comprehensiveness and accuracy of data collection.

[0170] 2. Optimize data analysis performance: Use a high-performance streaming analysis engine to achieve low resource usage and high-performance data analysis services, ensuring the timeliness and reliability of data processing.

[0171] 3. Improve the intelligence level of data mining and standardization: Through AI-driven intelligent analysis and data mining, improve the efficiency of data access customization and delivery, and provide reliable guarantees for the construction of upper-level data services.

[0172] 4. Enhanced link monitoring and troubleshooting capabilities: Through correlation detection and link monitoring, problems in data links can be quickly identified and resolved to ensure the stability and continuity of the data governance process.

[0173] See also Fig.13 , which is a structural diagram of a data processing device provided in an embodiment of the present application.

[0174] The embodiment of the present application further provides a data processing device 800 applied to a security detection platform, wherein the security detection platform is used to process data to be processed from different source devices, including: a monitoring unit 801.

[0175] A monitoring unit 801 is used to monitor the resident monitoring node in the processing flow of the data to be processed based on a first monitoring parameter to monitor anomalies in the resident monitoring node; wherein the first monitoring parameter includes a target monitoring indicator and a target troubleshooting strategy corresponding to the resident monitoring node;

[0176] The monitoring unit 801 is also used to monitor the triggered partition monitoring node related to the first resident monitoring node based on the second monitoring parameters if the monitoring determines that there is an abnormality in the first resident monitoring node; wherein the second monitoring parameters include the target monitoring indicators and target troubleshooting strategies corresponding to the triggered partition monitoring node.

[0177] In an embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to parse the data to be processed based on parsing rules to determine the target data corresponding to the data to be processed; wherein the parsing rules are determined based on the data log of the source device; the data to be processed originates from the source device; and the target data is used to analyze potential security issues in the data to be processed.

[0178] In the embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to obtain the data to be processed and the data log corresponding to the data to be processed from the source device based on the universal interface;

[0179] The data log is input into the first preset model to obtain the parsing rule; the parsing rule supports data parsing for a target data format, and the target data format is the data format of the data log.

[0180] In the embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to parse the data to be processed based on the parsing rule to generate corresponding parsing data; wherein the parsing data corresponds to the value data in the data to be processed;

[0181] The analyzed data is enriched to determine the target data.

[0182] In the embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to analyze and process the data to be processed using the second preset model to determine the corresponding attack category information and key enrichment fields;

[0183] The target data is determined based on the attack category information, the key enriched fields and the parsed data.

[0184] In an embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to compare each of the data to be processed with the target data formed by analysis to determine the comparison result; wherein, the comparison result is used to indicate whether the corresponding target data is obtained by parsing the data to be processed.

[0185] In the embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to perform a difference comparison between each of the to-be-processed data and the corresponding target data, and determine the difference information between the first to-be-processed data and the corresponding target data;

[0186] Based on the difference information, an abnormal node is determined in the resident monitoring point and the triggered partition monitoring node, and the abnormal node is restored.

[0187] In the embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to monitor the intermediate data generated by the corresponding resident monitoring node based on the target monitoring indicator, and determine the abnormal situation and abnormal cause in the resident monitoring node.

[0188] In an embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to restore the first resident monitoring node based on the target troubleshooting strategy corresponding to the first resident monitoring node if the abnormal situation represents that the intermediate data corresponding to the first resident monitoring node is abnormal.

[0189] In the embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to monitor any one of the resident monitoring node and the triggered partition monitoring node indicated by the monitoring instruction in response to the acquired monitoring instruction.

[0190] In the embodiment of the present application, the monitoring unit 801 in the data processing device 800 is used to analyze the target data in response to the acquired analysis requirement parameters, and output the analysis results corresponding to the target data.

[0191] It should be noted that in the embodiment of the present application, if the above-mentioned data processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including a number of instructions to enable a data processing device (which can be a personal computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0192] Correspondingly, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method on the data processing device side are implemented.

[0193] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0194] It should be noted that Fig.14 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Fig.14 As shown, an embodiment of the present application provides an electronic device 900, including a memory 902 and a processor 901, wherein the memory 902 stores a computer program that can be run on the processor 901, and the processor 901 implements the steps in the above method when executing the program, wherein;

[0195] The processor 901 generally controls the overall operation of the electronic device 900 .

[0196] The memory 902 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or processed by the processor 901 and various modules in the electronic device 900 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0197] Correspondingly, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by a processor 901 of an electronic device 901 to complete the steps in the method on the data processing device 800 side.

[0198] It should be noted that Fig.15 A hardware entity diagram of a security detection platform provided in an embodiment of the present application, such as Fig.15 As shown, an embodiment of the present application also provides a security detection platform 1000, including a memory 902, a processor 901 and a communication interface 903, wherein the memory 902 stores a computer program that can be executed on the processor 901, and the processor 901 implements the steps in the above method when executing the computer program; the communication interface 903 is used to connect to one or more source devices.

[0199] Among them, the security detection platform 1000 can be deployed by hardware or by software operation service (Software as a Service, SaaS). When SaaS deployment is adopted, the data to be processed of different source devices is accessed through the software virtual interface.

[0200] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0201] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0202] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0203] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0204] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0205] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a magnetic disk or an optical disk, and other media that can store program codes.

[0206] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a mobile storage device, a ROM, a magnetic disk, or an optical disk.

[0207] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: Applied to a security detection platform, the security detection platform is used to process data to be processed from different source devices, the method includes: Based on the first monitoring parameter, the resident monitoring node in the processing flow of the data to be processed is monitored to monitor the abnormality in the resident monitoring node; wherein the first monitoring parameter includes the target monitoring indicator and the target troubleshooting strategy corresponding to the resident monitoring node; If monitoring determines that there is an abnormality in the first resident monitoring node, the triggered partition monitoring node related to the first resident monitoring node is monitored based on the second monitoring parameters; wherein the second monitoring parameters include the target monitoring indicators and target troubleshooting strategies corresponding to the triggered partition monitoring nodes.

2. The data processing method according to claim 1, characterized in that: The method further comprises: The data to be processed is parsed based on parsing rules to determine target data corresponding to the data to be processed; wherein the parsing rules are determined based on the data log of the source device; the data to be processed originates from the source device; and the target data is used to analyze potential security issues in the data to be processed.

3. The data processing method according to claim 2, characterized in that: The method further comprises: Acquire the data to be processed and the data log corresponding to the data to be processed from the source device based on a universal interface; The data log is input into the first preset model to obtain the parsing rule; the parsing rule supports data parsing for a target data format, and the target data format is the data format of the data log.

4. The data processing method according to claim 2, characterized in that: The step of parsing the data to be processed based on the parsing rules to determine the target data corresponding to the data to be processed includes: Parsing the data to be processed based on the parsing rules to generate corresponding parsing data; wherein the parsing data corresponds to the value data in the data to be processed; The analyzed data is enriched to determine the target data.

5. The data processing method according to claim 4, characterized in that: The enrichment processing is performed on the parsed data to determine the target data, including: Analyzing and processing the data to be processed by using a second preset model to determine corresponding attack category information and key enriched fields; The target data is determined based on the attack category information, the key enriched fields and the parsed data.

6. The data processing method according to claim 2, characterized in that: The method further comprises: Each of the data to be processed is compared with the target data formed by analysis to determine a comparison result; wherein the comparison result is used to indicate whether the corresponding target data is obtained by analyzing the data to be processed.

7. The data processing method according to claim 2, characterized in that: The method further comprises: Performing a difference comparison between each of the to-be-processed data and the corresponding target data to determine difference information between the first to-be-processed data and the corresponding target data; Based on the difference information, an abnormal node is determined in the resident monitoring point and the triggered partition monitoring node, and the abnormal node is restored.

8. The data processing method according to any one of claims 1 to 7, characterized in that: The monitoring of the resident monitoring node in the processing flow of the data to be processed based on the first monitoring parameter to monitor the abnormality in the resident monitoring node includes: Based on the target monitoring index, the intermediate data generated by the corresponding resident monitoring node is monitored to determine the abnormal situation and the abnormal cause in the resident monitoring node.

9. The data processing method according to claim 8, characterized in that: The method further comprises: If the abnormal situation indicates that the intermediate data corresponding to the first resident monitoring node is abnormal, the first resident monitoring node is restored based on the target troubleshooting strategy corresponding to the first resident monitoring node.

10. The data processing method according to any one of claims 1 to 7, characterized in that: The method further comprises: In response to the acquired monitoring instruction, monitoring is performed on any one of the resident monitoring node and the triggered partition monitoring node indicated by the monitoring instruction.

11. The data processing method according to any one of claims 1 to 7, characterized in that: The method further comprises: In response to the acquired analysis requirement parameters, the target data is analyzed, and analysis results corresponding to the target data are output.

12. A data processing device, characterized in that: Applied to a security detection platform, the security detection platform is used to process data to be processed from different source devices, including: A monitoring unit, configured to monitor a resident monitoring node in a processing flow of the data to be processed based on a first monitoring parameter to monitor an abnormality in the resident monitoring node; wherein the first monitoring parameter includes a target monitoring indicator and a target troubleshooting strategy corresponding to the resident monitoring node; The monitoring unit is also used to monitor the triggered partition monitoring node related to the first resident monitoring node based on the second monitoring parameters if the monitoring determines that there is an abnormality in the first resident monitoring node; wherein the second monitoring parameters include the target monitoring indicators and target troubleshooting strategies corresponding to the triggered partition monitoring nodes.

13. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps in the method according to any one of claims 1 to 11 when executing the computer program.

14. A safety detection platform, characterized in that: The method comprises a memory, a processor and a communication interface, wherein the memory stores a computer program executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 11 when executing the computer program; The communication interface is used to connect to one or more source devices.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 11 are implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 11.

Images