Data circulation monitoring method and system based on data processing unit

By using DPU-based data circulation monitoring methods in high-bandwidth networks, the data circulation map is built, which solves the problem of low data circulation security caused by CPU performance bottlenecks, real-time monitoring of data circulation and source positioning, and improves network security sex.

CN119071068APending Publication Date: 2024-12-03BEIJING UNIV OF POSTS & TELECOMM
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411198200.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In high-bandwidth networks, CPU performance bottlenecks lead to low data circulation security, especially under the cloud edge architecture, the real-time, multi-source heterogeneity and privacy sensitivity of data flow increase network security risks, and the edge-side device's Security is uncontrollable and faces the risk of unknown attacks.

Method used

Using data circulation monitoring method based on data processing unit (DPU), the DPU obtains external network data and service terminals at the edge gateway to obtain business application data and database data, builds a data circulation map, and performs it on the full-link data detection cloud platform. Feature extraction and monitoring to realize real-time monitoring and source positioning of data circulation.

Benefits of technology

It has expanded the network security boundaries and provided an integrated data circulation monitoring system, real-time monitoring and source positioning for abnormal data behaviors, improved the security of data circulation, and reduced the risk of unknown attacks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071068A_ABST
    Figure CN119071068A_ABST
Patent Text Reader

Abstract

The invention provides a data circulation monitoring method and system based on a data processing unit, and the method comprises the steps: obtaining external network data at an edge gateway based on the data processing unit DPU, and obtaining service application data and database data at a service terminal based on the DPU; carrying out feature extraction on the external network data, the service application data and the database data on the basis of a data global circulation atlas construction module on a data full-link detection cloud platform, and constructing a data global circulation atlas; and monitoring data circulation based on the data global circulation graph. According to the data circulation monitoring method and system based on the data processing unit, the network security boundary can be expanded, an integrated data circulation monitoring system with the DPU as the network axis is provided, and underlying logic enhancement facing abnormal data behavior real-time monitoring and source positioning is achieved, so that the security of data circulation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a data circulation monitoring method and system based on a data processing unit. Background Art

[0002] In recent years, with the rapid development of big data, cloud computing, and the Internet of Things, as well as the popularization of 5G network technology, the total number of network access devices has increased sharply. Under the new cloud-edge-end architecture, the current communication network has the characteristics of real-time, multi-source heterogeneity, and privacy sensitivity in data flow, and the data flow is manifested as a multi-level flow structure of cloud-edge-end. In addition, data usually exists in various forms such as digital signals, text, images, and videos. The data sources are more diverse, and the data transmission requirements and the adopted communication protocols are different. There are often differences in security management between data interaction subjects, and the data flow process has higher requirements for the legality and compliance of data behavior. At the same time, in cloud computing, interactive data, user information data, etc. are fully used, and such data often have strong privacy attributes. Data leakage or tampering may cause serious economic losses.

[0003] Due to the complexity and dynamism of network systems, network security boundaries are becoming increasingly blurred. Data transmission and sharing often cross subjects and security zones, greatly increasing the security risks of cross-domain data. In the IoT system, the security of edge devices and external subjects is relatively uncontrollable, and the risk of unknown attacks faced by data flow increases dramatically. Therefore, it is urgent to improve the global data security mechanism in network construction to adapt to the current network development trend and security situation. Summary of the invention

[0004] The present invention provides a data flow monitoring method and system based on a data processing unit, which are used to solve the technical problem of low data flow security caused by CPU performance bottleneck in a high-bandwidth network in the prior art.

[0005] The present invention provides a data circulation monitoring method based on a data processing unit, comprising the following steps.

[0006] Acquire external network data at the edge gateway based on the data processing unit DPU, and acquire business application data and database data at the business terminal based on the DPU; Based on the data global circulation map construction module, feature extraction is performed on the external network data, the business application data and the database data on the data full-link detection cloud platform to construct a data global circulation map; The data circulation is monitored based on the global data circulation map.

[0007] In some embodiments, the DPU includes: NIC subsystem, SoC and rule policy library; The NIC subsystem is used to collect data features; The SoC is used for non-real-time data processing and uploading the processed data to the data full-link detection cloud platform; The rule policy library is used to define data collection rules and data processing rules.

[0008] In some embodiments, the non-real-time data processing includes one or more of the following: Data fusion; Data modeling; Data flow control; Data preprocessing.

[0009] In some embodiments, the data full-link detection cloud platform further includes a DPU management module; The DPU management module is used to obtain the external network data, the business application data and the database data.

[0010] In some embodiments, the monitoring of data circulation based on the global data circulation map includes one or more of the following: Data topology restoration; Data breach tracing; Security situation awareness.

[0011] The present invention also provides a data flow monitoring system based on a data processing unit, comprising: DPU deployed on edge gateways, DPU deployed on business terminals, and data full-link detection cloud platform; The DPU deployed on the edge gateway is used to obtain external network data; The DPU deployed in the service terminal is used to obtain service application data and database data; The data full-link detection cloud platform is used to extract features from the external network data, the business application data and the database data, construct a data global circulation map, and monitor data circulation based on the data global circulation map.

[0012] In some embodiments, the DPU includes: NIC subsystem, SoC and rule policy library; The NIC subsystem is used to collect data features; The SoC is used for non-real-time data processing and uploading the processed data to the data full-link detection cloud platform; The rule policy library is used to define data collection rules and data processing rules.

[0013] In some embodiments, the SoC includes one or more of the following modules: Data fusion module; Data modeling module; Data flow control module; Data preprocessing module.

[0014] In some embodiments, the data full-link detection cloud platform further includes a DPU management module; The DPU management module is used to obtain the external network data, the business application data and the database data.

[0015] In some embodiments, the data full-link detection cloud platform is used to perform one or more of the following data monitoring: Data topology restoration; Data breach tracing; Security situation awareness.

[0016] The data flow monitoring method and system based on the data processing unit provided by the present invention obtain external network data at the edge gateway through the DPU, and obtain business application data and database data at the business terminal, and then build a data global flow map through the data full-link detection cloud platform, and finally use the data global flow map to monitor data flow. It can expand the network security boundary and provide an integrated data flow monitoring system with DPU as the network axis, realize the underlying logic enhancement for real-time monitoring of abnormal data behavior and source positioning, thereby improving the security of data flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of a data circulation monitoring method based on a data processing unit provided by the present invention.

[0019] Figure 2 It is a general architecture diagram of the data flow monitoring system based on the data processing unit provided by the present invention.

[0020] Figure 3 It is a structural diagram of the DPU deployed on the edge gateway provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the structure of the DPU deployed in the service terminal provided by the present invention.

[0022] Figure 5This is a flow chart of external network data collection provided by the present invention.

[0023] Figure 6 This is a data fusion flow chart provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of the data full-link detection cloud platform provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Figure 1 is a flow chart of a data flow monitoring method based on a data processing unit provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Acquire external network data at the edge gateway based on the data processing unit DPU, and acquire business application data and database data at the business terminal based on the DPU.

[0027] Specifically, Figure 2 is the overall architecture diagram of the data flow monitoring system based on the data processing unit provided by the present invention, such as Figure 2 As shown in the figure, the system architecture is divided into three parts: cloud, edge, and end.

[0028] First, in the preparation stage, the edge and end sides complete the deployment of data processing unit (DPU) components, and the corresponding components are connected to the cloud platform DPU management module at the same time, and the rules and policies for DPU information collection and processing are automatically issued. The DPU log is connected to the cloud platform, and all components are uniformly supervised by the management module in subsequent work.

[0029] Then in the data collection stage, the edge-side and end-side DPU data feature collection modules work synchronously with the data processing module to collect real-time information from the external network, business applications, and databases; the data cleaning, data desensitization, and data fusion sub-modules perform pre-processing on the collected information at the data level, such as eliminating redundancy, eliminating sensitive patterns, and aligning structures.

[0030] Among them, the edge gateway has an embedded DPU component, which is widely deployed on key network nodes. While using its own hardware advantages to accelerate network protocol parsing and data packet forwarding, it collects external network data of various granularities in real time. At the same time, the DPU component is embedded in the business server of the business terminal, acting as a network data circulation agent to release server computing resources. At the same time, it collects business application data from the kernel interface, including system behavior context logs and data flow information, and directly connects with the original data in the database by accessing the database-specific data interface to obtain database data.

[0031] Step 102: On the data full-link detection cloud platform, based on the data global circulation map construction module, feature extraction is performed on the external network data, the business application data and the database data to construct a data global circulation map.

[0032] Specifically, in the data management stage, the cloud platform DPU management module receives the information of each DPU node, and the data global circulation map construction module reduces and extracts features from the data again at the semantic level to construct, store and maintain the data global circulation map.

[0033] Among them, the data full-link detection cloud platform, through the DPU management module, is responsible for data transmission with all DPUs on the one hand, and aggregates multi-party data collected by DPU through a dedicated interface; on the other hand, it is responsible for comprehensive and efficient management of DPUs distributed at the edge and end, covering device policy configuration, operation status monitoring, edge maintenance and upgrades and other aspects to ensure the stable and reliable operation of the entire system.

[0034] Then, based on the information collected by DPU from edge gateways and business servers, a global data flow map is generated.

[0035] Step 103: Monitor data circulation based on the global data circulation map.

[0036] Specifically, in the application stage, the data global circulation map application module calls the data global circulation map to perform applications such as security situation awareness, data leakage tracing, and topology restoration.

[0037] DPU components are embedded in edge gateways and end-side business servers, and are widely deployed at key data access and processing points in business systems. As a data circulation agent, they sink the information acquisition level from the business and application system side to the internal network boundary, unify the programmable plane, and directly expand the physical range of circulation monitoring points. At the same time, in addition to the source and destination nodes of the data, the monitoring capabilities of the entire data link and all nodes for malicious behaviors such as false data injection, data tampering, and data leakage are enhanced.

[0038] The data flow monitoring method based on the data processing unit provided by the present invention obtains external network data at the edge gateway through the DPU, and obtains business application data and database data at the business terminal, and then builds a data global flow map through the data full-link detection cloud platform. Finally, the data global flow map is used to monitor data flow. This can expand the network security boundary and provide an integrated data flow monitoring system with the DPU as the network axis, so as to realize the underlying logic enhancement for real-time monitoring of abnormal data behavior and source positioning, thereby improving the security of data flow.

[0039] In some embodiments, the DPU includes: NIC subsystem, SoC and rule policy library; The NIC subsystem is used to collect data features; The SoC is used for non-real-time data processing and uploading the processed data to the data full-link detection cloud platform; The rule policy library is used to define data collection rules and data processing rules.

[0040] Specifically, Figure 3 is a schematic diagram of the structure of a DPU deployed on an edge gateway provided by the present invention, Figure 4 It is a structural schematic diagram of the DPU deployed at the service terminal provided by the present invention. As shown in the figure, the DPU is divided into three parts: a network card (NIC) subsystem based on a field programmable gate array (FPGA), a system-on-chip (SoC), and a rule policy library. FPGA assumes the data plane in the form of a NIC subsystem, provides a high-speed data path, and is responsible for data feature collection. The collected data mainly covers the upper-layer business information collected from the kernel interface, that is, business application data, the data call information collected from the database dedicated interface, that is, database data, and the traffic information collected from the network interface, that is, external network data. Among them, the network traffic information is collected by the edge DPU built into the edge gateway, and the database call information and the upper-layer business information are collected by the end-side DPU.

[0041] The NIC subsystem performs real-time data operations on the above information, including load verification, protocol parsing, instruction parsing, and traffic feature extraction, providing a basis for further data processing and analysis on the control plane. The SoC uses the LINUX operating system and assumes the control plane function. While managing the local DPU function, it performs non-real-time data processing and uploads the processed data to the data full-link detection platform. The rule policy library is located in the DPU external memory and is responsible for defining data collection rules and data processing rules.

[0042] Furthermore, DPU components are widely deployed at key network nodes and business system key points, responsible for: Upper-layer business information collected from kernel interfaces, including system behavior context logs and data flow information, as well as packet content-level information detected using deep packet inspection technology, including malware, specific data, and application types; Data call information collected from the database dedicated interface, including data demander information and operation information such as adding, deleting, modifying, and searching data in the database, to ensure that data tags run through the entire life cycle of the data; The traffic information collected from the network interface includes network information at three granularities: packet level (such as end-to-end delay, jitter, throughput, queue occupancy, link utilization, hop delay, timestamp, node ID and other performance parameters), data flow level (such as packet delay distribution, flow path changes, etc.), and event level (such as congestion events and path change events).

[0043] The NIC subsystem parses various pre-defined protocol headers in real time, and while performing the basic function of data forwarding, it provides a basis for further data processing and analysis on the control plane.

[0044] To obtain business application data, the DPU is deployed at key data access and processing points of the business system to collect system behavior context logs and data flow information. The upper-layer business application calls the system kernel interface through a dedicated API to send data packets to the DPU data plane. The DPU supports deep packet inspection technology (DPI). On the basis of analyzing the content below layer 4 of the IP packet (including source address, destination address, source port, destination port, and protocol type), it adds application layer analysis to identify various applications and their contents. When IP data packets, TCP or UDP data streams pass through the DPU, the DPU's NIC subsystem reorganizes the application layer information in the OSI seven-layer protocol by deeply reading the content of the IP packet payload to obtain the content of the entire application, and then shapes the traffic according to the management policy defined by the system.

[0045] The business application data acquisition module of this system supports the following three identification functions: 1. Recognition based on “feature words”

[0046] Different applications usually rely on different protocols, and different protocols have their own special fingerprints, which may be specific ports, specific strings or specific bit sequences. Identification based on "signature words" determines the application carried by the service flow by detecting the "fingerprint" information in specific data packets in the service flow. According to the specific detection method, the recognition technology based on "signature words" can be divided into three categories: fixed position signature word matching, variable position signature matching and state signature matching. By upgrading the "fingerprint" information, feature-based recognition can easily expand functions and realize the detection of new protocols.

[0047] 2. Application layer gateway identification

[0048] The control flow and business flow of some services are separated. When the business flow has no characteristics, an application layer gateway is needed to identify it. The application layer gateway needs to identify the control flow first, and parse it through a specific application layer gateway according to the protocol of the control flow, and identify the corresponding business flow from the protocol content. For each protocol, a different application layer gateway is required to analyze it.

[0049] 3. Behavioral pattern recognition

[0050] Behavior pattern recognition is based on the analysis of the terminal's existing behaviors to determine the user's current or upcoming actions. Behavior pattern recognition is usually used to identify services that cannot be determined based on protocols, such as SPAM (spam) service flows and ordinary Email service flows.

[0051] The collection method used by this system requires that the application types must be known to the system, which means that an application feature database needs to be maintained in the business server. Whenever business traffic passes through the DPU, the DPU compares the unpacked application information with the feature database to determine the business type. When a new application appears, the feature database also needs to be updated accordingly.

[0052] For obtaining external network data, the emergence of programmable data planes provides a new design space for network telemetry solutions. Different from traditional network measurement, this system supports in-band network telemetry (INT) technology, combining packet forwarding with network measurement. The forwarding nodes (such as gateways) deployed with DPU collect the internal state of the network and insert telemetry packets, which has the advantages of strong real-time performance, fine measurement granularity and rich measurement status.

[0053] In this system, the DPU data plane can directly collect and report network status information without the participation of the control plane. In addition, this system supports multiple measurement objects covering multiple granularities, including: Packet-level network status: including traditional end-to-end delay, jitter, throughput, queue occupancy, link utilization, hop delay, timestamp, node ID and other performance parameters. This granular data can provide network administrators with refined network indicators and lay the foundation for handling network failures and performing traffic engineering.

[0054] Flow-level network status: For specific data flows, such as the delay distribution of packets within the flow, changes in the flow path, etc., data at this granularity can be tracked and processed for specific applications to provide support for optimizing the application experience.

[0055] Event-level network status: Focus on important events that affect network performance, such as congestion events and path change events. This granularity of data monitors the entire network and can quickly locate the location of network events. In addition, by accessing the internal data of the switch, the system can collect more detailed network information such as queue length and transmitted traffic.

[0056] Figure 5 This is a flow chart of external network data collection provided by the present invention, such as Figure 5 As shown, host 1 sends a data packet to host 2. Switch 1 deployed with DPU as the INT source node inserts the INT header containing the INT instruction and the corresponding telemetry information into the data packet after receiving it. Switch 2 deployed with DPU as the INT intermediate node embeds the corresponding telemetry information into the data packet according to the INT instruction therein after receiving the data packet. Switch 3 as the INT sink node first inserts the telemetry information into the received data packet, then extracts the INT header and INT metadata from the data packet, generates a telemetry report and forwards it to the data full-link monitoring platform, and the data packet continues to be forwarded to host 2.

[0057] Regarding the transmission of telemetry commands and telemetry information, this system adopts the operation mode INT-MD (INT eMbed data) proposed in INTspecv2.1, embedding both INT commands and INT data into data packets, making it easier to adjust the measurement object and achieving more flexible and fine-grained telemetry.

[0058] For obtaining database data, in order to ensure that the DPU can fully monitor the flow of raw data in the database and make full use of the inherent security capabilities of the DPU, this system requires that all access operations of the server and other switching devices in the network to a specific database must be performed through the dedicated data interface provided by the DPU. The DPU directly connects to the raw data in the database by accessing the database dedicated data interface. The DPU will record the data demander and generate a data behavior log, covering operations such as adding, deleting, modifying, and searching data in the database, to ensure that data tags run through the entire life cycle of the data.

[0059] The data flow monitoring method based on the data processing unit provided by the present invention uses the DPU as the data flow agent on the network card side, sinks the information acquisition level from the business and application system side to the internal network boundary, comprehensively applies the packet parsing and traffic analysis technologies represented by DPI and INT, and constructs a reliable and parsable dedicated data interface for the database, so as to collect the full amount of data from the network plane and upper-level data applications more completely and efficiently.

[0060] In some embodiments, the non-real-time data processing includes one or more of the following: Data fusion; Data modeling; Data flow control; Data preprocessing.

[0061] Specifically, the control plane is undertaken by the CPU of the built-in Linux system, which is responsible for preprocessing the information collected by the data plane. Non-real-time data processing includes data fusion, data modeling, data flow control, and data preprocessing. Preprocessing includes data cleaning, data alignment, data desensitization, etc. After processing, the data is handed over to the cloud platform through a dedicated network interface to build a circulation map.

[0062] Among them, for data fusion, in the context of a significant increase in network data volume and multi-source heterogeneity in data circulation, although the DPU deployed in each network node can record and describe detailed information about the current network status in real time, these log information have problems such as different structures and time asynchrony, and are multi-dimensional and multi-granular.

[0063] In order to effectively support upper-layer applications such as security situation awareness at the data level, the DPU control plane needs to fuse the information generated by the data plane. Figure 6 is a data fusion flow chart provided by the present invention, such as Figure 6 As shown in the figure, the data fusion module is divided into four levels: data acquisition, data preprocessing, feature extraction and data fusion classification: Data acquisition mainly involves the collection and acquisition of relevant basic data by the data plane.

[0064] Data preprocessing implements preprocessing such as eliminating redundant and invalid data.

[0065] Feature extraction counts the frequency of each record between logs and combines them into their own feature space to evaluate the importance of the record.

[0066] Data fusion will reduce and combine similar message contents into a single rich text message based on the message contents in the data set, and finally upload this message to the cloud platform for data link analysis.

[0067] Data cleaning aims to detect and correct erroneous data, inaccurate data, inconsistent and incomplete data collected by the data plane to improve the quality and reliability of data, including processing missing values, data deduplication, outlier detection and processing, etc. First, for fuzzy data and inconsistent data, this system uses data source importance measurement, data attributes and tuples for weight labeling and classification, and finally uses data probability values ​​to complete the classification and cleaning of fuzzy data; secondly, for a large amount of redundant and repeated data, this system completes the cleaning of repeated data by building a data attribute standard library, attribute reduction and multiple sorting.

[0068] In the network, the release of certain data may expose sensitive patterns that model confidential knowledge. The data desensitization module aims to remove sensitive patterns in the data and prevent the leakage of sensitive information after data circulation and sharing, so as to ensure the security of private information while maintaining the original data characteristics, business rules and data relevance. This module describes sensitive information in the form of strings and desensitizes the byte stream by hiding the occurrence of sensitive patterns. The goal of this module is to generate a string of minimum length that retains the order and frequency of occurrence of all non-sensitive patterns. Such a string can accurately perform tasks based on the sequential nature and pattern frequency of the string. At the same time, in order to prevent the constructed string from detecting sensitive patterns through the "reverse" connection operation, this module encrypts the letters in the string output by the algorithm to avoid reverse recovery of sensitive patterns.

[0069] The data flow monitoring method based on the data processing unit provided by the present invention aims at the CPU performance bottleneck problem in high-bandwidth networks. The DPU is used as the data flow agent on the network card side, and a multi-party data feature collection method with cloud-edge collaboration is designed. By sinking the information acquisition level from the business and application system side to the internal network boundary, the full amount of data from the network plane and upper-level data applications is collected more completely and efficiently, releasing the CPU network traffic calculation load.

[0070] In some embodiments, the data full-link detection cloud platform further includes a DPU management module; The DPU management module is used to obtain the external network data, the business application data and the database data.

[0071] Specifically, Figure 7 It is a structural diagram of the data full-link detection cloud platform provided by the present invention, and the data full-link detection cloud platform also includes a DPU management module.

[0072] On the one hand, the DPU management module is responsible for data transmission with all DPUs, and aggregates multi-party data collected by the DPU through a dedicated interface; on the other hand, it is responsible for comprehensive and efficient management of DPUs distributed at the edge and end, covering device policy configuration, operation status monitoring, edge maintenance and upgrades, etc., to ensure stable and reliable operation of the entire system.

[0073] First, the DPU management module is responsible for device configuration, including initialization and network parameter configuration of the DPU on the edge gateway and business server when the device is deployed, and uses its own monitoring and logging system to integrate the DPU log into the monitoring framework of the entire system to meet the management and maintenance needs of the system. Through a unified management interface, cloud administrators can easily centrally manage a large number of DPUs, improving the consistency of system configuration and the convenience of management.

[0074] Secondly, by real-time monitoring of key indicators such as the operating status, resource utilization, and traffic of edge and end DPUs, backend administrators can promptly identify potential problems and take corresponding measures, including DPU troubleshooting, error repair, and log management, to ensure long-term stable operation of the system. At the same time, it reduces the need for on-site processing due to failures and reduces maintenance costs.

[0075] Finally, the DPU management module is also responsible for the upgrade management of the DPU. When the cloud model is updated and the DPU acquisition and control strategy needs to be upgraded synchronously, the cloud administrator can update the DPU rule policy library while the system is running to obtain the latest functions, performance improvements and security patches to ensure the performance of the system at different stages.

[0076] Furthermore, the data full-link detection cloud platform can generate a global data circulation map based on the information collected by the DPU from the edge gateway and business server. Using the PROV-DM model, the circulation map is defined as the set of all subjects, objects and events within the system monitoring network, expressed as G=(S;O;E). Among them, S represents the subject set, which is the entity that operates on another entity in the system, including processes, threads, etc.; O represents the object set, which is the entity being operated, including files, sockets, etc.; E represents the event set, represented by the four-tuple <subject; object; time; operation>. This system converts the data features collected by the DPU into a data management module, and uses filters to remove redundant events in the original data according to the edge compression algorithm and the node compression algorithm to reduce the storage pressure of the circulation map and improve the search efficiency. The circulation map generated by the system supports the traceability of a specific data and a specific granularity (including data packets, flows, files, etc.), and can restore the topology of the data in the network environment.

[0077] In addition, this module provides a data query interface for upper-layer applications in the cloud platform, supporting two access modes: frequent queries and persistent access, so that the circulation graph can support various traceability requirements. The data query interface supports the following functions: Causal tracking: Data circulation has strong temporal and spatial correlation and causality, so the circulation spectrum has backward tracking and forward tracking capabilities.

[0078] Pattern matching: Graph pattern matching is one of the core functions of graph query, which can represent attack behaviors through semantics in threat detection.

[0079] Stream-based query: Supports real-time event input from multiple hosts to reduce query response latency.

[0080] The data flow monitoring method based on the data processing unit provided by the present invention expands the network security boundary through the integrated modeling of network security and multi-dimensional characteristics of data security. On the one hand, it can realize computing acceleration and security protection capability enhancement for the current computing system. On the other hand, by adapting and integrating the computing power and security protection resources of decentralized construction, an integrated data flow monitoring system with DPU as the network axis is constructed.

[0081] In some embodiments, the monitoring of data circulation based on the global data circulation map includes one or more of the following: Data topology restoration; Data breach tracing; Security situation awareness.

[0082] Specifically, in a complex network, participants may not directly share their own data circulation related information with a third-party cloud platform for the purpose of protecting privacy data, resulting in incomplete and fuzzy data collected based on distributed DPU data features, making it difficult to directly construct a global data circulation map. Homomorphic encryption is an effective means of privacy protection. This system supports a topology multi-party restoration method based on homomorphic encryption, which can balance the needs of privacy protection and accuracy in network topology construction. The specific steps are as follows: Use homomorphic encryption algorithms to encrypt network data to ensure that sensitive information is effectively protected during network data transmission and calculation. Each participant can use his or her own key to encrypt network data.

[0083] Homomorphic computing functions are introduced to perform calculations in an encrypted state without first decrypting the data. This ensures that participants can perform calculations together without exposing sensitive information. Under the guidance of the cloud platform, homomorphic computing is used to aggregate multi-party network data to generate encrypted intermediate results. The cloud platform only obtains the encrypted intermediate results and cannot access the participants' plaintext data.

[0084] The cloud platform uses a homomorphic decryption function to decrypt the encrypted intermediate results and obtain a global data circulation map. This step is only performed in the final result generation stage, which protects the privacy of participants to the greatest extent.

[0085] The data leakage tracing module is designed to trace, confirm, describe, analyze and finally save the dynamic process of the source data of specific target data and its subsequent evolution. It mainly involves the tracking and description of the source data of specific target data, the tracing and recording of the information of the whole stage of the source data evolution process, including the movement, evolution, execution, dissemination and communication of target data, and the derived data generated in the whole process. Starting from an attack point / leakage point on the data circulation map, a reverse search is performed on the map to find all the nodes that can reach the attack point. The path formed by these nodes on the circulation map is the possible attack path.

[0086] The system first extracts the pre-built data flow graph as a feature sequence; then uses a codec model (GRU) combined with a local attention mechanism to train a sequence feature extractor to extract data leakage features; finally, attack behavior modeling is performed, and a clustering algorithm is used to establish K cluster centers for the normal behavior of the system, and data far away from the cluster center will be identified as attack behavior.

[0087] The security situation awareness module combines the multi-source heterogeneous environment data provided by security sensors deployed at all levels of data circulation, including security log information and network traffic, and combines with cloud security situation information platforms, such as threat intelligence platforms, asset management platforms, EDR (endpoint detection and response) platforms, vulnerability databases, etc., to actively discover and perceive security risks based on behavior analysis, data content analysis, and traffic analysis. It comprehensively considers factors that reflect changes in security situation, expands detection objects from known threats to unknown threats, obtains, associates, understands and displays various network security situation elements from a global and dynamic perspective, and conducts situation analysis, judgment, early warning and disposal of the current network security situation and possible threats in the future, so as to improve the ability to discover, identify, analyze, predict, respond and deal with network security threats.

[0088] The data flow monitoring method based on the data processing unit provided by the present invention aims at the unknown attack risk problem faced by the data flow process, and designs a cloud-edge data flow global monitoring system. By constructing a global data flow map, the network security boundary is expanded, and an integrated data flow monitoring system with DPU as the network axis is provided to achieve underlying logic enhancement for real-time monitoring of abnormal data behavior and source location.

[0089] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data flow monitoring method based on a data processing unit, characterized in that: include: Acquire external network data at the edge gateway based on the data processing unit DPU, and acquire business application data and database data at the business terminal based on the DPU; Based on the data global circulation map construction module, feature extraction is performed on the external network data, the business application data and the database data on the data full-link detection cloud platform to construct a data global circulation map; The data circulation is monitored based on the global data circulation map.

2. The data flow monitoring method based on the data processing unit according to claim 1 is characterized in that: The DPU includes: NIC subsystem, SoC and rule policy library; The NIC subsystem is used to collect data features; The SoC is used for non-real-time data processing and uploading the processed data to the data full-link detection cloud platform; The rule policy library is used to define data collection rules and data processing rules.

3. The data flow monitoring method based on the data processing unit according to claim 2 is characterized in that: The non-real-time data processing includes one or more of the following: Data fusion; Data modeling; Data flow control; Data preprocessing.

4. The data flow monitoring method based on a data processing unit according to claim 1, characterized in that: The data full-link detection cloud platform also includes a DPU management module; The DPU management module is used to obtain the external network data, the business application data and the database data.

5. The data flow monitoring method based on a data processing unit according to claim 1, characterized in that: The monitoring of data circulation based on the global data circulation map includes one or more of the following: Data topology restoration; Data breach tracing; Security situation awareness.

6. A data flow monitoring system based on a data processing unit, characterized in that: include: DPU deployed on edge gateways, DPU deployed on business terminals, and data full-link detection cloud platform; The DPU deployed on the edge gateway is used to obtain external network data; The DPU deployed in the service terminal is used to obtain service application data and database data; The data full-link detection cloud platform is used to extract features from the external network data, the business application data and the database data, construct a data global circulation map, and monitor data circulation based on the data global circulation map.

7. The data flow monitoring system based on the data processing unit according to claim 6, characterized in that: The DPU includes: NIC subsystem, SoC and rule policy library; The NIC subsystem is used to collect data features; The SoC is used for non-real-time data processing and uploading the processed data to the data full-link detection cloud platform; The rule policy library is used to define data collection rules and data processing rules.

8. The data flow monitoring system based on the data processing unit according to claim 7 is characterized in that: The SoC includes one or more of the following modules: Data fusion module; Data modeling module; Data flow control module; Data preprocessing module.

9. The data flow monitoring system based on the data processing unit according to claim 6, characterized in that: The data full-link detection cloud platform also includes a DPU management module; The DPU management module is used to obtain the external network data, the business application data and the database data.

10. The data flow monitoring system based on the data processing unit according to claim 6, characterized in that: The data full-link detection cloud platform is used to monitor one or more of the following data: Data topology restoration; Data breach tracing; Security situation awareness.

Citation Information

Patent Citations

  • Hierarchical reduction combined cleaning method for multi-source heterogeneous big data

    CN108959395A

  • Computer data processing system and method based on data fusion

    CN111597399A

  • Security situation awareness processing method, system and equipment with built-in data processing unit

    CN117914547A