Method, device and medium for protecting full-link of API level data flow

Abstract

The application relates to the technical field of data processing, and provides an API-level data flow transfer full-link protection method and device, equipment and a medium, which can collect application layer business log data and kernel state network flow data as to-be-processed data in real time by adopting a non-buried point mechanism, realizes non-intrusive full-amount data collection, identifies to-be-protected data by adopting a non-feature data supplement identification mechanism, improves the accuracy and generalization ability of data identification, solves the problem that non-feature data cannot be controlled, constructs an API-level full-link data flow transfer map according to the to-be-protected data and the to-be-processed data, provides a complete topology basis for subsequent risk detection and tracing, performs risk detection according to the API-level full-link data flow transfer map, improves the effectiveness of risk detection, performs bidirectional tracing according to the detection result and the API-level full-link data flow transfer map, obtains a tracing report, can quickly locate a data anomaly source and an influence range, and provides a complete evidence chain.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for end-to-end protection of API-level data flow. Background Technology

[0002] With the rapid development of digital business, the internal data flow path of enterprises is becoming increasingly complex, and data security risks are also showing characteristics of being hidden, cross-level, and end-to-end.

[0003] However, existing data protection methods still have the following shortcomings: (1) Data collection relies on event tracking: Most existing solutions rely on event tracking, SDK (Software Development Kit) implantation, or proxy gateways to collect data, which is highly intrusive to business operations and has high maintenance costs; (2) Insufficient ability to identify sensitive data and difficulty in managing featureless data: Traditional data identification schemes mostly rely on fixed regular expression matching to identify privacy data such as ID card numbers and mobile phone numbers. For business data without fixed characteristics such as financial flow, order amount, and operational indicators, there is a lack of effective identification methods. (3) Incomplete data flow chain: Existing solutions can usually only record user access logs or database operation logs, lacking cross-level correlation capabilities. They cannot fully correlate user access behavior, API (Application Programming Interface) calls, database operations, data field access, etc., and therefore cannot achieve API-level full-link tracing. (4) Risk detection relies on static rules: Traditional risk detection is mostly based on preset rules (such as IP (Internet Protocol) whitelists, access frequency thresholds, etc.), which makes it difficult to detect hidden risks and has a high false alarm rate; (5) It only supports one-way tracing, making it difficult to quickly locate the source of the anomaly and the scope of its impact.

[0004] Therefore, it is necessary to provide a method for end-to-end protection of API-level data flow. Summary of the Invention

[0005] In view of the above, it is necessary to provide a method, device, equipment and medium for full-link protection of API-level data flow, which aims to solve the problem of the inability to effectively protect the data flow process.

[0006] A method for end-to-end protection of API-level data flow, the method comprising: A no-code sampling mechanism is used to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed. A featureless data supplementation and identification mechanism is used to identify the data to be protected in the data to be processed. Construct an API-level end-to-end data flow map based on the data to be protected and the data to be processed. Risk detection is performed based on the API-level end-to-end data flow map to obtain the detection results; Based on the detection results and the API-level full-link data flow map, bidirectional tracing is performed to obtain a tracing report.

[0007] An API-level data flow end-to-end protection device, the API-level data flow end-to-end protection device comprising: The data acquisition unit is used to collect application layer business log data and kernel-mode network traffic data in real time using a no-code tracking mechanism as data to be processed. The identification unit is used to identify the data to be protected in the data to be processed using a featureless data supplementation identification mechanism; The construction unit is used to construct an API-level end-to-end data flow map based on the data to be protected and the data to be processed. The detection unit is used to perform risk detection based on the API-level full-link data flow map and obtain the detection results; The tracing unit is used to perform bidirectional tracing based on the detection results and the API-level full-link data flow map to obtain a tracing report.

[0008] A computer device, the computer device comprising: A memory for storing at least one instruction; and a processor for executing the instructions stored in the memory to implement the API-level data flow end-to-end protection method.

[0009] A computer-readable storage medium storing at least one instruction, which is executed by a processor in a computer device to implement the API-level data flow end-to-end protection method.

[0010] As can be seen from the above technical solutions, this invention can use a no-code sampling mechanism to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed, achieving non-intrusive full-volume data collection; it adopts a featureless data supplementation and identification mechanism to identify data to be protected in the data to be processed, improving the accuracy and generalization ability of data identification and solving the problem of uncontrollable featureless data; it constructs an API-level full-link data flow map based on the data to be protected and the data to be processed, providing a complete topological foundation for subsequent risk detection and tracing; it performs risk detection based on the API-level full-link data flow map, improving the effectiveness of risk detection; and it performs bidirectional tracing based on the detection results and the API-level full-link data flow map, obtaining a tracing report, which can quickly locate the source and scope of data anomalies and provide a complete chain of evidence. Attached Figure Description

[0011] Figure 1 This is a flowchart of a preferred embodiment of the API-level data flow end-to-end protection method of the present invention; Figure 2 This is a functional block diagram of a preferred embodiment of the API-level data flow end-to-end protection device of the present invention; Figure 3 This is a schematic diagram of the structure of a computer device that is a preferred embodiment of the method for implementing full-link protection of API-level data flow according to the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the API-level data flow end-to-end protection method of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0014] The API-level data flow end-to-end protection method is applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0015] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), interactive network television (IPTV), smart wearable device, etc.

[0016] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0017] The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0018] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0019] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0020] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0021] S10 uses a no-code sampling mechanism to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed.

[0022] In this embodiment, the application layer business log data may include, but is not limited to: user access logs, API (Application Programming Interface) call logs, business operation logs, database access logs, etc.

[0023] The user access log may include, but is not limited to: user identifiers (such as user ID, account, role, terminal ID), client information (such as IP address, device type, operating system, browser information, etc.), request metadata (such as user request identifier, tracking ID, session ID, request time, request status code, etc.).

[0024] The API call log may include, but is not limited to: basic API information (such as request path, request method, interface name, service name), request parameters (such as URL (Uniform Resource Locator) parameters, form parameters, request body fields), response data (such as response status code, response time, response body fields (including returned data values, data types)), etc.

[0025] The business operation log may include, but is not limited to: business events (such as user login or logout, order creation or payment, data query, modification or export), operation results (such as success or failure indicators, error codes, error messages), and permission information (such as operation roles, permission change records, approval process information), etc.

[0026] The database access log may include, but is not limited to: SQL (Structured Query Language) statements initiated by the business, database connection information (such as database name, table name, connection session ID), operation results (such as the number of rows affected, execution time, transaction commit or rollback status), etc.

[0027] In this embodiment, the kernel-mode network traffic data may include, but is not limited to: TCP / IP (Transmission Control Protocol / Internet Protocol) layer traffic data, application layer plaintext payload data, process or thread context data, SSL / TLS (Secure Sockets Layer / Transport Layer Security) handshake information, etc.

[0028] The TCP / IP layer traffic data may include, but is not limited to: five-tuple information (such as source IP, source port, destination IP, destination port, protocol type), connection status (such as handshake packet, connection establishment or disconnection event, TCP sequence number or acknowledgment number), traffic statistics (such as packet size, transmission time, transmission volume, number of retransmissions), etc.

[0029] The application layer plaintext payload data (including the decryption results of encrypted traffic) may include, but is not limited to: HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol Secure) requests (such as request headers, request bodies, response headers, and response bodies), RPC (Remote Procedure Call) protocol data (such as frame headers, message bodies, and serialized data), database protocol data, etc.

[0030] The process or thread context data may include, but is not limited to: process information (such as process ID, process name, user ID, and service to which it belongs), thread or coroutine information (such as thread ID, coroutine ID, and call stack information), system call information, etc.

[0031] The SSL / TLS handshake information may include, but is not limited to: SSL session ID, handshake status, cipher suite version, SNI (Server Name Indication) field, certificate information (such as certificate fingerprint, validity period, issuing authority), etc.

[0032] The application layer business log data is used to provide business semantics and user behavior context, while the kernel-mode network traffic data is used to provide underlying transmission details and encrypted plaintext data. The two can be associated through request ID, 5-tuple, or process ID to form a complete link data from user to data object, which solves the technical defect of incomplete data collection in traditional solutions.

[0033] In this embodiment, the real-time collection of application layer business log data and kernel-mode network traffic data as data to be processed using a no-code sampling mechanism includes: The application layer business log data is collected in real time using an intelligent agent plugin deployed on the application system server; The kernel-mode eBPF probe is used to capture socket data packets in real time as the kernel-mode network traffic data; The application layer service log data and the kernel-mode network traffic data are correlated and aligned to obtain the data to be processed.

[0034] Among them, a lightweight Agent plugin is deployed on the business application server. Without modifying the business code or adding tracking points, application layer logs can be collected in real time through process injection or file system awareness.

[0035] One approach is to load the eBPF (Extended Berkeley Packet Filter) program into the operating system kernel and attach kernel probe hooks to system calls and socket events, thereby achieving non-intrusive traffic collection.

[0036] To reduce invalid data transmission, preliminary filtering of traffic can also be performed in kernel mode (such as capturing only traffic generated by business processes and excluding traffic from non-business ports).

[0037] To avoid the overhead of context switching in business processes, the captured kernel-mode network traffic data can also be asynchronously transmitted to the user-mode agent through a zero-copy ring buffer.

[0038] The socket data packets collected by the kernel-mode eBPF probe also need to be analyzed in real time using CFG (Control Flow Graph) to capture SYN / ACK (Synchronize Sequence Numbers / Acknowledgment) and SSL handshake information, reverse engineer the API definition, and perform AST (Abstract Syntax Tree) parsing on the SQL execution statements to generate syntactic fingerprints, thereby generating API assets and associating the API with the SQL, providing a foundation for subsequent processing.

[0039] After collecting the application layer service log data and the kernel-mode network traffic data, the association and matching between the two can be achieved based on the user request identifier or the 5-tuple. Specifically, if the application layer service log data includes the user request identifier, the TCP stream carrying the user request identifier is searched in the kernel-mode network traffic data, and a one-to-one correspondence is established; if the application layer service log data does not include the user request identifier, the association can be based on process ID, thread ID, timestamp window, or API request path, request body hash, source end of TCP stream, destination end of TCP stream, etc.

[0040] Furthermore, the associated data can be encapsulated into a unified format to obtain the data to be processed.

[0041] Through the above embodiments, non-intrusive and comprehensive data collection can be achieved through a no-code point mechanism.

[0042] S11, a featureless data supplementation and identification mechanism is used to identify the data to be protected in the data to be processed.

[0043] In this embodiment, the data to be protected may include fixed features such as mobile phone numbers and featureless data such as business data.

[0044] In this embodiment, the identification of the data to be protected in the data to be processed using the featureless data supplementation identification mechanism includes: Extract the data with features and the data without features from the data to be processed; The characteristic data is classified and graded using a pre-constructed classification and grading template to obtain the first data to be protected; Obtain the target API interface corresponding to the featureless data, and obtain the protection data identifier field corresponding to the target API interface; Extract the field content corresponding to the protected data identifier field from the featureless data; The second data to be protected is obtained from the featureless data based on the content of the field; The first data to be protected and the second data to be protected are integrated to obtain the data to be protected.

[0045] The classification and grading template can be configured according to the data classification and grading specifications in industry standards.

[0046] The protected data identifier field can be a defined extended field or a request header field.

[0047] When the content of the field is a predefined field, it can be determined as the second data to be protected.

[0048] In the above embodiments, the featureless data supplementation and identification mechanism can realize the full-type identification of the data to be protected, and avoid the omission of featureless data.

[0049] In this embodiment, to further reduce the false negative rate, for featureless data that is not identified as the second data to be protected through the protection data identification field, further verification can be performed, specifically including: The model is based on a pre-trained Transformer encoder and fine-tuned using a small number of manually labeled featureless data samples (such as financial statements and operational indicators) to enable the model to learn the mapping relationship between field context sequences and business data types and levels, so that the model can extract business-related semantic features from field names, descriptions, API paths, and value examples. For featureless data that is not identified as the second data to be protected through the protection data identifier field, the Transformer encoder extracts the context semantic vector of the featureless data and outputs a high-dimensional context semantic vector for each field. This vector contains the business semantics, API path information and implicit information of the initial label of the field. Obtain the OpenAPI Specification (Swagger) interface document, parse the OpenAPI Specification interface document to obtain the parsed fields; vectorize the parsed fields to obtain the Swagger parsing features; Extract behavioral characteristics such as historical access frequency, access subject type, data transmission volume, whether it is batch export, and whether it is cross-system transmission from the corresponding API fields in the historical data stream, and construct historical flow frequency characteristics based on the behavioral characteristics; By using an attention mechanism, the high-dimensional contextual semantic vector of each field, the Swagger parsing features, and the historical flow frequency features are weighted and fused to obtain a multimodal feature vector that simultaneously integrates core semantic features, business definition features, and behavioral statistical features. The multimodal feature vectors are input into a trained classification model (such as a fully connected neural network or support vector machine, trained using historical multimodal feature vectors as training samples) to output the prediction result and confidence level of the featureless data. When the confidence level is greater than or equal to a preset threshold (e.g., ≥90%), the classification and grading labels are directly output.

[0050] The above embodiments enable further verification of the identification results of the featureless data supplementation identification mechanism, thereby improving the accuracy of identification.

[0051] S12, construct an API-level full-link data flow map based on the data to be protected and the data to be processed.

[0052] In this embodiment, the API-level end-to-end data flow map can be converted into a visual topology map to facilitate an intuitive display of various flow relationships.

[0053] In this embodiment, constructing an API-level end-to-end data flow map based on the data to be protected and the data to be processed includes: Extract user request identifiers from the application layer business log data, and associate the user layer with the application layer based on the user request identifiers to obtain a user-API association mapping table; The structured query language (SQL) statements in the kernel-mode network traffic data are parsed to obtain the parsing results; The parsing results are matched with the API requests in the application layer business log data to obtain matching results; The database information bound to the structured query language statement is determined based on the database connection information in the kernel-mode network traffic data. An API-SQL association mapping table is established based on the matching results and the database information bound to the structured query language statement; Bind the data to be processed with the structured query language statement to obtain an SQL-data field association mapping table; wherein, the SQL-data field association mapping table includes annotations added based on the data to be protected; The user-API association mapping table and the API-SQL association mapping table are linked in chronological order to construct the user access link; The API-SQL association mapping table and the SQL-data field association mapping table are linked together according to the API call order to construct the application call chain; By integrating the user access link and the application call link, a four-layer interconnected API-level full-link data flow map is obtained, comprising the user layer, application layer, database layer, and data object layer.

[0054] Specifically, when linking the user-API association mapping table and the API-SQL association mapping table in chronological order, all API requests of a user can be linked in chronological order by user ID, and the link information can be completed to form the user access link.

[0055] In the process of connecting the API-SQL association mapping table and the SQL-data field association mapping table according to the API call order, the API can be used as the starting point to connect the services, SQL operations, and data objects it calls, and complete the link information to form the application call link.

[0056] Through the above embodiments, the four-layer full-link connection of user layer, application layer, database layer and data object layer is realized, while covering user access link and application call link, without breakpoints and blind spots, providing a complete API-level data flow view for subsequent risk detection and two-way traceability.

[0057] S13, perform risk detection based on the API-level full-link data flow map to obtain the detection results.

[0058] In this embodiment, the risk detection based on the API-level end-to-end data flow map, and the resulting detection results, include: The anomaly score for each link in the API-level end-to-end data flow map is calculated using the isolated forest algorithm. For risk links with abnormal scores greater than a preset threshold, a compliance baseline map constructed based on domain knowledge and historical data is obtained; Based on the compliance baseline map, baseline deviation detection is performed on the risk links to obtain the risk types; The detection results are generated based on the risk link and the risk type.

[0059] The isolated forest algorithm calculates the anomaly score for each link based on the principle that outliers are more easily isolated.

[0060] The preset threshold can be configured according to actual accuracy requirements.

[0061] Specifically, when detecting baseline deviations in the risk links based on the compliance baseline map, if an API path not present in the baseline map appears in the real-time link, it can be marked as "New Unregistered API Risk"; if the accessing entity's IP is not within the whitelist allowed by the baseline (e.g., overseas IP, non-office network segment IP), or the user role does not have access to the API, it can be marked as "Unauthorized Access / Abnormal Access Risk"; if the destination IP or domain name of the API request is an overseas address, and the transmitted data contains data to be protected, it can be marked as "Data Outbound Risk"; comparing the SQL statements in the real-time link with the standard SQL fingerprint in the baseline, if non-business SQL (e.g., full table scans without filtering conditions, batch export statements) is found, it can be marked as "High-Risk SQL Operation Risk"; if the data flow does not match the baseline (e.g., user data flows to an unregistered third-party system), it can be marked as "Abnormal Data Flow Risk".

[0062] To enhance the depth of baseline drift detection, the SQL statements in the API-level end-to-end data flow map can be parsed using an Abstract Syntax Tree (AST). Constant values ​​are removed, retaining core structures such as table names, field names, operation types, and filtering conditions. A depth-first traversal is performed on the AST, outputting node types and key node values ​​in a fixed order to generate a structured sequence. A fixed-length hash value is calculated from this structured sequence to obtain the SQL syntax fingerprint, which is then compared with the standard SQL fingerprint of the corresponding API in the compliance baseline map. If fingerprint matching fails, or the parsing result shows one of the following types, it is marked as a high-risk operation: full table scan, batch export, or unauthorized operation. It is also marked as a "non-business SQL high-risk operation," and the corresponding data fields are associated with the data tags to be protected.

[0063] To reduce the false alarm rate, a training set can be periodically constructed based on the accuracy of the detection results using a feedback mechanism to update the isolated forest and the compliance baseline map.

[0064] Different risk types can correspond to different risk levels, and standardized alarm information can be generated based on different risk levels.

[0065] The above embodiments enable the discovery of hidden risks that traditional rules cannot identify, significantly reducing the false alarm rate and achieving accurate alerts.

[0066] S14. Based on the detection results and the API-level full-link data flow map, perform bidirectional tracing to obtain a tracing report.

[0067] In this embodiment, the step of performing bidirectional tracing based on the detection results and the API-level end-to-end data flow map to obtain a tracing report includes: The abnormal entity is identified based on the detection results, and the source tracing direction is determined based on the abnormal entity. When the tracing direction is the user access behavior direction, locate all links associated with the detection result in the API-level full-link data flow map under the user index dimension, and extract the user access path from all links; and / or When the tracing direction is the data being called direction, locate all links associated with the detection result in the API-level full-link data flow map under the data index dimension, and extract the data being called path from all links; The source tracing report is generated based on the user access path and / or the data being invoked path.

[0068] Among these, multidimensional indexes can be pre-built, including: (1) User-dimensional index: All access links are associated through user ID, account, IP, etc.; (2) Data dimension index: associate all access links through data fields, sensitive fingerprints, table names, etc.; (3) Time and API index: Link links and risk events through time windows and API paths.

[0069] The multidimensional index can be stored in time-series databases and graph databases, and supports both structured queries and topological association queries.

[0070] To improve tracing efficiency, after generating the search results, the API-level end-to-end data flow map can be stored in layers based on these results. Specifically, for normal links, only the link fingerprint, color ID, timestamp, API path, and data tag can be stored, and high-frequency repetitive links can be filtered by bitmap frequency, retaining only the summary to reduce storage pressure. For abnormal links, the complete link data is stored, including user ID, IP, API request, response body, SQL statement, call stack, etc., and is marked with a risk event ID.

[0071] Specifically, when determining the abnormal subject based on the detection results, if the abnormality focuses on a person, account, IP address, or terminal, the abnormal subject is determined to be a person, and the tracing direction is from the person to the user's access behavior towards the data; if the abnormality focuses on data, fields, tables, or files, the abnormal subject is determined to be data, and the tracing direction is from the data to the person's data access direction.

[0072] To ensure the integrity of the evidence chain, after obtaining the user access path and / or the data call path, the API-level full-link data flow map, the detection results, the user access path and / or the data call path can be cross-validated with the user request identifier as the core to confirm that the timestamp, user ID, API path, SQL statement, data field, etc. of the link are completely consistent, ensuring that the evidence chain has no breaks.

[0073] Furthermore, the source tracing path can be highlighted in the visualization topology of the API-level full-link data flow map based on the source tracing report, so as to intuitively present the flow process.

[0074] The above embodiments can provide a complete chain of evidence for tracing the source.

[0075] As business continues to develop, outbound interfaces are also changing dynamically. For example, when a new outbound interface is launched, the content transmitted by existing outbound interfaces will change. Due to the complexity of the business system and the large number of interfaces, it is difficult to provide real-time dynamic protection for outbound interfaces.

[0076] Therefore, in this embodiment, after identifying the data to be protected in the data to be processed using the featureless data supplementation identification mechanism, the method further includes: When the data to be protected is outbound data, the outbound rule engine is invoked to detect the destination, data type, sensitivity level, and transmission volume of the outbound data as statistical features of the outbound data; An outbound safety assessment report is generated based on the statistical characteristics of the outbound data.

[0077] The interface for the data to be exported needs to be dynamically protected throughout its entire lifecycle, including changes to the interface status and the fields transmitted, as well as changes to the transmission range, to promptly detect illegal data exports and effectively reduce the risks and costs associated with data exports.

[0078] The above embodiments enable secure management of cross-border data, provide a basis for compliant control of data export, and further enhance enterprises' ability to control data export.

[0079] As can be seen from the above technical solutions, this invention can use a no-code sampling mechanism to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed, achieving non-intrusive full-volume data collection; it adopts a featureless data supplementation and identification mechanism to identify data to be protected in the data to be processed, improving the accuracy and generalization ability of data identification and solving the problem of uncontrollable featureless data; it constructs an API-level full-link data flow map based on the data to be protected and the data to be processed, providing a complete topological foundation for subsequent risk detection and tracing; it performs risk detection based on the API-level full-link data flow map, improving the effectiveness of risk detection; and it performs bidirectional tracing based on the detection results and the API-level full-link data flow map, obtaining a tracing report, which can quickly locate the source and scope of data anomalies and provide a complete chain of evidence.

[0080] like Figure 2 The diagram shown is a functional block diagram of a preferred embodiment of the API-level data flow end-to-end protection device of the present invention. The API-level data flow end-to-end protection device 11 includes a data acquisition unit 110, an identification unit 111, a construction unit 112, a detection unit 113, and a tracing unit 114. The module / unit referred to in this invention refers to a series of computer program segments that can be executed by a processor and perform a fixed function, and are stored in memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0081] The acquisition unit 110 is used to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed using a no-code mechanism. The identification unit 111 is used to identify the data to be protected in the data to be processed using a featureless data supplementation identification mechanism. The construction unit 112 is used to construct an API-level full-link data flow map based on the data to be protected and the data to be processed. The detection unit 113 is used to perform risk detection based on the API-level full-link data flow map and obtain the detection result. The tracing unit 114 is used to perform bidirectional tracing based on the detection results and the API-level full-link data flow map to obtain a tracing report.

[0082] As can be seen from the above technical solutions, this invention can use a no-code sampling mechanism to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed, achieving non-intrusive full-volume data collection; it adopts a featureless data supplementation and identification mechanism to identify data to be protected in the data to be processed, improving the accuracy and generalization ability of data identification and solving the problem of uncontrollable featureless data; it constructs an API-level full-link data flow map based on the data to be protected and the data to be processed, providing a complete topological foundation for subsequent risk detection and tracing; it performs risk detection based on the API-level full-link data flow map, improving the effectiveness of risk detection; and it performs bidirectional tracing based on the detection results and the API-level full-link data flow map, obtaining a tracing report, which can quickly locate the source and scope of data anomalies and provide a complete chain of evidence.

[0083] like Figure 3 The diagram shown is a schematic representation of the structure of a computer device that implements a preferred embodiment of the API-level data flow end-to-end protection method of the present invention.

[0084] The computer device 1 may include a memory 12, a processor 13, and a bus (the arrow in the figure represents the bus), and may also include a computer program stored in the memory 12 and capable of running on the processor 13, such as an API-level data flow end-to-end protection program.

[0085] Those skilled in the art will understand that the schematic diagram is merely an example of computer device 1 and does not constitute a limitation on computer device 1. Computer device 1 can be either a bus topology or a star topology. Computer device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, computer device 1 may also include input / output devices, network access devices, etc.

[0086] It should be noted that the computer device 1 described is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0087] The memory 12 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a portable hard drive of the computer device 1. In other embodiments, the memory 12 can be an external storage device of the computer device 1, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the computer device 1. Furthermore, the memory 12 can include both internal and external storage units of the computer device 1. The memory 12 can be used not only to store application software and various types of data installed on the computer device 1, such as the code of an API-level data flow end-to-end protection program, but also to temporarily store data that has been output or will be output.

[0088] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the computer device 1, connecting various components of the computer device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing API-level data flow full-link protection programs) and calls data stored in the memory 12 to perform various functions of the computer device 1 and process data.

[0089] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes these applications to implement the steps in the above embodiments of the API-level data flow end-to-end protection method, for example... Figure 1 The steps are shown.

[0090] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a data acquisition unit 110, an identification unit 111, a construction unit 112, a detection unit 113, and a tracing unit 114.

[0091] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the API-level data flow end-to-end protection method described in the various embodiments of this invention.

[0092] If the modules / units integrated in the computer device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0093] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0094] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0095] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0096] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The bus is represented by only one straight line, but this does not mean that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0097] Although not shown, the computer device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The computer device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0098] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the computer device 1 and other computer devices.

[0099] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the computer device 1 and to display a visual user interface.

[0100] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0101] It will be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0102] Combination Figure 1 The memory 12 in the computer device 1 stores multiple instructions to implement an API-level data flow end-to-end protection method, and the processor 13 can execute the multiple instructions to achieve the following: A no-code sampling mechanism is used to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed. A featureless data supplementation and identification mechanism is used to identify the data to be protected in the data to be processed. Construct an API-level end-to-end data flow map based on the data to be protected and the data to be processed. Risk detection is performed based on the API-level end-to-end data flow map to obtain the detection results; Based on the detection results and the API-level full-link data flow map, bidirectional tracing is performed to obtain a tracing report.

[0103] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0104] It should be noted that all the data involved in this case was legally obtained.

[0105] If any AI models, software tools, or components not belonging to this company appear in the embodiments of this invention, they are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this invention has been obtained by an entity authorized (with the knowledge and consent) or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0106] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0107] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0108] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0110] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0111] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0112] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this invention can also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for end-to-end protection of API-level data flow, characterized in that, The API-level data flow end-to-end protection method includes: A no-code sampling mechanism is used to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed. A featureless data supplementation and identification mechanism is used to identify the data to be protected in the data to be processed. Construct an API-level end-to-end data flow map based on the data to be protected and the data to be processed. Risk detection is performed based on the API-level end-to-end data flow map to obtain the detection results; Based on the detection results and the API-level full-link data flow map, bidirectional tracing is performed to obtain a tracing report.

2. The API-level data flow end-to-end protection method as described in claim 1, characterized in that, The method of using a no-code tracking mechanism to collect application layer business log data and kernel-mode network traffic data in real time as data to be processed includes: The application layer business log data is collected in real time using an intelligent agent plugin deployed on the application system server; The kernel-mode extended Berkeley packet filter probe is used to capture socket data packets in real time as kernel-mode network traffic data. The application layer service log data and the kernel-mode network traffic data are correlated and aligned to obtain the data to be processed.

3. The API-level data flow end-to-end protection method as described in claim 1, characterized in that, The method of using featureless data supplementation and identification to identify the data to be protected in the data to be processed includes: Extract the data with features and the data without features from the data to be processed; The characteristic data is classified and graded using a pre-constructed classification and grading template to obtain the first data to be protected; Obtain the target API interface corresponding to the featureless data, and obtain the protection data identifier field corresponding to the target API interface; Extract the field content corresponding to the protected data identifier field from the featureless data; The second data to be protected is obtained from the featureless data based on the content of the field; The first data to be protected and the second data to be protected are integrated to obtain the data to be protected.

4. The API-level data flow end-to-end protection method as described in claim 1, characterized in that, The step of constructing an API-level end-to-end data flow map based on the data to be protected and the data to be processed includes: Extract user request identifiers from the application layer business log data, and associate the user layer with the application layer based on the user request identifiers to obtain a user-API association mapping table; The structured query language (SQL) statements in the kernel-mode network traffic data are parsed to obtain the parsing results; The parsing results are matched with the API requests in the application layer business log data to obtain matching results; The database information bound to the structured query language statement is determined based on the database connection information in the kernel-mode network traffic data. An API-SQL association mapping table is established based on the matching results and the database information bound to the structured query language statement; Bind the data to be processed with the structured query language statement to obtain an SQL-data field association mapping table; wherein, the SQL-data field association mapping table includes annotations added based on the data to be protected; The user-API association mapping table and the API-SQL association mapping table are linked in chronological order to construct the user access link; The API-SQL association mapping table and the SQL-data field association mapping table are linked together according to the API call order to construct the application call chain; By integrating the user access link and the application call link, a four-layer interconnected API-level full-link data flow map is obtained, comprising the user layer, application layer, database layer, and data object layer.

5. The API-level data flow end-to-end protection method as described in claim 1, characterized in that, The risk detection based on the API-level end-to-end data flow map yields the following results: The anomaly score for each link in the API-level end-to-end data flow map is calculated using the isolated forest algorithm. For risk links with abnormal scores greater than a preset threshold, a compliance baseline map constructed based on domain knowledge and historical data is obtained; Based on the compliance baseline map, baseline deviation detection is performed on the risk links to obtain the risk types; The detection results are generated based on the risk link and the risk type.

6. The API-level data flow end-to-end protection method as described in claim 1, characterized in that, The bidirectional tracing based on the detection results and the API-level end-to-end data flow map, resulting in a tracing report, includes: The abnormal entity is identified based on the detection results, and the source tracing direction is determined based on the abnormal entity. When the tracing direction is the user access behavior direction, locate all links in the API-level full-link data flow map associated with the detection result under the user index dimension, and extract the user access path from all links; and / or When the tracing direction is the data being called direction, locate all links associated with the detection result in the API-level full-link data flow map under the data index dimension, and extract the data being called path from all links; The source tracing report is generated based on the user access path and / or the data being invoked path.

7. The API-level data flow end-to-end protection method as described in claim 1, characterized in that, After identifying the data to be protected in the data to be processed using a featureless data supplementation identification mechanism, the method further includes: When the data to be protected is outbound data, the outbound rule engine is invoked to detect the destination, data type, sensitivity level, and transmission volume of the outbound data as statistical features of the outbound data; An outbound safety assessment report is generated based on the statistical characteristics of the outbound data.

8. An API-level data flow end-to-end protection device, characterized in that, The API-level data flow end-to-end protection device includes: The data acquisition unit is used to collect application layer business log data and kernel-mode network traffic data in real time using a no-code tracking mechanism as data to be processed. The identification unit is used to identify the data to be protected in the data to be processed using a featureless data supplementation identification mechanism; The construction unit is used to construct an API-level end-to-end data flow map based on the data to be protected and the data to be processed. The detection unit is used to perform risk detection based on the API-level full-link data flow map and obtain the detection results; The tracing unit is used to perform bidirectional tracing based on the detection results and the API-level full-link data flow map to obtain a tracing report.

9. A computer device, characterized in that, The computer device includes: A memory for storing at least one instruction; and a processor for executing the instructions stored in the memory to implement the API-level data flow end-to-end protection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in a computer device to implement the API-level data flow end-to-end protection method as described in any one of claims 1 to 7.

Images