Method and device for identifying sensitive data on API side, equipment and storage medium

Abstract

The invention provides an API side sensitive data identification method and device, equipment and a storage medium. The method comprises the following steps: collecting and analyzing flow data called by an API (Application Program Interface), and extracting API fields to obtain an API field set; determining a database access record having a preset association relationship with the API call, a database accessed by the database access record being a target database; establishing a mapping relationship between the API field and a data column of the target database; obtaining a label of a data column of the target database to obtain a database label; spreading the database label to the corresponding API field according to the mapping relation to obtain a label of the API field; and identifying the sensitive data on the API side according to the tag of the API field. According to the method and the device, the manual workload can be reduced, the identification coverage of the sensitive label in the API can be quickly increased, and the stability and the accuracy of sensitive data identification at the API side can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and storage medium for identifying sensitive data on the API side. Background Technology

[0002] With the microservice architecture of internet services and the widespread adoption of mobile internet, enterprises are using Application Programming Interfaces (APIs) as the primary form of external and internal data exchange. Requests and responses transmitted between clients and servers, and between services, often contain potentially sensitive data such as identity verification information, contact information, geolocation, identification numbers, account credentials, payment details, and health and financial information. Automated, low-false-positive sensitive data identification and labeling for API transmissions is a fundamental capability in compliance governance and data security protection.

[0003] In current engineering practices, the identification of sensitive data on the API side mainly adopts a traffic detection scheme based on rule-based regular expressions. For example, at the traffic mirroring or gateway parsing layer, feature patterns such as ID card numbers, bank card numbers, email addresses, mobile phone numbers, geographical coordinates, and document images are collected. The security data team manually audits the traffic and compiles the rules, which are continuously maintained after deployment. The inventors' research found that the above-mentioned method for identifying sensitive data on the API side has the following problems: First, rule maintenance and optimization are highly dependent on manual labor, resulting in high labor costs; Second, the naming, structure hierarchy, format, and masking methods of interface fields often change, making the rules fragile and prone to false positives and false negatives; Third, when the API aggregates multiple data sources or performs hashing and de-identification derivation, it is difficult to maintain stable identification results using regular expressions alone.

[0004] Therefore, how to improve the accuracy and stability of identifying sensitive data on the API side while reducing manual maintenance costs has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, device, and storage medium for identifying sensitive data on the API side, in order to solve the aforementioned technical problems in the prior art.

[0006] On the one hand, in order to achieve the above objectives, the present invention provides a method for identifying sensitive data on the API side.

[0007] The method for identifying sensitive data on the API side includes: collecting and parsing traffic data of API calls, extracting API fields to obtain a set of API fields; identifying database access records that have a preset association with the API calls, wherein the database accessed by the database access records is the target database; establishing a mapping relationship between the API fields and the data columns of the target database; obtaining the labels of the data columns of the target database to obtain database labels; propagating the database labels to the corresponding API fields according to the mapping relationship to obtain the labels of the API fields; and identifying sensitive data on the API side based on the labels of the API fields.

[0008] Further, the step of obtaining the labels of the data columns in the target database to obtain the database labels includes: obtaining the configured labels of the data columns to obtain the database labels; when the data column is not configured with labels, obtaining the name and / or remarks of the data column; and determining the database labels based on the name and / or remarks of the data column.

[0009] Further, the step of determining the database label based on the name and / or remarks of the data column includes: obtaining sample data values ​​of the data column; performing keyword matching on the name and / or remarks of the data column according to a pre-built sensitive rule base to obtain candidate database labels; and determining the candidate database label as the database label of the data column when the sample data value satisfies the value taking rule corresponding to the candidate database label.

[0010] Furthermore, the steps of collecting and parsing API call traffic data and extracting API fields to obtain an API field set include: performing traffic mirroring on the communication link between the server and the user to obtain traffic data containing request and response messages of API calls; performing protocol parsing on the traffic data to obtain the request body and response body content of the API call; and parsing and extracting fields from the request body and response body content to obtain the API fields.

[0011] Further, the step of determining database access records with a preset association with the API call includes: determining the occurrence time of the API call based on the timestamps of the network data packets corresponding to the content of the API request body and response body; setting a time window including the occurrence time of the API call; obtaining database access records within the time window to obtain candidate database access records; and determining the candidate database access record as a database access record with a preset association with the API call when the interval between the access time of the candidate database access record and the occurrence time of the API call is less than or equal to a preset time interval threshold.

[0012] Furthermore, the step of determining the database access record with a preset association relationship to the API call further includes: parsing the traffic data to obtain the metadata of the API call; determining the session identifier of the API call based on the metadata of the API call; obtaining the database access log; parsing the database access log to obtain the session identifier of the candidate database access record; when the interval between the access time of the candidate database access record and the occurrence time of the API call is greater than the preset time interval threshold, determining whether the session identifier of the API call is consistent with the session identifier of the candidate database access record; when the session identifier of the API call is consistent with the session identifier of the candidate database access record, determining that the candidate database access record is a database access record with a preset association relationship to the API call.

[0013] Furthermore, the step of establishing a mapping relationship between the API field and the data columns of the target database includes: for the data table of the target database, determining the number of data columns that match the API field in the API field set; when the number of matching data columns meets a preset matching condition, determining the data table as the target data table; determining the data columns in the target data table that match the API field to obtain the mapping relationship.

[0014] On the other hand, in order to achieve the above objectives, the present invention provides an API-side sensitive data identification device.

[0015] The API-side sensitive data identification device includes: a first processing module for collecting and parsing API call traffic data and extracting API fields to obtain an API field set; a determination module for determining database access records that have a preset association with the API call, wherein the database accessed by the database access record is the target database; a second processing module for establishing a mapping relationship between the API fields and the data columns of the target database; an acquisition module for acquiring the tags of the data columns of the target database to obtain database tags; a third processing module for propagating the database tags to the corresponding API fields according to the mapping relationship to obtain the tags of the API fields; and an identification module for identifying API-side sensitive data according to the tags of the API fields.

[0016] On the other hand, to achieve the above objectives, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0017] On the other hand, to achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0018] The present invention provides a method, apparatus, device, and storage medium for identifying sensitive data on the API side. First, it collects and parses traffic data from API calls, extracting API fields to form an API field set. Then, it identifies database access records with a preset association with the API call and determines the database accessed by these records as the target database. Next, it establishes a mapping relationship between the API fields and the data columns of the target database, obtaining the labels of the target database's data columns to obtain database labels. These database labels are then propagated to the corresponding API fields according to the mapping relationship to obtain API field labels. Finally, it identifies the sensitive data on the API side based on the API field labels. This invention establishes a mapping relationship between API fields and target database data columns, and propagates existing data column labels from the database side to the corresponding API fields. It can automatically utilize data labels from structured database data to supplement semantic labels for unstructured or semi-structured API fields, rapidly expanding the scope of sensitive data identification on the API side. This enables automatic identification of sensitive data on the API side, reducing reliance on manually writing and maintaining sensitive regular expression rules, thereby lowering manual auditing costs. By extending the label information on structured data to the data carried by API calls, the coverage of sensitive labels extends from within the database to the interface transmission layer, improving the completeness and accuracy of sensitive data identification on the API side, and facilitating subsequent analysis and control of sensitive data flow. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of the API-side sensitive data identification method provided in Embodiment 1 of the present invention; Figure 2 This is a block diagram of the API-side sensitive data identification device provided in Embodiment 2 of the present invention; Figure 3 This is a hardware structure diagram of a computer device provided in Embodiment 3 of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0021] Example 1 This invention provides a method for identifying sensitive data on the API side. This method automatically enriches API fields with existing database tags without requiring manual compilation of numerous regular expression rules. The enriched tags are then used to identify sensitive API data, thereby improving the accuracy and stability of sensitive API data identification and reducing manual auditing costs. Specifically, Figure 1 The flowchart is as follows: This is a method for identifying sensitive API data according to Embodiment 1 of the present invention. Figure 1 As shown, the API-side sensitive data identification method provided in this embodiment includes the following steps S101 to S106.

[0022] Step S101: Collect and parse the traffic data of API calls, and extract API fields to obtain the API field set.

[0023] In this step, API calls are made to web services, microservice interfaces, mobile interface gateways, or internal RPC interfaces deployed on the server. API calls are typically transmitted between the client and server using protocols such as HTTP, HTTPS, and gRPC.

[0024] Optionally, in this embodiment, a traffic mirroring or traffic capture component is configured on the communication link between the server and the user to copy network data packets containing API request and response messages to the traffic analysis module, thereby avoiding intrusion into the business code. The traffic analysis module performs protocol parsing on the collected network data packets, reconstructs the request and response messages for each API call, and parses out the API request and response body content, such as JSON, XML, or application / x-www-form-urlencoded format.

[0025] Optionally, for JSON-formatted request and response bodies, the key-value pair structure is recursively parsed to extract the keys and corresponding values ​​at each level. Each key is treated as an API field name, and the value obtained from a single API call is used as the field value. For example, when the response body is {"name":"admin","birthday":"1900-01-01","gender":"W","time":"2025-01-01"}, the fields "name", "birthday", "gender", and "time" and their corresponding values ​​are extracted. For body content in XML, urlencoded, or other formats, field names and values ​​can also be extracted using preset parsing rules.

[0026] Specifically, for API calls of the same route and type, the fields appearing in the API call are statistically analyzed and merged to form an API field set, which is used for subsequent mapping with database data columns. During the reassembly of request and response messages, the traffic analysis module can also record information such as the timestamp, source IP address, destination IP address, and port number corresponding to each API call, which is used for subsequent association with database access logs. In this step, the API call metadata, request body, and response body content obtained after protocol parsing are uniformly organized into structured records as input for subsequent steps.

[0027] Step S102: Determine the database access records that have a preset association with the API call.

[0028] Among them, the database access record is the target database.

[0029] In this step, database access records with a pre-defined association with the API call refer to database access records that are presumed to be triggered by the API call. Theoretically, this refers to database access records triggered by the API call. In practice, when determining this step, the association conditions between the API call and the triggered database access records are used to filter and identify database access records with a pre-defined association with the API call, based on one or more of the following conditions: association conditions based on time dimension, association conditions based on network session dimension, association conditions based on access path or link consistency, association conditions based on access parameter consistency, and association conditions based on endpoint identity or service instance.

[0030] Specifically, this can be achieved by limiting the time interval between the API call and the database access record to be short; ensuring that the session identifiers of the two are consistent; ensuring that the access paths or links of the two are consistent; ensuring that the key parameters in the API request body (such as userId, orderId, etc.) are consistent with the field names or field values ​​involved in the database access SQL; and ensuring that the database user, connection ID, or connection pool source in the database access record is consistent with the service identity corresponding to the API call, etc.

[0031] Database access records are generated by database auditing systems, database agents, database middleware, or the database's own audit log module. These records document information for each database access event, such as access time, the accessed database instance, the database user, the executed SQL statement, the accessed table and field names, and the session connection identifier. Alternatively, bypass acquisition or port mirroring can be configured on the network path between the application server and the database server to copy network packets generated when the application server accesses the database to the traffic analysis module. For different types of databases (such as MySQL, PostgreSQL, SQL Server, etc.), the traffic analysis module has a built-in corresponding database protocol parser that reassembles and semantically parses the mirrored traffic to obtain the database access records.

[0032] Optionally, in one embodiment, a time window is first set for the API call based on the time of the API call. For example, the window extends forward and backward by a certain number of milliseconds or seconds, centered on the API call time. Within this time interval, database access logs are retrieved to obtain candidate database access records. Among the candidate database access records, those whose access time and API call time do not exceed a preset time interval threshold are considered to have a high correlation.

[0033] Alternatively, in another implementation, session identification information can be used to further improve the accuracy of the association. For example, on the API call side, a network session identifier for the API call is constructed based on the source IP address, source port, destination IP address, destination port, and transport layer protocol; on the database access side, the session identifier corresponding to the database connection or the network 5-tuple of the connection is parsed from the database audit log. For candidate database access records with large time intervals, the API call session identifier is further compared with the session identifier of the candidate database access record. When they match, it is still determined that the database access record and the API call have a preset association relationship.

[0034] Through the above association process, a set of database access records with a preset association relationship with the current API call is identified. Since database access records usually contain information about the accessed database instance, the database accessed by these records is identified and used as the target database for subsequent data column label acquisition and mapping relationship construction.

[0035] Step S103: Establish the mapping relationship between API fields and data columns in the target database.

[0036] In this step, the target database contains multiple tables, each with several columns. A comparison is made based on field names, field values, and other information to establish a mapping between the API fields and the database columns.

[0037] Optionally, in one specific implementation, for each data table in the target database, several rows of data actually returned or queried in the database access record corresponding to that data table in step S102 are selected, and sample values ​​of each data column in the corresponding record are extracted. Simultaneously, for the API fields in the request and response bodies corresponding to the API call, sample field values ​​in the current call and over a period of time are extracted. Then, the API fields are compared with the sample values ​​of each data column, for example, comparing whether the values ​​are equal, whether they are consistent within a certain format or range, etc., to count how many data columns in a certain data table can match the sample values ​​of certain API fields.

[0038] If, in a given data table, the number of data columns matching a field in the API field set meets a preset matching condition—for example, the number of matches reaches a certain percentage of the number of fields in the field set—then that data table can be identified as the target data table corresponding to that API call. In the target data table, for each API field, data columns matching the sample value or field name of that field are then identified, thus obtaining the mapping relationship between the API fields and the database data columns.

[0039] For example, the user_info table in database 10.0.0.1 contains columns such as user_id, name, birthday, gender, and time. When the request body of the API interface 10.0.0.2 / getUserInfo is userId=123456 and the response body is {"name":"admin","birthday":"1900-01-01","gender":"W","time":"2025-01-01"}, a comparison reveals that the value of the API field userId, 123456, corresponds to the value of the user_id column in the user_info table. The values ​​of the API fields name, birthday, gender, and time also match the values ​​of their corresponding columns in the user_info table. This establishes a logical mapping between the API call and the user_info table, and within this table, a mapping relationship is established between each API field and a specific data column.

[0040] Through the above mapping, the relationship between API fields and database columns can be automatically deduced based solely on the correspondence between data structures and data values, without understanding the business semantics.

[0041] Step S104: Obtain the labels of the data columns in the target database to obtain the database labels.

[0042] In this step, the data columns in the target database are pre-labeled using a data security system or data governance platform. For data columns involving personal information or important business fields, corresponding sensitive data labels have been configured, such as "name," "ID number," "mobile phone number," "date of birth," and "address." When pre-configured labels exist, this step directly reads the labels for the corresponding data columns from the database's metadata management module or label management system, forming a database label set. For data columns without configured labels, labels are determined using column names, column comments, and sample data values. Finally, for each data column in the target data table, the corresponding database labels are obtained for subsequent propagation to the API field side.

[0043] Alternatively, after obtaining the database tags corresponding to each database, the database tags of the target database can be extracted from them in this step.

[0044] Step S105: Propagate the database tags to the corresponding API fields according to the mapping relationship to obtain the tags of the API fields.

[0045] In this step, based on the mapping relationship between API fields and database data columns established in step S103, the labels of each data column in the target database are automatically propagated to the API fields they are mapped to. Specifically, for each pair of API fields and database data columns in the mapping relationship, the database label obtained in step S104 for that data column is found, and the label is assigned to the corresponding API field, so that the API field, which originally lacked a semantic label, obtains a label consistent with the database side. For example, if the database column 'time' is configured with the label 'creation time', this label is propagated to the 'time' field in the API response body through the mapping relationship, so that the API side can also recognize this field as the sensitive data type 'creation time', instead of just the vaguely named 'time'. 'Time' refers to the 'time' field in the response body, because 'time' is a very broad field, which could be creation time, modification time, etc. Without understanding the customer's business, even with context, it is impossible to determine the label of this field. Therefore, in the prior art, 'time' cannot be labeled with the 'creation time' label in the API's independent identification; after mapping, it is labeled with the 'creation time' label.

[0046] Optionally, when an API field is mapped to multiple data columns, a priority rule, confidence rule, or majority voting rule is used to select the label to determine the optimal label. In this step, the label of the data column that matches the field value the most is given priority.

[0047] Step S106: Identify sensitive data on the API side based on the labels of the API fields.

[0048] In this step, after the label propagation of API fields is completed, sensitive data on the API side can be identified based on the label information. A set of labels identified as sensitive is pre-configured, such as "name", "mobile phone number", "ID card number", "email address", "date of birth", "address", "account number", "device unique identifier", etc. When the label of an API field belongs to this set of sensitive labels, the field can be identified as a sensitive data field on the API side.

[0049] Optionally, further validation can be performed based on the characteristics of the field values. For example, checking whether the mobile phone number format meets expectations and whether the length and checksum of the ID card number are correct can filter out a small number of false positives that may occur during the label propagation process. Finally, a sensitive data identification result is generated for each API call, recording the name of the API field identified as sensitive, the field value, the corresponding label, and the API interface information to which it belongs, for use in data security auditing, access control policy configuration, or sensitive data flow analysis.

[0050] In the API-side sensitive data identification method provided in this embodiment, firstly, traffic data of API calls is collected and parsed to extract API fields and form an API field set; then, database access records with a preset association relationship with the API call are identified, and the database accessed by the database access records is identified as the target database; then, a mapping relationship is established between the API fields and the data columns of the target database, the labels of the data columns of the target database are obtained to obtain database labels, and the database labels are propagated to the corresponding API fields according to the mapping relationship to obtain the labels of the API fields; finally, sensitive data on the API side is identified according to the labels of the API fields. The API-side sensitive data identification method provided in this embodiment establishes a mapping relationship between API fields and target database data columns, and propagates existing database column labels to the corresponding API fields. It automatically utilizes data labels from structured database data to supplement semantic labels for unstructured or semi-structured API fields, rapidly expanding the scope of sensitive data identification on the API side. This achieves automatic identification of sensitive data on the API side, reducing reliance on manually writing and maintaining sensitive regular expression rules, thereby lowering manual auditing costs. It extends the application of label information from structured data to the data carried by API calls, expanding the coverage of sensitive labels from within the database to the interface transmission layer. This improves the completeness and accuracy of sensitive data identification on the API side, and is more conducive to subsequent analysis and control of sensitive data flows.

[0051] Optionally, in one embodiment, the step of obtaining the labels of data columns in the target database to obtain database labels includes: obtaining the labels of configured data columns to obtain database labels; when a data column is not labeled, obtaining the name and / or remarks of the data column; and determining the database label based on the name and / or remarks of the data column. Specifically, when obtaining the labels of data columns in the target database, in addition to directly reading the configured label information, database labels can also be automatically inferred for data columns without configured labels based on information such as the name and remarks of the data column.

[0052] Further optionally, in one embodiment, the step of determining the database label based on the name and / or remarks of the data column includes: obtaining sample data values ​​of the data column; performing keyword matching on the name and / or remarks of the data column according to a pre-built sensitive rule base to obtain candidate database labels; and determining the candidate database label as the database label of the data column when the sample data value meets the value rules corresponding to the candidate database label.

[0053] Specifically, for each data column without a configured label, the system first retrieves the column's name and remarks from the database metadata, such as column names like "user_name", "id_card_no", "mobile", and "birthday", along with corresponding explanatory text. Then, based on a pre-built sensitive rule base, keyword matching is performed on the column names and remarks. This sensitive rule base contains a set of keywords for common sensitive fields, such as "name, name, user_name, username", "ID card, id_card, identity_no", and "mobile number, phone". By matching the column names and remarks, one or more candidate database labels are obtained.

[0054] However, the inventors discovered that relying solely on names and notes might introduce misjudgments in certain semantically ambiguous scenarios, such as column names like "code" and "number" potentially having multiple meanings. Therefore, this embodiment also extracts a certain number of sample data values ​​from the data column, for example, randomly selecting column values ​​from several rows of records, and verifies the sample values ​​according to the value selection rules corresponding to the candidate labels. For example, for a data column identified as a "phone number" candidate label, most of its sample values ​​are required to be 11-digit strings starting with a specific number segment; for a data column identified as a "date of birth" candidate label, the sample values ​​are required to primarily conform to a date format. Only when the sample data values ​​meet the value selection rules corresponding to the candidate label are the candidate database labels determined as the final labels for that data column.

[0055] In the API-side sensitive data identification method provided in this embodiment, an automatic inference mechanism based on joint judgment of column name, column remarks, and sample data values ​​is introduced during the process of obtaining target database data column labels. This mechanism automatically completes the database-side label system in the absence of manual label configuration. The API-side sensitive data identification method provided in this embodiment reduces reliance on manual label configuration, lowers the maintenance cost of label system construction, and, combined with rule-based validation of sample data values, effectively reduces the propagation of erroneous labels caused by ambiguous column names when combined with label propagation and API-side sensitive data identification. This results in more accurate and reliable sensitive data fields ultimately identified on the API side.

[0056] Optionally, in one embodiment, the step of collecting and parsing traffic data of API calls and extracting API fields to obtain an API field set includes: performing traffic mirroring on the communication link between the server and the user to obtain traffic data containing request messages and response messages of API calls; performing protocol parsing on the traffic data to obtain the request body and response body content of the API calls; and parsing and extracting fields from the request body and response body content to obtain API fields.

[0057] Specifically, mirroring ports or bypass acquisition devices are deployed on the communication link between the server and the user to mirror network data passing through this link in real time, acquiring traffic data containing API request and response messages. The mirroring module operates at the switch, load balancer, gateway, or host network stack level. By configuring mirroring rules, it replicates traffic corresponding to specific ports or IPs and ports to the analysis server. The traffic analysis module performs protocol parsing on the acquired mirrored traffic, reconstructing the request and response messages corresponding to each API call based on different transport and application layer protocols. For example, when the API is implemented based on HTTP or HTTPS protocols, fields such as path, method, and request ID in the HTTP header are used to distinguish different calls, and request and response data related to the same call are aggregated and recombined to obtain the request and response body content of the API call.

[0058] After parsing the request and response bodies, appropriate parsing rules are applied to extract fields based on the message body encoding format, such as JSON, XML, and application / x-www-form-urlencoded. For JSON, the object structure is recursively traversed to extract the key and corresponding value at each level, forming a flat or hierarchical set of fields. For XML, fields are extracted based on tag names and attribute names. For urlencoded, fields are parsed using key-value pairs as delimiters. Finally, the set of API fields related to the API call is obtained.

[0059] In the API-side sensitive data identification method provided in this embodiment, API call data is collected by means of bypass traffic mirroring, and request and response messages are uniformly parsed and fields are extracted. This method comprehensively covers API call data of different interfaces and formats without requiring intrusive modifications to business code. Using the API-side sensitive data identification method provided in this embodiment, complete and accurate API field input can be provided for subsequent association and field mapping with database access records while minimizing the impact on the business system.

[0060] Optionally, in one embodiment, the step of determining database access records with a preset association with API calls includes: determining the occurrence time of the API call based on the timestamps of network data packets corresponding to the content of the API request body and response body; setting a time window including the occurrence time of the API call; obtaining database access records within the time window to obtain candidate database access records; and determining the candidate database access record as a database access record with a preset association with the API call when the interval between the access time of the candidate database access record and the occurrence time of the API call is less than or equal to a preset time interval threshold.

[0061] Specifically, after the traffic analysis module completes the reassembly of the API request and response body content in step S101, it determines the occurrence time of the API call based on the timestamps of the network data packets corresponding to the request and response body content. The timestamp representing the first receipt of the API request message, or the time interval between the request and response completion, is used as a time reference for the API call. Based on this, a time window including the API call occurrence time is set, for example, extending a preset duration forward and backward from the API call occurrence time to form a time window interval. Database access records whose access times fall within this time window are retrieved from the database access log, and these records are used as candidate database access records.

[0062] For each candidate database access record, first determine whether the time interval between its access time and the API call time is less than or equal to a preset time interval threshold. When this condition is met, directly determine the candidate database access record as a database access record with a preset association with the API call.

[0063] In the API-side sensitive data identification method provided in this embodiment, a two-layer time-based filtering method is adopted when determining database access records with a preset association with API calls. After determining the occurrence time of the API call, a time window containing that occurrence time is set, and records whose access times fall within the time window are obtained from the database access records as candidate database access records. Then, the time interval between the access time of the candidate database access record and the occurrence time of the API call is calculated. When the time interval is less than or equal to a preset time interval threshold, the candidate database access record is determined as a database access record with a preset association with the API call. By adopting the API-side sensitive data identification method provided in this embodiment, the two-layer time condition filtering method based on time window and time interval threshold limits the range of database access records to be retrieved through time window, reducing the overhead of association query; within the time window, the association is confirmed based on the time interval threshold, finding database access records whose time is close enough to the occurrence time of the API call. This makes the database access records used for subsequent field mapping and tag propagation more accurate, improving the reliability of API-side sensitive data identification results.

[0064] Optionally, in one embodiment, the step of determining the database access record with a preset association relationship to the API call further includes: parsing the traffic data to obtain the metadata of the API call; determining the session identifier of the API call based on the metadata of the API call; obtaining the database access log; parsing the database access log to obtain the session identifier of the candidate database access record; when the interval between the access time of the candidate database access record and the occurrence time of the API call is greater than a preset time interval threshold, determining whether the session identifier of the API call is consistent with the session identifier of the candidate database access record; when the session identifier of the API call is consistent with the session identifier of the candidate database access record, determining the candidate database access record as a database access record with a preset association relationship to the API call.

[0065] The inventors discovered that in certain scenarios, network jitter, queuing, slow queries, and retries between the application server and the database can lead to large time intervals between database accesses triggered by the same API call. Based on this, in this embodiment, candidate database access records with time intervals exceeding a preset threshold but still falling within the time window are further filtered using session identifiers. Specifically, when parsing the traffic data, network metadata of the API call is obtained, such as the client IP address, client port, server IP address, server port, and the transport layer protocol used, and a session identifier for the API call is constructed accordingly. Simultaneously, for the database access logs, the client address, port, and database server address and port corresponding to each database access event are parsed out, and a session identifier for the database access record is constructed based on this.

[0066] If the interval between the access time of a candidate database access record and the API call occurrence time exceeds a preset time interval threshold, the system further determines whether the session identifier of the record matches the session identifier of the API call. If the session identifiers match, the candidate database access record is still identified as a database access record with a preset association relationship to the API call. If the time interval is large and the session identifiers do not match, the record is excluded from the association results.

[0067] In the API-side sensitive data identification method provided in this embodiment, database access records that are close to API calls in terms of time dimension are first filtered out using time windows and time interval thresholds. Then, session identifiers are used to further determine some boundary cases. Under the premise of controlling the scope of database log retrieval and ensuring performance, the accuracy of the association between API calls and database access is significantly improved. Using the API-side sensitive data identification method provided in this embodiment can effectively reduce erroneous label enrichment caused by misassociations, while avoiding the loss of scenarios that need to be associated, thus reducing enriched labels and affecting the comprehensiveness of API-side sensitive data identification.

[0068] Optionally, in one embodiment, the step of establishing a mapping relationship between API fields and data columns in the target database includes: determining the number of data columns in the target database that match the API fields in the API field set; when the number of matching data columns meets a preset matching condition, determining the data table as the target data table; and determining the data columns in the target data table that match the API fields to obtain the mapping relationship.

[0069] Specifically, after identifying the target database and obtaining multiple data tables in step S102, for each data table, the number of data columns in that table that match fields in the API field set is counted. Matching relationships are determined based on rules such as field name similarity and the proportion of equal field values. For example, if the API field name and the data table header name are identical after removing capitalization and underscores, or if the field value is highly consistent with the data column value in the sample data, the data column is considered to match the corresponding API field. Then, for the same API call, the number of matches in different data tables is compared. When the number of data columns in a data table that match fields in the API field set meets preset matching conditions, such as reaching a certain proportion of the total number of fields in the API field set or the maximum number of matches across all data tables, that data table is determined as the target data table for that API call.

[0070] After the target data table is determined, a matching data column is further identified for each API field within the target data table, constructing a one-to-one mapping relationship. The mapping results are stored in a metadata table or mapping configuration for subsequent tag propagation and sensitive data identification. Optionally, field names and sample values ​​are mapped to vectors using a preset embedding model. When the similarity between an API field and the vector corresponding to a data column is not lower than a preset similarity threshold, the data column is considered to match the API field.

[0071] In the API-side sensitive data identification method provided in this embodiment, target data tables are first identified at the data table level based on the number of matches, and then field-level matching is performed within the target tables. When multiple data tables with similar structures exist in the database, the risk of mistakenly associating API calls with incorrect data tables is effectively reduced. Using the API-side sensitive data identification method provided in this embodiment can improve the stability and accuracy of the mapping from API fields to database columns, further ensuring the reliability and trustworthiness of the sensitive tag enrichment results based on this mapping relationship.

[0072] Example 2 Corresponding to Embodiment 1 above, Embodiment 2 of the present invention provides an API-side sensitive data identification device. The corresponding technical features and effects can be referred to Embodiment 1 above, and will not be repeated in this embodiment. Figure 3 This is a block diagram of the API-side sensitive data identification device provided in Embodiment 2 of the present invention, as shown below. Figure 3 As shown, the device includes: a first processing module 201, a determining module 202, a second processing module 203, an acquiring module 204, a third processing module 205, and an identification module 206.

[0073] The first processing module 201 is used to collect and parse the traffic data of API calls, and extract API fields to obtain an API field set; the determination module 202 is used to determine the database access records that have a preset association relationship with the API calls, wherein the database accessed by the database access records is the target database; the second processing module 203 is used to establish a mapping relationship between the API fields and the data columns of the target database; the acquisition module 204 is used to acquire the tags of the data columns of the target database to obtain database tags; the third processing module 205 is used to propagate the database tags to the corresponding API fields according to the mapping relationship to obtain the tags of the API fields; and the identification module 206 is used to identify sensitive data on the API side according to the tags of the API fields.

[0074] Optionally, in one embodiment, the acquisition module includes: a first acquisition unit, configured to acquire the configured labels of the data column to obtain the database label; a second acquisition unit, configured to acquire the name and / or remarks of the data column when the data column is not configured with a label; and a first determination unit, configured to determine the database label based on the name and / or remarks of the data column.

[0075] Optionally, in one embodiment, when the determining unit determines the database label based on the name and / or remarks of the data column, the specific steps include: obtaining sample data values ​​of the data column; performing keyword matching on the name and / or remarks of the data column according to a pre-built sensitive rule base to obtain candidate database labels; and determining the candidate database label as the database label of the data column when the sample data value satisfies the value taking rule corresponding to the candidate database label.

[0076] Optionally, in one embodiment, the first processing module includes: a collection unit, configured to perform traffic mirroring on the communication link between the server and the user to obtain traffic data containing request messages and response messages of API calls; a first parsing unit, configured to perform protocol parsing on the traffic data to obtain the request body and response body content of the API call; and a second parsing unit, configured to parse and extract fields from the request body and response body content to obtain the API fields.

[0077] Optionally, in one embodiment, the determining module includes: a second determining unit, configured to determine the occurrence time of the API call based on the timestamp of the network data packet corresponding to the content of the API request body and response body; a processing unit, configured to set a time window including the occurrence time of the API call; a third obtaining unit, configured to obtain database access records within the time window to obtain candidate database access records; and a third determining unit, configured to determine the candidate database access record as a database access record with a preset association relationship with the API call when the interval between the access time of the candidate database access record and the occurrence time of the API call is less than or equal to a preset time interval threshold.

[0078] Optionally, in one embodiment, the determining module further includes: a third parsing unit, configured to perform protocol parsing on the traffic data to obtain the metadata of the API call; a fourth determining unit, configured to determine the session identifier of the API call based on the metadata of the API call; a fourth obtaining unit, configured to obtain database access logs; a fourth parsing unit, configured to parse the database access logs to obtain the session identifier of the candidate database access record; a judging unit, configured to judge whether the session identifier of the API call is consistent with the session identifier of the candidate database access record when the interval between the access time of the candidate database access record and the occurrence time of the API call is greater than the preset time interval threshold; and a fourth determining unit, configured to determine that the candidate database access record is a database access record with a preset association relationship with the API call when the session identifier of the API call is consistent with the session identifier of the candidate database access record.

[0079] Optionally, in one embodiment, the second processing module includes: a fifth determining unit, configured to determine the number of data columns in the target database that match the API fields in the API field set; a sixth determining unit, configured to determine the data table as the target data table when the number of matching data columns meets a preset matching condition; and a seventh determining unit, configured to determine the data columns in the target data table that match the API fields, thereby obtaining the mapping relationship.

[0080] Example 3 This embodiment also provides a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including a standalone server or a server cluster composed of multiple servers) that executes a program. Figure 3 As shown, the computer device 01 in this embodiment includes, but is not limited to, a memory 012 and a processor 011 that can be interconnected via a system bus, such as... Figure 3 As shown. It should be noted that, Figure 3 Only a computer device 01 with component memory 012 and processor 011 is shown; however, it should be understood that it is not required to implement all of the components shown, and alternatively, more or fewer components may be implemented.

[0081] In this embodiment, the memory 012 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 012 may be an internal storage unit of the computer device 01, such as the hard disk or memory of the computer device 01. In other embodiments, the memory 012 may also be an external storage device of the computer device 01, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 01. Of course, the memory 012 may include both the internal storage unit and its external storage device of the computer device 01. In this embodiment, the memory 012 is typically used to store the operating system and various reference software installed on the computer device 01, such as the program code of the API-side sensitive data identification device in Embodiment 2. In addition, memory 012 can also be used to temporarily store various types of data that have been output or will be output.

[0082] In some embodiments, processor 011 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 011 is typically used to control the overall operation of computer device 01. In this embodiment, processor 011 is used to run program code stored in memory 012 or process data, such as methods for identifying sensitive data on the API side.

[0083] Example 4 This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App reference store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. The computer-readable storage medium of this embodiment is used to store an API-side sensitive data identification device, and when executed by a processor, it implements the API-side sensitive data identification method of Embodiment 1.

[0084] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0085] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0087] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An API-side sensitive data identification method, characterized by, The method comprises: collecting and parsing traffic data of API calls to obtain an API field set by extracting API fields; determining database access records having a preset association relationship with the API calls, wherein a database accessed by the database access records is a target database; establishing a mapping relationship between the API fields and data columns of the target database; obtaining labels of the data columns of the target database to obtain database labels; propagating the database labels to corresponding API fields according to the mapping relationship to obtain labels of the API fields; and identifying API-side sensitive data according to the labels of the API fields. 2.The API-side sensitive data identification method of claim 1, wherein, The step of obtaining labels of the data columns of the target database to obtain database labels comprises: obtaining labels of the data columns configured to obtain the database labels; when the data columns are not configured with labels, obtaining names and / or notes of the data columns; determining the database labels according to the names and / or notes of the data columns. 3.The API-side sensitive data identification method of claim 2, wherein, The step of determining the database labels according to the names and / or notes of the data columns comprises: obtaining sample data values of the data columns; performing keyword matching on the names and / or notes of the data columns according to a pre-constructed sensitive rule library to obtain candidate database labels; when the sample data values meet a value rule corresponding to the candidate database labels, determining that the candidate database labels are the database labels of the data columns.

4. The method of claim 1, wherein, The step of collecting and parsing traffic data of API calls to obtain an API field set comprises: performing traffic mirroring on a communication link between a server and a user to obtain traffic data containing request messages and response messages of API calls; performing protocol analysis on the traffic data to obtain request body and response body contents of the API calls; parsing and extracting fields in the request body and response body contents to obtain the API fields.

5. The method of claim 4, wherein, The step of determining database access records having a preset association relationship with the API calls comprises: determining a time of occurrence of the API calls according to a timestamp of a network packet corresponding to the request body and response body contents of the API calls; setting a time window including the time of occurrence of the API calls; obtaining database access records in the time window to obtain candidate database access records; when an interval between an access time of the candidate database access records and the time of occurrence of the API calls is less than or equal to a preset time interval threshold, determining that the candidate database access records are the database access records having the preset association relationship with the API calls.

6. The method of claim 5, wherein, The step of determining database access records having a preset association relationship with the API calls further comprises: performing protocol analysis on the traffic data to obtain metadata of the API calls; determining a session identifier of the API calls according to the metadata of the API calls; obtaining a database access log; parsing the database access log to obtain a session identifier of the candidate database access records; determining whether the session identifier of the API call is consistent with the session identifier of the candidate database access record when the interval between the access time of the candidate database access record and the occurrence time of the API call is greater than the preset time interval threshold; determining that the candidate database access record is a database access record having a preset association relationship with the API call when the session identifier of the API call is consistent with the session identifier of the candidate database access record.

7. The method of claim 1, wherein, The step of establishing the mapping relationship between the API field and the data column of the target database comprises: determining the number of data columns matched with the API field in the API field set in the data table of the target database; determining the data table as a target data table when the number of matched data columns meets a preset matching condition; determining the data column matched with the API field in the target data table to obtain the mapping relationship.

8. A device for identifying sensitive data on the API side, characterized in that, comprise: a first processing module configured to collect and parse traffic data of an API call, extract API fields to obtain an API field set; a determining module configured to determine a database access record having a preset association relationship with the API call, wherein the database accessed by the database access record is a target database; a second processing module configured to establish a mapping relationship between the API field and the data column of the target database; an obtaining module configured to obtain a label of the data column of the target database to obtain a database label; a third processing module configured to propagate the database label to the corresponding API field according to the mapping relationship to obtain a label of the API field; and an identifying module configured to identify API-side sensitive data according to the label of the API field.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images