Data processing method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a data processing method and device, equipment, a medium and a product, and is applied to the technical field of sensitive data processing. The method comprises the steps of obtaining fields and field annotations of sensitive data of a target project; determining a sensitive data type and a sensitive data level of the sensitive data according to the field and the field annotation of the sensitive data; determining sensitive data information based on the field of the sensitive data, the field annotation, the sensitive data type and the sensitive data level; and writing the sensitive data information into the sensitive object information table and a preset dictionary. The technical effect of improving the sensitive data processing efficiency is achieved.

Description

Technical Field

[0001] This application relates to the field of sensitive data processing technology, and in particular to a data processing method, apparatus, device, medium and product. Background Technology

[0002] In software development, sensitive data management has always been a crucial aspect of system security. With the continuous development of the internet age, societal demands for sensitive data protection are constantly increasing, making sensitive data protection a prerequisite for a good product. Traditionally, sensitive data detection relies on manual review and rule matching. However, manual review is inefficient, costly, and suffers from inconsistent review quality.

[0003] In existing technologies, the main methods used to detect sensitive data are: sensitive data detection techniques based on regular expressions, or sensitive data classification based on traditional machine learning models.

[0004] Because existing rule-based and model-based sensitive data detection technologies lack comprehensive sensitive data detection capabilities, they suffer from low sensitivity data processing efficiency. Summary of the Invention

[0005] This application provides data processing methods, apparatus, devices, media, and products to achieve the technical effect of improving the efficiency of sensitive data processing.

[0006] In a first aspect, embodiments of this application provide a data processing method, including:

[0007] The sensitive data information obtained from the target project is written into the sensitive object information table and the preset dictionary;

[0008] For each sensitive data item in the sensitive object information table, a de-identification detection is performed to obtain the de-identification detection results;

[0009] Generate a system sensitivity data table based on the desensitization detection results;

[0010] A sensitive data ledger is generated based on the system's sensitive data table.

[0011] In one possible implementation, the sensitive data information obtained from the target project is written into a sensitive object information table and a preset dictionary, including:

[0012] Retrieve the fields and field annotations of sensitive data in the target project;

[0013] Based on the fields and field annotations of the sensitive data, determine the sensitive data type and sensitivity level of the sensitive data;

[0014] Based on the fields, field annotations, sensitive data types, and sensitivity levels of sensitive data, determine the sensitive data information;

[0015] Sensitive data information is written into the sensitive object information table and the preset dictionary.

[0016] In one possible implementation, the fields and field annotations for obtaining sensitive data of the target project include:

[0017] Obtain the class files of multiple target objects in the target project;

[0018] Retrieves the class name, field definitions, and field annotations from the class file;

[0019] Define at least one field corresponding to each class name as a set of fields;

[0020] Accordingly, sensitive data information is written into the sensitive object information table and the preset dictionary, including:

[0021] The class name and field set are stored as key-value pairs in a predefined dictionary; the class name is the value in the key-value pair, and the field set is the key; and the type, field set, field definition, and field annotation of the target object corresponding to the class name are stored in the sensitive object information table.

[0022] In one possible implementation, the sensitive data type and sensitivity level are determined based on the fields and field annotations of the sensitive data, including:

[0023] Extract multiple field annotations corresponding to the fields of sensitive data from the sensitive object information table;

[0024] For the target field annotation, the target field annotation is combined with each data type in the preset sensitive data classification table to obtain the sensitive data detection combination; the target field annotation can be any one of multiple field annotations;

[0025] The sensitive data detection combination is input into the sensitive data detector to obtain the detection response value; the sensitive data detector is pre-trained based on historical sensitive data;

[0026] When the detection response value is greater than the preset detection threshold, the target field annotation is determined to be a sensitive data type in the sensitive data detection combination;

[0027] Based on the preset sensitive data classification table, determine the sensitive data level corresponding to the target field annotation;

[0028] Accordingly, sensitive data information is written into the sensitive object information table, including:

[0029] Write the sensitive data type and sensitivity level of the target field annotation into the sensitive object information table.

[0030] In one possible implementation, when the sensitive data is an interface response message obtained based on a preset AOP aspect class, de-identification detection is performed on each sensitive data item in the sensitive object information table to obtain the de-identification detection result, including:

[0031] Parse the interface response message data to obtain the interface path information, interface fields, and interface field values;

[0032] Determine the target field set based on the interface fields;

[0033] When it is determined that there is a set of fields in the preset dictionary that is the same as the target set of fields, the sensitive data type and sensitivity level corresponding to the target set of fields are obtained from the sensitive object information table.

[0034] The detection results are obtained by using regular expressions to check whether the values ​​of interface fields have been de-identified.

[0035] Accordingly, a system sensitive data table is generated based on the desensitization detection results, including:

[0036] Write the interface path information, interface fields, interface field values, and desensitization detection results into the system's sensitive data table.

[0037] In one possible implementation, when the sensitive data is an object of an entity type in the target project, a de-identification detection is performed on each sensitive data item in the sensitive object information table to obtain the de-identification detection results, including:

[0038] Get the data table name corresponding to each entity type;

[0039] Obtain database connection information based on the target project's configuration file;

[0040] Obtain the target fields of the sensitive data corresponding to each entity type from the sensitive object information table;

[0041] Based on the database connection information and the data table name, determine the field value of the target field;

[0042] Based on the field values, determine whether sensitive data has been de-identified, and obtain the de-identification detection results;

[0043] Accordingly, a system sensitive data table is generated based on the desensitization detection results, including:

[0044] Write the data table name, target field, target field value, and desensitization detection result into the system's sensitive data table.

[0045] In one possible implementation, the method further includes:

[0046] Update the sensitive data detector based on the sensitive data ledger.

[0047] Secondly, embodiments of this application provide a data processing apparatus, including:

[0048] The acquisition module is used to write the acquired sensitive data information from the target project into a sensitive object information table and a preset dictionary;

[0049] The first processing module is used to perform desensitization detection on each sensitive data information in the sensitive object information table and obtain the desensitization detection result.

[0050] The second processing module is used to generate a system sensitive data table based on the desensitization detection results;

[0051] The third processing module is used to generate a sensitive data ledger based on the system's sensitive data table.

[0052] In one possible implementation, the acquisition module is further configured to:

[0053] Retrieve the fields and field annotations of sensitive data in the target project;

[0054] Based on the fields and field annotations of the sensitive data, determine the sensitive data type and sensitivity level of the sensitive data;

[0055] Based on the fields, field annotations, sensitive data types, and sensitivity levels of sensitive data, determine the sensitive data information;

[0056] Sensitive data information is written into the sensitive object information table and the preset dictionary.

[0057] In one possible implementation, the acquisition module is further configured to:

[0058] Obtain the class files of multiple target objects in the target project;

[0059] Retrieves the class name, field definitions, and field annotations from the class file;

[0060] Define at least one field corresponding to each class name as a set of fields;

[0061] Correspondingly, the acquisition module is also used for:

[0062] The class name and field set are stored as key-value pairs in a predefined dictionary; the class name is the value in the key-value pair, and the field set is the key; and the type, field set, field definition, and field annotation of the target object corresponding to the class name are stored in the sensitive object information table.

[0063] In one possible implementation, the acquisition module is further configured to:

[0064] Extract multiple field annotations corresponding to the fields of sensitive data from the sensitive object information table;

[0065] For the target field annotation, the target field annotation is combined with each data type in the preset sensitive data classification table to obtain the sensitive data detection combination; the target field annotation can be any one of multiple field annotations;

[0066] The sensitive data detection combination is input into the sensitive data detector to obtain the detection response value; the sensitive data detector is pre-trained based on historical sensitive data;

[0067] When the detection response value is greater than the preset detection threshold, the target field annotation is determined to be a sensitive data type in the sensitive data detection combination;

[0068] Based on the preset sensitive data classification table, determine the sensitive data level corresponding to the target field annotation;

[0069] Correspondingly, the acquisition module is also used for:

[0070] Write the sensitive data type and sensitivity level of the target field annotation into the sensitive object information table.

[0071] In one possible implementation, the first processing module is further configured to:

[0072] Parse the interface response message data to obtain the interface path information, interface fields, and interface field values;

[0073] Determine the target field set based on the interface fields;

[0074] When it is determined that there is a set of fields in the preset dictionary that is the same as the target set of fields, the sensitive data type and sensitivity level corresponding to the target set of fields are obtained from the sensitive object information table.

[0075] The detection results are obtained by using regular expressions to check whether the values ​​of interface fields have been de-identified.

[0076] Correspondingly, the second processing module is also used for:

[0077] Write the interface path information, interface fields, interface field values, and desensitization detection results into the system's sensitive data table.

[0078] In one possible implementation, the first processing module is further configured to:

[0079] Get the data table name corresponding to each entity type;

[0080] Obtain database connection information based on the target project's configuration file;

[0081] Obtain the target fields of the sensitive data corresponding to each entity type from the sensitive object information table;

[0082] Based on the database connection information and the data table name, determine the field value of the target field;

[0083] Based on the field values, determine whether sensitive data has been de-identified, and obtain the de-identification detection results;

[0084] Correspondingly, the second processing module is also used for:

[0085] Write the data table name, target field, target field value, and desensitization detection result into the system's sensitive data table.

[0086] In one possible implementation, the device further includes a fourth processing module for:

[0087] Update the sensitive data detector based on the sensitive data ledger.

[0088] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0089] The memory stores the instructions that the computer executes;

[0090] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect above and various possible implementations of the first aspect.

[0091] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and various possible implementations thereof.

[0092] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and various possible implementations thereof.

[0093] This application provides a data processing method, apparatus, device, medium, and product. The method involves writing sensitive data information from a target project into a sensitive object information table and a preset dictionary; performing de-identification detection on each sensitive data information in the sensitive object information table to obtain a de-identification detection result for each sensitive data information; generating a system sensitive data table based on the de-identification detection results; and then generating a sensitive data ledger based on the system sensitive data table. This application utilizes detailed sensitive data information obtained from the target project, combined with de-identification detection methods to perform de-identification detection on each sensitive data information, thereby achieving sensitive data detection at the basic project level. Simultaneously, it utilizes the system sensitive data table and sensitive data ledger for sensitive data management. Compared with existing technologies, this application combines the acquisition, storage, de-identification detection, and final ledger generation of sensitive data information to achieve complete sensitive data processing, avoiding problems such as insufficient coverage of preset rule matching and inaccurate machine learning detection, thus achieving the technical effect of improving the efficiency of sensitive data processing. Attached Figure Description

[0094] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0095] Figure 1 Flowchart of the data processing method provided in this application Figure 1 ;

[0096] Figure 2 Flowchart of the data processing method provided in this application Figure 2 ;

[0097] Figure 3 A schematic diagram of the framework of the sensitive data detector provided in the embodiments of this application;

[0098] Figure 4 Flowchart of the data processing method provided in this application Figure 3 ;

[0099] Figure 5 Flowchart of the data processing method provided in this application Figure 4 ;

[0100] Figure 6 A flowchart illustrating the sensitive data detection method provided in this application;

[0101] Figure 7 A schematic diagram of the data processing apparatus provided in this application;

[0102] Figure 8 A schematic diagram of the structure of the electronic device provided in this application.

[0103] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0104] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0105] First, the proper nouns used in this application will be explained:

[0106] Aspect-Oriented Programming (AOP) refers to a programming paradigm that enables the horizontal segmentation of code through pre-compiled or runtime dynamic proxies.

[0107] Web services refer to cross-platform services based on standardized protocols used for data exchange between systems.

[0108] Single-Page Application (SPA) framework: refers to a front-end framework that achieves page refresh-free operation by dynamically loading content.

[0109] Server-Side Rendering Framework (SSR) refers to the process of generating a complete page on the server side and sending it to the client.

[0110] Object-Relational Mapping (ORM) is a technical framework used in programming to map data represented by an object model to data in a relational database.

[0111] NoSQL (Not Only SQL) databases refer to a type of database technology designed to handle large-scale data and high-concurrency scenarios.

[0112] Continuous Integration / Continuous Deployment (CI / CD) tools refer to a software development practice that aims to frequently build, test, and release software in an automated manner.

[0113] Spring Boot refers to an open-source framework based on the Java ecosystem.

[0114] Masked And Correction BERT (MacBERT) model: This refers to an improved version of the BERT pre-trained model, primarily optimized by improving the language masking method. The Transformer in this model is a deep learning model architecture based on a self-attention mechanism.

[0115] AdamW optimizer: refers to an optimization algorithm used for training deep learning models. When optimizing a model based on AdamW optimizer, the accuracy of the model's classification is evaluated by calculating the model's performance metric F1, where F1 refers to the comprehensive performance metric formed by the combination of model precision and recall.

[0116] JSON refers to a lightweight data interchange format that uses key-value pairs to represent data, and collections are similar to objects or dictionaries in programming languages.

[0117] XML refers to an editing language used to identify structured data, and is a cross-platform data exchange format.

[0118] In existing technologies, the main methods for handling sensitive data are: using preset rule matching to match from a large amount of business data to detect sensitive data, and then further processing the detected sensitive data; or using machine learning models to extract features of sensitive data, using machine learning models to classify sensitive data, and then further processing the sensitive data based on the detection results.

[0119] However, existing rule-based matching technologies have limited coverage of manual rules and cannot identify semantically related but diverse sensitive fields. At the same time, due to the continuous changes in sensitive data, manual rules need to be updated frequently. Traditional machine learning models rely on manual feature engineering, which has limited feature expression capabilities and requires continuous feature updates based on changes in sensitive data to ensure detection accuracy. Therefore, existing technologies suffer from low efficiency in processing sensitive data.

[0120] To address the aforementioned technical problems, this application proposes the following technical concept: Sensitive data is collected and stored through a sensitive object information table and a preset dictionary; a de-identification mechanism is used to perform de-identification detection on each piece of sensitive data; the de-identification detection result for each piece of sensitive data is determined; and the de-identification detection results are integrated into an output sensitive data ledger. Compared with existing technologies, this application solves the problems of insufficient rule coverage and limited machine learning features by utilizing a detailed sensitive data detection mechanism. Simultaneously, it achieves automated, full-lifecycle management of sensitive data through system tables and ledgers, enabling sensitive data detection and management with low maintenance costs, thereby improving the efficiency of sensitive data processing.

[0121] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0122] Figure 1 Flowchart of the data processing method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0123] S101. Write the sensitive data information in the target project obtained into the sensitive object information table and the preset dictionary.

[0124] In this step, the target project includes multiple class files; sensitive data information includes: sensitive data corresponding to multiple class files; the sensitive data of each class file includes: the object type to which the class file belongs, the class name, the field set corresponding to the class name, at least one field definition in the field set, the field annotation corresponding to each field definition, the sensitive data type corresponding to each field annotation, and the sensitivity level corresponding to each field annotation; a sensitive object information table is used to store sensitive data information; a preset dictionary is used to store the mapping relationship between each class name and field set in the sensitive data information.

[0125] For example, a possible way to obtain sensitive data information from a target project is as follows:

[0126] S1011. Access the target project's directory, traverse the different target objects in the target project directory, and the class files under each target object.

[0127] S1012. For each class file, determine the class name, traverse the code of the structure corresponding to each class name, and obtain multiple field definitions and field annotations corresponding to each field definition by character matching.

[0128] It should be noted that the acquisition and storage of sensitive data information are as follows: Figure 2 Further explanation will be provided in the embodiments shown, and will not be repeated here.

[0129] S102. Perform desensitization detection on each sensitive data information in the sensitive object information table to obtain the desensitization detection results.

[0130] In this step, the method for de-identifying sensitive data is as follows: First, obtain the interface response message corresponding to the preset AOP aspect class. Compare the information in the preset dictionary and the interface response message to determine whether the sensitive data has been de-identified. Second, obtain the data table name corresponding to the entity type object in the target project. Combine the data table name to identify sensitive data, and perform de-identification detection based on this sensitive data to obtain the de-identification detection result.

[0131] The main implementation method of the desensitization detection in this step is as follows: compare each sensitive data information in the sensitive object information table with the information in the interface response message and the data table. If sensitive data information is found in the interface response message and the data table, determine whether there is a desensitization mark for the field based on the field corresponding to the sensitive data information in the interface response message and the data table. If there is a desensitization mark, it indicates that the sensitive data information corresponding to the field has been desensitized.

[0132] It should be noted that the sensitive data de-identification detection in this step uses different processing methods for interface response messages and data tables. The specific processing methods are as follows. Figure 4 Further explanation will be provided in the embodiments shown, and will not be repeated here.

[0133] S103. Generate a system sensitivity data table based on the desensitization detection results.

[0134] In this step, the desensitization detection result is used to indicate whether sensitive data has been desensitized; the system sensitive data table generated based on the desensitization detection result includes: the storage path or retrieval path of the sensitive data, the field definition of the sensitive data, the field value or field annotation corresponding to the field definition, and the desensitization detection result.

[0135] Optionally, the system sensitive data table may also include the detection object corresponding to the sensitive data. The detection object may be an interface response message or a data table; it may also include the sensitive data type corresponding to the field value or field annotation, as well as the sensitivity level of the data.

[0136] For example, the system's sensitive data table includes two data entries. The first entry includes: {Detection object: Interface response message; Acquisition path: / api / v1 / users; Field definition: phone; Field value: 18611183617; Sensitive data type: Mobile phone number; Sensitivity level: 1; Whether to de-identify: No}; The second entry includes: {Detection object: Data table; Acquisition path: ArchiveTraceEntity; Field definition: case; Field value: Hypertension; Sensitive data type: Medical history; Sensitivity level: 3; Whether to de-identify: Yes}. Here, "ArchiveTraceEntity" refers to the name of the data table; " / api / v1 / users" refers to the acquisition path of the interface response message.

[0137] S104. Generate a sensitive data ledger based on the system's sensitive data table.

[0138] In this step, after generating the sensitive data ledger, the sensitive data detector can also be updated based on the sensitive data ledger; the sensitive data detector is used to determine the corresponding sensitive data type and sensitive data level of each field in the sensitive data information during the process of writing sensitive data information into the sensitive object information table.

[0139] Alternatively, one possible implementation for generating a sensitive data ledger is as follows:

[0140] S1041. Output the system sensitive data table to the local machine and save it in an Excel spreadsheet.

[0141] S1042. Push the Excel data table to the management personnel of the target project.

[0142] The purpose of this step is to push the sensitive data information of the target project to the relevant management personnel so that they can understand the sensitive data situation and de-identification status of the target project.

[0143] S1043. If new information is detected in the system's sensitive data table, the new data will be manually reviewed and then added to the training dataset. The sensitive data detector will be retrained by periodically updating the training dataset.

[0144] The data processing method provided in this application involves writing sensitive data information from a target project into a sensitive object information table and a preset dictionary; performing de-identification detection on each sensitive data information in the sensitive object information table to obtain a de-identification detection result for each sensitive data information; generating a system sensitive data table based on the de-identification detection results; and then generating a sensitive data ledger based on the system sensitive data table. This application utilizes detailed sensitive data information obtained from the target project, combined with de-identification detection methods, to perform de-identification detection on each sensitive data information, thereby achieving sensitive data detection at the basic project level. Simultaneously, it manages sensitive data using the system sensitive data table and the sensitive data ledger. Compared with existing technologies, this application achieves complete sensitive data processing by combining the acquisition, storage, de-identification detection, and final ledger generation of sensitive data information, avoiding problems such as insufficient coverage of preset rule matching and inaccurate machine learning detection, thus achieving the technical effect of improving the efficiency of sensitive data processing.

[0145] Figure 2 Flowchart of the data processing method provided in this application Figure 2 ,like Figure 2 As shown, the method includes:

[0146] S201. Obtain the fields and field annotations of sensitive data for the target project.

[0147] Alternatively, one possible implementation for retrieving fields and field annotations that contain sensitive data is as follows:

[0148] S2011. Obtain the class files of multiple target objects in the target project.

[0149] In this step, the target project refers to an enterprise application or a modern web service. The target project is mainly used to quickly build standardized, scalable, and loosely coupled applications.

[0150] For example, the framework types used to build the target project include: backend frameworks, frontend frameworks, data layer frameworks, and infrastructure frameworks. Backend frameworks include full-stack frameworks, microservice frameworks, reactive frameworks, and lightweight frameworks. Frontend frameworks include SPA frameworks, SSR frameworks, and mobile frameworks; data layer frameworks include ORM frameworks, NoSQL tools, and big data processing frameworks; and infrastructure frameworks include containerization / orchestration, CI / CD tools, and monitoring / logging.

[0151] For example, this application uses the full-stack framework Spring Boot as an example. The target objects include Entity Class (Entity), Data Transfer Object (DTO), and View Object (VO). Entity defines the structure of the database table and provides the ability to interact with table records; DTO decouples Entity from business logic, improving security and performance; VO isolates backend data from frontend requirements, avoiding direct exposure of Entity or DTO.

[0152] S2012, retrieve the class name, field definitions of the class name, and field annotations of the field definitions from the class file.

[0153] For example, when the target object is an Entity, the class name, field definition, and field annotation are obtained as follows:

[0154] a1. Access the Entity directory and use the wildcard *Entity.java to match all files in the directory that end with Entity, resulting in multiple Entity class files.

[0155] For example, when the class file names are BankTraceEntity.java and UserInfoEntity.java, these two class files are the ones where the wildcard match is successful.

[0156] a2. For each class file, determine whether the file contains the string annotation "@data".

[0157] a3. If the string annotation "@data" exists, then obtain the class name corresponding to the string annotation.

[0158] a4. Scan the lines of code in the struct corresponding to the class name and extract the field definitions and field annotations from the lines of code.

[0159] For example: The detected class name is BankTraceEntity; the field definitions in the structure corresponding to this class name include: id, bankSerial, accountName, amount; the annotations corresponding to the field definitions are: primary key ID, bank transaction number, account name, and transaction amount.

[0160] For example, when the target object is a DTO, the class name, field definition, and field annotation are obtained as follows:

[0161] b1. Access the VO directory and use the wildcard *DTO.java to match all DTO class files.

[0162] b2. For each class file, check if the file contains the string annotation "@data".

[0163] b3. If the string annotation "@data" exists, then obtain the class name corresponding to the string annotation.

[0164] b4. Scan the lines of code in the struct corresponding to the class name and extract the field definitions and field annotations from the lines of code.

[0165] For example: The detected class name is: GuaranteedTenantsDTO; the field definitions in the structure corresponding to this class name include: tenantsId, leaseStartDate, monthlyRent; the annotations corresponding to the field definitions are: tenant ID, lease start date, monthly rent.

[0166] For example, when the target object is a VO, the class name, field definition, and field annotation are obtained as follows:

[0167] b1. Access the VO directory and use the wildcard *VO.java to match all DTO class files.

[0168] b2. For each class file, check if the file contains the string annotation "@data".

[0169] b3. If the string annotation "@data" exists, then obtain the class name corresponding to the string annotation.

[0170] b4. Scan the lines of code in the struct corresponding to the class name and extract the field definitions and field annotations from the lines of code.

[0171] For example: The detected class name is: GuaranteedTenantsVO; the field definitions in the structure corresponding to this class name include: tenantsName, idCardNumber, contactPhone; the annotations corresponding to the field definitions are: tenant's name, ID card number, and contact phone number.

[0172] S2013. Define at least one field corresponding to each class name as a set of fields.

[0173] In this step, a class name corresponds to at least one field definition. The purpose of defining the field definitions corresponding to each type as a set of fields is to create a mapping relationship between class names and field definitions.

[0174] S202. Based on the fields and field annotations of the sensitive data, determine the sensitive data type and sensitivity level of the sensitive data.

[0175] Optionally, a possible implementation of determining the sensitive data type and sensitive data level of sensitive data is as follows:

[0176] S2021. Extract multiple field annotations corresponding to the fields of the sensitive data from the sensitive object information table.

[0177] In this step, each sensitive data information in the sensitive object information table includes a field set, and each field definition in the field set corresponds to a field annotation.

[0178] S2022. For the target field annotation, combine the target field annotation with each data type in the preset sensitive data grading table to obtain a sensitive data detection combination.

[0179] In this step, the target field annotation is any one of the multiple field annotations.

[0180] Exemplarily, when detecting sensitive data, the sensitive data belongs to the information of a tenant, and the target field annotation is "ID number"; the data types in the preset sensitive data grading table include: identity identification information and financial account information; combining the target field annotation with each data type, the obtained sensitive data detection combinations are: ("ID number", "identity identification information"), ("ID number", "financial account information").

[0181] S2023. Input the sensitive data detection combination into the sensitive data detector to obtain a detection response value.

[0182] In this step, the sensitive data detector is pre-trained based on historical sensitive data.

[0183] Figure 3 It is a schematic structural diagram of the sensitive data detector provided by the embodiment of the present application; exemplarily, as Figure 3 shown, the sensitive data detector consists of an input layer 301, an analysis layer 302, and a classification layer 303. The input layer 301 splits the target field annotation and data type of the sensitive field to be detected into single characters, converts them into an input format acceptable to the MacBERT model, and inputs them into the analysis layer 302. The input sensitive data detection combination is ("mobile phone", "phone number"); use the input layer 302 to unpack, and mark the head of the string, the connection of the two strings, and the tail of the string. The information after splitting is: [CLS, mo, bi, le, di, an, SEP, shou, ji, hao, SEP]. Here, CLS refers to the beginning of the input sequence, and SEP refers to the end of the input sequence or the separation of two sentences.

[0184] Among them, parsing layer 302 is a MacBERT pre-trained model. This model uses 12 Transformer layers, has a hidden layer dimension of 768, and has 12 heads, which can perform semantic analysis tasks well. Figure 3 As shown, after parsing by the MacBERT pre-trained model, the parsing layer 302 obtains the final output vector C, and inputs the final output vector C into the classification layer 303.

[0185] Classification layer 303 is a fully connected neuron that is fully connected to the final output vector C of the MacBERT model through a neural network. It performs classification processing on the final output vector C of parsing layer 302 to obtain the classification result L. After classification layer 303 outputs the classification result, loss and activation functions can be used to calculate the loss and activate the data. Binary cross-entropy loss is chosen as the loss function, and normalization is chosen as the activation function to fix the value of the output result in the range of 0 to 1.

[0186] The training process for the sensitive data detector is as follows:

[0187] d1. By calling a preset script, scan all Entity classes, DTO classes, and VO classes in the historical codebase to obtain field definitions and field annotations. This ensures that the data covers all sensitive data categories and more comprehensive annotation writing habits, while also ensuring that the data volume of each type is relatively large, with no less than 1,000 samples for each type, thus obtaining historical sample data.

[0188] d2. Perform data cleaning on historical sample data, delete completely duplicate samples, and clean up useless punctuation and symbols.

[0189] d3. Based on business needs, determine a pre-defined sensitive data classification table as the standard for classifying and classifying sensitive data. The pre-defined sensitive data classification table should cover information such as data classification, definition, sensitive data types, and sensitivity levels. The pre-defined sensitive data classification table is set according to business characteristics and the data management requirements of business units.

[0190] d4. Based on the pre-defined sensitivity data classification table, the collected historical sample data is labeled to create a labeled sample dataset. According to the data classification, 70% of the samples from each class are randomly selected and placed into the training dataset for model parameter updates, 20% of the samples are selected and placed into the validation dataset for parameter tuning and training termination, and 10% of the samples are selected and placed into the test dataset for evaluating model performance.

[0191] d6. Input the sample training data based on the sample training dataset, and use MacBERT's built-in word segmenter to convert the raw text data in each sample training data into an input format that the model can understand.

[0192] d7. Load the MacBERT pre-trained model. Specifically, select the Base version of MacBERT and add a fully connected layer to perform binary classification.

[0193] It should be noted that MacBERT Base is a standard-sized pre-trained model.

[0194] d8. Configure training parameters, specifically: use the AdamW optimizer to prevent weight overfitting, and set the initial learning rate to 2e. -5 ,3e -5 ,4e -5 The maximum number of training rounds is set to 15. If the accuracy of the validation dataset does not improve by more than 1% for three consecutive rounds or the F1 score does not improve by more than 1% for three consecutive rounds, then the training is terminated, the model is saved, and models A1, A2, and A3 are obtained.

[0195] d9. Input the test dataset into models A1, A2, and A3 for detection, and select the model with the highest accuracy as the sensitive data detector.

[0196] S2024. When the detection response value is greater than the preset detection threshold, the target field annotation is determined to be a sensitive data type in the sensitive data detection combination.

[0197] The purpose of this step is to select combinations with confidence levels exceeding the threshold.

[0198] S2025. Based on the preset sensitive data classification table, determine the sensitive data level corresponding to the target field annotation.

[0199] The purpose of this step is to obtain the corresponding level based on the preset sensitive data classification table.

[0200] For example, the preset sensitive data classification table includes: {(Sensitive data source: individual customer; Field definition: basic information; Sensitive data type: mobile phone number; Sensitive data level: 2; Remarks: ordinary sensitive); (Sensitive data source: individual customer; Field definition: biometrics; Sensitive data type: fingerprint; Sensitive data level: 3; Remarks: critical sensitive); (Sensitive data source: individual customer; Field definition: management information; Sensitive data type: customer number; Sensitive data level: 1; Remarks: internally controllable); (Sensitive data source: organization; Field definition: organization information; Sensitive data type: social credit code; Sensitive data level: 0; Remarks: publicly available)}.

[0201] S203. Determine sensitive data information based on the fields, field annotations, sensitive data types, and sensitive data levels of the sensitive data.

[0202] In this step, determining sensitive data information refers to combining the fields, field annotations, sensitive data types, and sensitivity levels of sensitive data to obtain sensitive data information.

[0203] For example, the structure of sensitive data information can be: {"Field Definition": "idCardNumber", "Class Name": "GuaranteedTenantsVO", "Field Set": (idCardNumber), "Field Annotation": "ID Card Number", "Sensitive Data Type": "Identification Information", "Sensitive Data Level": "3"}. Here, the sensitive data information at this stage includes not only fields but also class names and the corresponding field set. That is, a single piece of sensitive data information includes sensitive data information from a class, which refers to the classes within the Entity, DTO, and VO objects mentioned in the above embodiments.

[0204] S204. Write the sensitive data information into the sensitive object information table and the preset dictionary.

[0205] The purpose of this step is to persistently store sensitive data information. The sensitive data information persistently stored in the sensitive object information table and preset fields can be used for subsequent data de-identification detection and the final generation of the sensitive data ledger.

[0206] Optionally, one possible implementation of writing sensitive data information into a sensitive object information table and a preset dictionary is as follows:

[0207] S2041. Store the class name and field set in a preset dictionary as key-value pairs, and store the type, field set, field definition, and field annotation of the target object corresponding to the class name in the sensitive object information table.

[0208] In this step, the class name serves as the value in the key-value pair, and the field set serves as the key. The presence of corresponding sensitive data information in the preset fields can be determined by matching the field sets; alternatively, the field sets can be queried by matching the class names. The information stored in the sensitive object information table must not only include the information in the preset fields but also the field definitions, field annotations, and the type of the target object corresponding to each piece of sensitive data. The target object can be an Entity, DTO, or VO object mentioned in the previous embodiments. The target object refers to the object from which the sensitive data information is obtained, typically the code project files and database files in the target project.

[0209] S2042. Write the sensitive data type and sensitivity level of the target field annotation into the sensitive object information table.

[0210] For example, the sensitive object information table, after storing the sensitive data type and sensitive data level, includes three sensitive data entries.

[0211] The first piece of sensitive data is as follows: {Target object type: Entity, Class name: BankMatchEntity, Field set: frozenset({“incomeAmt”, “dateAcc”, ...}), Field definition 1: incomeAmt, Field annotation 1: Income amount, Sensitive data type 1: Transaction amount information, Sensitivity level 1: 2, Field definition 2: dateAcc, Field annotation 2: Transaction date, Sensitive data type 2: Product transaction information, Sensitivity level 2: 1}.

[0212] The second piece of sensitive data information is: {Target object type: DTO, Class name: GuaranteedTeanantsDTO, Field set: frozenset({“teanantsName”, “managerPhone”, ...}), Field definition 1: teanantsName, Field annotation 1: Tenant name, Sensitive data type 1: Name, Sensitivity level 1: 2, Field definition 2: managerPhone, Field annotation 2: Tenant phone number, Sensitivity data type 2: Mobile phone number, Sensitivity level 2: 2}.

[0213] The third piece of sensitive data information is: {Target object type: VO, Class name: GuaranteedTenantsVO, Field set: frozenset({“teanantsName”, “tenantsType”, ...}), Field definition 1: teanantsName, Field annotation 1: Tenant name, Sensitive data type 1: Name, Sensitivity level 1: 2, Field definition 2: tenantsType, Field annotation 2: Tenant type, Sensitivity data type 2: Customer type, Sensitivity level 2: 1}.

[0214] Figure 4 Flowchart of the data processing method provided in this application Figure 3 ,like Figure 4 As shown, the method includes:

[0215] S401. Parse the interface response message data to obtain the interface path information, interface fields, and interface field values.

[0216] This step involves structured parsing of the interface response message data returned by the preset AOP interface to extract key metadata. The interface response message is usually in JSON or XML format and contains the interface access path, returned fields, and their corresponding values.

[0217] For example, the information obtained by parsing the interface response message may include the interface path information: / api / tenant / detail, the interface fields: ["tenantName", "idCard", "phone"], and the field value corresponding to each interface field: {"idCard": "310************123", "phone": "138****5678"}.

[0218] S402. Determine the target field set based on the interface fields.

[0219] In this step, all field names returned by the interface are combined into a set for subsequent comparison; this set reflects the data structure characteristics returned by the interface.

[0220] For example, the set of fields extracted by the interface is determined from the example in step S401 as: {“tenantName”, “idCard”, “phone”}.

[0221] In this step, the target field set is compared with the field set in the preset dictionary to determine whether there is sensitive data information in the sensitive object information table that corresponds to the interface response message.

[0222] S403. When it is determined that there is a set of fields in the preset dictionary that is the same as the target field set, the sensitive data type and sensitive data level corresponding to the target field set are obtained from the sensitive object information table.

[0223] In this step, the purpose of matching the target field set with the preset dictionary is to compare the field set of the current interface with the preset dictionary; the preset dictionary stores the mapping relationship between each class name and the field set; the field order does not affect the set comparison because the set is an unordered data structure; even if the order of the fields returned by the interface changes, as long as the field composition is the same, it can still be matched correctly.

[0224] For example, the dictionary is searched for a field combination that is exactly the same as {"tenantName", "idCard", "phone"}; if it exists, the sensitivity level of each field in the target field set is obtained from the sensitive object information table.

[0225] It should be noted that the sensitive data types and sensitivity levels corresponding to different fields in the target field set are not all the same. The field definition and field annotations are related to the detailed information in the sensitive object information table.

[0226] S404. Use regular expressions to detect whether the interface field values ​​have been de-identified, and obtain the de-identification detection results.

[0227] In this step, the format of the field values ​​marked as sensitive is validated to determine whether they have been de-identified according to the specifications, and regular expressions are used to match the industry-standard de-identification rules.

[0228] For example, the regular expression for detection is: ID card number: first 6 digits + asterisk + last 4 digits; mobile phone number: first 3 digits + asterisk + last 4 digits. The detected field values ​​are: "310************123": conforms to the ID card anonymization rules; "138****5678": conforms to the mobile phone number anonymization rules.

[0229] S405. Write the interface path information, interface fields, interface field values, and desensitization detection results into the system sensitive data table.

[0230] The purpose of this step is to store the complete evidence chain of interface sensitive data detection in the system ledger to form an audit trail.

[0231] For example, the system sensitive data table obtained from the record includes two data entries; the first data entry includes {path: " / api / tenant / detail", field: "idCard", field value: "310************123", sensitivity type: "identity information", sensitivity level: "3", whether to de-identify: "yes"}.

[0232] The second data entry includes: {API path: " / api / tenant / detail", field: "phone", field value: "138****5678", sensitivity type: "contact information", sensitivity level: "1", whether to de-identify: "yes"}.

[0233] Figure 5 Flowchart of the data processing method provided in this application Figure 4 ,like Figure 5 As shown, the method includes:

[0234] S501. Obtain the data table name corresponding to each entity type.

[0235] In this step, the correspondence between entity classes and database tables is established through object-relational mapping configuration or entity class annotations; the correspondence between entity classes and database tables is the bridge connecting the code layer and the database layer.

[0236] The mapping can be obtained from the target project's configuration file or from the annotation information in the entity class code package.

[0237] For example: the entity class is annotated with `@Entity(name = "t_tenant_info")`; by default, the corresponding database annotation is `t_tenant_info`. Alternatively, the corresponding database annotation can be determined using the `@table` annotation.

[0238] S502. Obtain database connection information based on the target project's configuration file.

[0239] In this step, database connection parameters, including network address, port, database name, username and password, are read from the target project's configuration file. Based on this information, a data acquisition channel is established to achieve database connection.

[0240] For example, the database connection information could be: Database address: jdbc:mysql: / / 10.0.1.25:3406 / tenant_db; Username: data_sec_admin; Username and password: 8888888; Connection pool size: 10.

[0241] S503. Obtain the target field of the sensitive data corresponding to each entity type from the sensitive object information table.

[0242] In this step, based on the previously established sensitive object information table, database fields that need to be monitored are selected; that is, the field definitions in the sensitive data information are used as target fields. These target fields have been marked as potentially containing sensitive information during the code scanning phase.

[0243] It should be noted that the purpose of identifying the target field in this step is to query the desensitization status of fields that may contain sensitive information.

[0244] S504. Based on the database connection information and the data table name, determine the field value of the target field.

[0245] In this step, a database query is executed to retrieve the actual stored data of the target field, i.e., the field value; the data anonymization is then verified based on the field value. It should be noted that to avoid performance impact from frequent queries and large amounts of data queries, the number and duration of data queries are typically limited.

[0246] For example, the relationship between the corresponding field values ​​obtained from querying the target field can be: {Target field: "id_card_no"; corresponding field values: ["310112199001011234", "310113*******5678"]}, {Target field: "mobile_phone"; corresponding field values: ["13812345678", "139****8888"]}. Here, "id_card_no" and "mobile_phone" refer to the target fields, and their corresponding values ​​refer to the field values.

[0247] S505. Determine whether sensitive data has been de-identified based on field values, and obtain the de-identification detection results.

[0248] The purpose of this step is to perform a compliance check on the original data stored in the database to ensure it is de-identified; the resulting statuses include: not de-identified at all, not de-identified separately, and completely de-identified.

[0249] For example, ID number "310112199001011234" has a desensitization test result of "not desensitized"; ID number "310113*******5678" has a desensitization test result of "desensitized"; mobile phone number "13812345678" has a desensitization test result of "not desensitized"; and mobile phone number "139****8888" has a desensitization test result of "desensitized".

[0250] S506. Write the data table name, target field, target field value, and desensitization detection result into the system sensitive data table.

[0251] In this step, the detection results of sensitive fields corresponding to sensitive data in the database are persistently stored to form a complete list of sensitive data assets, supporting subsequent rectification and tracking.

[0252] For example: The storage result of the system sensitive data table is: {Path: "t_tenant_base", Field definition: "id_card_no", Field value: "310112199001011234", Sensitive data type: "Identification information", Whether to de-identify: "No", Detection time: "2023-08-2014:40"; Path: "t_tenant_base", Field definition: "mobile_phone", Field value: "139****8888", Sensitive type: "Contact information", Whether to de-identify: "Yes", Detection time: "2023-08-2014:40"}.

[0253] Based on the above embodiments, this application also provides a complete process for generating a system sensitive data table from rice cultivation training with a sensitive data detector. Figure 6 A flowchart illustrating the sensitive data detection method provided in this application is shown below. Figure 6 As shown, the method includes:

[0254] A1. Create a sample dataset based on historical project files and databases.

[0255] A2. A sensitive data detector is obtained by training the model based on the sample dataset.

[0256] A3. Collect data based on the target project to obtain sensitive data.

[0257] A4. Sensitive data is detected using a sensitive data detector to obtain a table of sensitive object information.

[0258] A5. Capture the interface response message from the preset AOP interface.

[0259] A6. Traverse the database of the target project to obtain database table information.

[0260] A7. Data fusion is performed based on interface response messages, database table information, and sensitive object information table to obtain the system sensitive data table.

[0261] Figure 7 A schematic diagram of the data processing apparatus provided in this application is shown below. Figure 7 As shown, the data processing apparatus provided in this embodiment includes:

[0262] The acquisition module 701 is used to write the acquired sensitive data information from the target project into a sensitive object information table and a preset dictionary;

[0263] The first processing module 702 is used to perform desensitization detection on each sensitive data information in the sensitive object information table and obtain the desensitization detection result.

[0264] The second processing module 703 is used to generate a system sensitive data table based on the desensitization detection results.

[0265] The third processing module 704 is used to generate a sensitive data ledger based on the system's sensitive data table.

[0266] In one possible implementation, the acquisition module 701 is also used for:

[0267] Retrieve the fields and field annotations of sensitive data from the target project.

[0268] Based on the fields and field annotations of the sensitive data, determine the sensitive data type and sensitivity level of the sensitive data.

[0269] Sensitive data information is determined based on the fields, field annotations, sensitive data types, and sensitivity levels of the sensitive data.

[0270] Sensitive data information is written into the sensitive object information table and the preset dictionary.

[0271] In one possible implementation, the acquisition module 701 is also used for:

[0272] Obtain the class files of multiple target objects in the target project.

[0273] Retrieves the class name, field definitions of the class name, and field annotations of the field definitions from the class file.

[0274] Define at least one field corresponding to each class name as a set of fields.

[0275] Accordingly, the acquisition module 701 is also used for:

[0276] The class name and field set are stored as key-value pairs in a predefined dictionary; the class name is the value in the key-value pair, and the field set is the key; and the type, field set, field definition, and field annotation of the target object corresponding to the class name are stored in the sensitive object information table.

[0277] In one possible implementation, the acquisition module 701 is also used for:

[0278] Extract multiple field annotations corresponding to the fields of sensitive data from the sensitive object information table.

[0279] For the target field annotation, the target field annotation is combined with each data type in the preset sensitive data classification table to obtain the sensitive data detection combination; the target field annotation can be any one of multiple field annotations.

[0280] The sensitive data detection combination is input into the sensitive data detector to obtain the detection response value; the sensitive data detector is pre-trained based on historical sensitive data.

[0281] When the detection response value is greater than the preset detection threshold, the target field annotation is determined to be a sensitive data type in the sensitive data detection combination.

[0282] Based on the preset sensitive data classification table, determine the sensitive data level corresponding to the target field annotation;

[0283] Accordingly, the acquisition module 701 is also used for:

[0284] Write the sensitive data type and sensitivity level of the target field annotation into the sensitive object information table.

[0285] In one possible implementation, the first processing module 702 is further configured to:

[0286] Parse the interface response message data to obtain the interface path information, interface fields, and interface field values.

[0287] Determine the target field set based on the interface fields.

[0288] When it is determined that there is a set of fields in the preset dictionary that is the same as the target set of fields, the sensitive data type and sensitivity level corresponding to the target set of fields are obtained from the sensitive object information table.

[0289] The results of the de-identification detection are obtained by using regular expressions to check whether the values ​​of interface fields have been de-identified.

[0290] Accordingly, the second processing module 703 is also used for:

[0291] Write the interface path information, interface fields, interface field values, and desensitization detection results into the system's sensitive data table.

[0292] In one possible implementation, the first processing module 702 is further configured to:

[0293] Get the data table name corresponding to each entity type.

[0294] Obtain database connection information based on the target project's configuration file.

[0295] Obtain the target fields of the sensitive data corresponding to each entity type from the sensitive object information table.

[0296] Based on the database connection information and the table name, determine the field value of the target field.

[0297] Based on the field values, determine whether sensitive data has been anonymized, and obtain the anonymization detection results.

[0298] Accordingly, the second processing module 703 is also used for:

[0299] Write the data table name, target field, target field value, and desensitization detection result into the system's sensitive data table.

[0300] In one possible implementation, the device further includes a fourth processing module 705, for:

[0301] Update the sensitive data detector based on the sensitive data ledger.

[0302] The apparatus provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0303] Figure 8 A schematic diagram of the structure of the electronic device provided in this application. Figure 8As shown, the electronic device provided in this embodiment includes at least one processor 801 and a memory 802. Optionally, the device further includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.

[0304] In a specific implementation, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to execute the aforementioned data processing method or procedure.

[0305] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0306] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0307] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0308] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0309] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0310] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0311] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0312] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0313] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0314] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0315] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0316] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0317] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0318] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A data processing method, characterized in that, include: The sensitive data information obtained from the target project is written into the sensitive object information table and the preset dictionary; Desensitization detection is performed on each sensitive data item in the sensitive object information table to obtain the desensitization detection result; A system sensitivity data table is generated based on the desensitization detection results; A sensitive data ledger is generated based on the system's sensitive data table.

2. The method according to claim 1, characterized in that, The step of writing the obtained sensitive data information from the target project into the sensitive object information table and the preset dictionary includes: Obtain the fields and field annotations of the sensitive data of the target project; Based on the fields of the sensitive data and the field annotations, determine the sensitive data type and sensitivity level of the sensitive data; The sensitive data information is determined based on the fields of the sensitive data, the field annotations, the sensitive data type, and the sensitive data level. The sensitive data information is written into the sensitive object information table and the preset dictionary.

3. The method according to claim 2, characterized in that, The fields and field annotations for obtaining sensitive data of the target project include: Obtain the class files of multiple target objects in the target project; Obtain the class name of the class file, the field definitions of the class name, and the field annotations of the field definitions; Define at least one field corresponding to each class name as a set of fields; Accordingly, writing the sensitive data information into the sensitive object information table and the preset dictionary includes: The class name and the field set are stored in a preset dictionary as key-value pairs; wherein the class name is the value in the key-value pair and the field set is the key in the key-value pair; and the type of the target object corresponding to the class name, the field set, the field definition, and the field annotation are stored in the sensitive object information table.

4. The method according to claim 3, characterized in that, The step of determining the sensitive data type and sensitivity level of the sensitive data based on the fields of the sensitive data and the field annotations includes: Extract multiple field annotations corresponding to the fields of the sensitive data from the sensitive object information table; For the target field annotation, the target field annotation is combined with each data type in the preset sensitive data classification table to obtain a sensitive data detection combination; the target field annotation is any one of the multiple field annotations; The sensitive data detection combination is input into the sensitive data detector to obtain the detection response value; the sensitive data detector is pre-trained based on historical sensitive data. When the detection response value is greater than a preset detection threshold, the target field annotation is determined to be the sensitive data type in the sensitive data detection combination; Based on the preset sensitive data classification table, the sensitive data level corresponding to the target field annotation is determined; Accordingly, writing the sensitive data information into the sensitive object information table includes: Write the sensitive data type and sensitive data level of the target field annotation into the sensitive object information table.

5. The method according to claim 2, characterized in that, When the sensitive data is an interface response message obtained based on a preset AOP aspect class, the process of performing desensitization detection on each sensitive data information in the sensitive object information table to obtain the desensitization detection result includes: Parse the interface response message data to obtain the interface path information, interface fields, and interface field values; Based on the interface fields, determine the target field set; When it is determined that there is a set of fields in the preset dictionary that is the same as the target field set, the sensitive data type and sensitive data level corresponding to the target field set are obtained from the sensitive object information table. The desensitization detection result is obtained by detecting whether the interface field value has been desensitized using regular expressions; Accordingly, the generation of the system sensitive data table based on the desensitization detection results includes: The interface path information, the interface field, the interface field value, and the desensitization detection result are written into the system sensitive data table.

6. The method according to claim 2, characterized in that, When the sensitive data is an object of an entity type in the target project, the step of performing desensitization detection on each sensitive data information in the sensitive object information table to obtain the desensitization detection result includes: Obtain the data table name corresponding to each entity type; Obtain database connection information based on the configuration file of the target project; Obtain the target fields of the sensitive data corresponding to each entity type from the sensitive object information table; Based on the database connection information and the data table name, determine the field value of the target field; Based on the field value, it is determined whether the sensitive data has been de-identified, and the de-identification detection result is obtained; Accordingly, the generation of the system sensitive data table based on the desensitization detection results includes: Write the data table name, the target field, the field value of the target field, and the de-identification detection result into the system sensitive data table.

7. The method according to claim 4, characterized in that, The method further includes: The sensitive data detector is updated based on the sensitive data ledger.

8. A data processing apparatus, characterized in that, include: The acquisition module is used to write the acquired sensitive data information from the target project into a sensitive object information table and a preset dictionary; The first processing module is used to perform desensitization detection on each sensitive data information in the sensitive object information table and obtain the desensitization detection result. The second processing module is used to generate a system sensitive data table based on the desensitization detection results; The third processing module is used to generate a sensitive data ledger based on the system's sensitive data table.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images