Structured sensitive data identification system and method based on Roberta model

Abstract

The invention discloses a structured sensitive data identification system and method based on a Roberta model, and belongs to the field of data processing and information security. Aiming at the problems of low recognition accuracy and poor generalization ability of a traditional method in complex structured data, the invention constructs a multi-modal feature fusion and joint task learning architecture: performing field fine tuning on a Roberta model to extract multi-modal features such as texts, numerical values, dates and the like; designing cross-modal feature splicing and a multi-head self-attention mechanism to realize feature enhancement; and constructing an explicit-implicit sensitive data identification joint task model, and realizing adaptive identification optimization in combination with a dynamic Prompt project. According to the scheme of the invention, the recognition capability of deformation data and implicit sensitive information can be remarkably improved, F1 values on data sets in the fields of finance, medical treatment and the like all exceed 98%, the problems of sensitive information detection and protection in a structured data environment are effectively solved, and the compliance requirements of multiple industries are met.

Description

Technical Field

[0001] The present invention belongs to the field of data processing and information security technology, and specifically relates to a structured sensitive data recognition technology that integrates natural language processing (NLP) and multimodal feature analysis. Background Art

[0002] In today's rapidly advancing digital age, data has become a core driver of development across all industries. Structured sensitive data, a key component of data assets, encompasses important content such as personal privacy information, commercial secrets, and critical national data. The importance of its security is self-evident. With the rapid advancement of information technology, businesses and institutions have accumulated massive amounts of structured data, stored in various databases, spreadsheets, and business systems. However, the widespread circulation and sharing of data has also posed unprecedented security challenges to structured sensitive data.

[0003] As a key line of defense for data security, structured sensitive data identification technology aims to accurately identify sensitive information within structured data through advanced algorithms and models, providing a solid foundation for secure data storage, transmission, and use. This technology comprehensively analyzes the characteristics and semantics of data, deeply mining potentially sensitive information, including but not limited to ID numbers, bank card numbers, medical records, and trade secrets. By effectively identifying this sensitive information, structured sensitive data identification technology can help businesses and institutions promptly identify data security risks and implement targeted protective measures, thereby reducing the risk of data leakage and misuse.

[0004] With the rapid development of emerging technologies such as the Internet, the Internet of Things, and Big Data, the generation and dissemination of data is experiencing explosive growth, and the network environment is becoming increasingly complex and volatile. New attack methods and security threats, such as data theft, malicious tampering, and data leakage, are constantly emerging, placing enormous pressure on the security of structured sensitive data. Furthermore, the forms and structures of data are becoming increasingly diverse, with traditional structured data intertwined with semi-structured and unstructured data, further increasing the difficulty of identifying sensitive data.

[0005] However, existing technologies for identifying structured sensitive data have numerous limitations. For one thing, some technologies rely on fixed rules and templates, lacking a deep understanding of data semantics. This makes them difficult to adapt to dynamic changes and complex data structures, resulting in low accuracy in identifying deformed data and implicit sensitive information. Furthermore, some technologies struggle with multimodal data, failing to fully explore the connections and potential value between data, limiting their effectiveness. Furthermore, existing technologies often struggle to cope with emerging security threats and complex network environments, lacking effective adaptive and evolutionary capabilities.

[0006] Therefore, providing a structured sensitive data identification solution that can adapt to changes in the network environment and the evolution of security threats and has deep semantic understanding, multimodal processing and adaptive optimization capabilities has become a technical problem that needs to be urgently solved in this field. Summary of the Invention

[0007] In response to the problems existing in existing structured sensitive data identification technologies, the purpose of the present invention is to provide a structured sensitive data identification solution based on the Roberta model. This solution, through an improved Roberta pre-trained model, has deep semantic understanding, multimodal processing and adaptive optimization capabilities. It can adapt to changes in network environments and the evolution of security threats, and achieve accurate, efficient and comprehensive detection and identification of explicit and implicit sensitive information in structured data such as databases, spreadsheets, and corporate reports.

[0008] To achieve the above objectives, the present invention provides a structured sensitive data identification system based on the Roberta model, comprising:

[0009] A multimodal feature extraction module configured to extract features from three fields in structured data, namely text, numeric, and date, using the Roberta model to generate corresponding semantic feature vectors, numeric feature vectors containing format and semantics, and time series feature vectors;

[0010] a cross-modal feature fusion module, the cross-modal feature fusion module being configured to be associated with the multimodal feature extraction module and capable of processing the extracted multimodal feature vectors to generate a weighted fused feature vector;

[0011] A joint task identification module is configured to be associated with a cross-modal feature fusion module to construct an explicit-implicit sensitive data identification dual-task architecture that shares the Roberta model feature extraction layer, wherein the explicit task branch adopts a Bi-LSTM+CRF sequence labeling model to output field-level sensitive labels for identifying sensitive data of known formats; the implicit task branch adopts a TextCNN convolutional neural network to output sentence-level sensitive probability values ​​for detecting potential sensitive information associated with contextual semantics; and a joint loss function L=αL is used to calculate the sensitivity of the data. 显示 +βL 隐含 +γL reg Perform model training, where L 显示 is the focal loss to solve the problem of sample imbalance, L 隐含 It adopts binary cross entropy loss and introduces contrastive learning mechanism. The dynamic weight parameters α and β are automatically adjusted according to the training rounds through the AdaGrad algorithm.

[0012] A dynamic prompt engineering optimization module is associated with the joint task identification module to build a prompt library containing templates specific to multiple fields. The module uses the T5 model to generate the initial prompt. The module uses the reinforcement learning algorithm with the F1 value as the reward function to dynamically adjust the keyword weights in the prompt, forming a closed-loop optimization system.

[0013] In some embodiments of the present invention, the multimodal feature extraction module includes:

[0014] A text feature extraction submodule, which uses a pre-trained Roberta model to perform bidirectional semantic enhancement fine-tuning on text fields in structured data, improves the model's robustness to deformed data through adversarial training strategies, and generates high-dimensional semantic feature vectors containing contextual dependencies;

[0015] A numerical feature parsing submodule uses a regular expression-semantic fusion algorithm for numerical fields. It first extracts format features through regular expressions, then uses the Roberta model to parse the business meaning of the field, generating a high-dimensional numerical feature vector that contains both format and semantics.

[0016] The date feature extraction submodule extracts the local dependency features of the time series through TCN, combines the contextual semantics generated by the Roberta model, extracts absolute features and relative features, and generates a multi-dimensional time series feature vector.

[0017] In some embodiments of the present invention, the cross-modal feature fusion module includes:

[0018] The normalization submodule uses a normalization layer to perform Z-score normalization on text, numerical, and date feature vectors, mapping each dimension data to the interval [-1, 1] to eliminate the impact of numerical scale differences on model training;

[0019] Position coding submodule: The position coding submodule generates a unique position identifier for each field through the position coding layer, and retains the field order information after splicing with the feature vector to avoid semantic dislocation during fusion;

[0020] The multi-head self-attention submodule adopts an 8-head self-attention mechanism to calculate the attention weight of each modal feature. Each head independently calculates the dot product similarity of the query, key, and value vectors. The output results of the 8 heads are concatenated and integrated through a linear layer to generate a 2048-dimensional fused feature vector.

[0021] In some embodiments of the present invention, the joint task identification module includes:

[0022] A shared feature extraction layer, which is a Roberta model consisting of 12 Transformer encoding layers and a 768-dimensional hidden layer, outputs cross-modal fusion features;

[0023] The explicit sensitive data identification branch inputs cross-modal fusion features, extracts sequence dependency features through the Bi-LSTM layer, and then generates field-level labels through the CRF layer;

[0024] The implicit sensitive data recognition branch inputs the cross-modal fusion features, extracts local semantic features through the TextCNN layer, and outputs sentence-level sensitivity probabilities through the pooling layer and the fully connected layer;

[0025] A joint loss calculation submodule calculates the dynamic weighted joint loss function L = αL_explicit + βL_implicit + γL_reg, where L_explicit is the focal loss to solve the sample imbalance problem, L_implicit is the binary cross entropy loss and introduces a contrastive learning mechanism, and the dynamic weight parameters α and β are automatically adjusted according to the training rounds through the AdaGrad algorithm, and λ is the L2 regularization coefficient; and the parameters are updated after the losses of the explicit task and the implicit task are calculated.

[0026] In some embodiments of the present invention, the dynamic prompt engineering optimization module includes:

[0027] A Prompt template library management submodule manages a domain-specific Prompt template library. The Prompt template library includes two types of templates: explicit templates and implicit templates. Explicit templates contain field-level detection instructions, while implicit templates contain sentence-level semantic analysis instructions.

[0028] A prompt generation submodule, which generates an optimized prompt including variable replacement based on the T5 model, the input data segment, and the preliminary recognition result;

[0029] The reinforcement learning submodule uses the F1 value as the reward function, performs a strategy update every time a preset number of prompts are generated, adjusts the weight distribution of keywords in the template through gradient descent, and supports dynamic expansion of custom sensitive data type sets.

[0030] To achieve the above objectives, the present invention further provides a structured sensitive data identification method based on the Roberta model, the identification method comprising:

[0031] Perform multimodal feature extraction on structured data, using the pre-trained Roberta model to extract feature vectors for text, numeric, and date fields respectively;

[0032] Perform cross-modal feature fusion on multimodal feature vectors and generate fused feature vectors through normalization, position encoding and multi-head self-attention mechanism;

[0033] For the generated fusion feature vector, sensitive data is identified through the explicit-implicit dual-task model, and the joint loss function is calculated to update the model parameters;

[0034] Based on the dynamic prompt engineering, we optimized detection instructions, built a prompt library, used the T5 model to generate the initial prompt, and adjusted the keyword weights through the reinforcement learning algorithm to improve the model's adaptability to different scenarios.

[0035] In some embodiments of the present invention, the recognition method, when extracting multimodal features, includes:

[0036] The Roberta model is used to perform semantic modeling on text fields, and dynamic word embedding technology is used to process complex texts to generate semantic feature vectors.

[0037] Perform three-layer parsing on numeric fields to extract format features, business meaning, and semantic associations, and generate numeric feature vectors.

[0038] For date fields, TCN and Roberta models are combined to extract time series features and generate time series feature vectors.

[0039] In some embodiments of the present invention, the identification method, during dynamic prompt engineering optimization, includes:

[0040] A dedicated prompt template library for the management field, divided into explicit and implicit templates, used according to different needs;

[0041] Generate an initial prompt based on the T5 model, input data fragments and preliminary recognition results for optimization;

[0042] Using the F1 value as the reward function, the keyword weights in Prompt are adjusted through a reinforcement learning algorithm, dynamically expanding the set of sensitive data types.

[0043] The solution provided by the present invention uses an improved Roberta pre-trained model to achieve accurate detection of explicit and implicit sensitive information in structured data such as databases, spreadsheets, and corporate reports. It is suitable for fields with strict data security requirements such as finance, medical care, and government affairs, and solves the problem of sensitive data identification and protection in complex data environments.

[0044] The solution provided by the present invention solves the problems of low recognition accuracy, poor generalization ability, and insufficient multimodal processing in complex data environments in existing technologies through methods such as multimodal feature extraction, cross-modal feature fusion, joint task recognition model, and dynamic prompt engineering optimization. It can provide strong support for enterprises and institutions in data security protection and effectively reduce the risk of data leakage and abuse.

[0045] The solution of the present invention can significantly improve the ability to identify deformed data and implicit sensitive information. The F1 value on data sets in the financial, medical and other fields exceeds 98%, effectively solving the problem of sensitive information detection and protection in structured data environments and meeting the compliance requirements of multiple industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0047] Figure 1 This is a schematic diagram of the overall architecture of the structured sensitive data identification system of the present invention;

[0048] Figure 2 This is a schematic diagram of the composition of the multimodal feature extraction module in the present invention;

[0049] Figure 3 This is a schematic diagram of the composition of the cross-modal feature fusion module in the present invention;

[0050] Figure 4 This is a schematic diagram of the structure of the joint task identification module in the present invention;

[0051] Figure 5 This is a schematic diagram of the structure of the dynamic prompt engineering optimization module in the present invention;

[0052] Figure 6 This is a deployment architecture diagram of the structured sensitive data identification software system in an example of the present invention;

[0053] Figure 7 This is an example diagram of the structure of the sensitive data identification system in the example of the present invention. DETAILED DESCRIPTION

[0054] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below with reference to specific illustrations.

[0055] The present invention aims to provide a structured sensitive data identification solution based on the Roberta model, which can accurately identify explicit and implicit sensitive information in structured data such as databases and spreadsheets through multimodal feature fusion, joint task learning and dynamic prompt engineering.

[0056] The present invention provides a structured sensitive data identification system based on the Roberta model. Figure 1 , which is the overall architectural principle diagram of this structured sensitive data identification system.

[0057] Based on the diagram, the structured sensitive data identification system 100 is mainly composed of a data preprocessing engine 110, a multimodal feature extraction module 120, a cross-modal feature fusion module 130, a joint task identification module 140 and a dynamic prompt engineering optimization module 150.

[0058] The data preprocessing engine 110 in this system is configured to perform field type identification, null value filling and outlier filtering operations on the input structured data, wherein field type identification adopts a regular expression-based pattern matching algorithm, null value filling implements the KNN interpolation strategy, and outlier filtering applies the isolation forest detection model.

[0059] The multimodal feature extraction module 120 in this system is configured to extract features from three types of fields: text, numerical values, and dates in structured data;

[0060] Specifically, for text fields, the pre-trained Roberta model is used to perform bidirectional semantic enhancement fine-tuning on text fields in structured data. An adversarial training strategy is used to improve the model's robustness to deformed data and generate a 2048-dimensional semantic feature vector containing contextual dependencies.

[0061] For numeric fields, a regular expression-semantic fusion algorithm is used. Regular expressions are used to extract format features (such as the length and prefix rules of ID card numbers). The Roberta model is then used to parse the business meaning of the field (such as whether a bank card number prefix corresponds to the issuing bank information). This generates a 128-dimensional numerical feature vector that contains both format and semantics.

[0062] For date fields, the Temporal Convolutional Network (TCN) and the Roberta model are combined to extract absolute features such as year / month / day and relative features such as the length of time from the current date and the sequence of key time points, generating a 64-dimensional time series feature vector.

[0063] The cross-modal feature fusion module 130 in the present system is configured to be associated with the multimodal feature extraction module 120 and is capable of processing the extracted multimodal feature vectors to generate weighted fused feature vectors.

[0064] This cross-modal feature fusion module 130 specifically processes the scale differences of text, numerical, and date feature vectors through Z-score normalization, uses positional encoding technology to retain field order information, and splices the three types of features by channel to form a 2240-dimensional comprehensive feature vector; then uses the 8-head self-attention mechanism to calculate the global similarity weight of each modal feature, and generates a weighted fusion feature vector after normalization through the Softmax function, highlighting the contribution of the key modality to the recognition result.

[0065] This cross-modal feature fusion module 130 achieves feature enhancement through cross-modal feature splicing and multi-head self-attention mechanism.

[0066] The joint task identification module 140 in this system is configured to be associated with the cross-modal feature fusion module 130, configured with a Roberta model of a bidirectional Transformer architecture, and synchronously output explicit sensitive data identification results and implicit sensitive data identification results.

[0067] Specifically, the joint task identification module 140 constructs an explicit-implicit sensitive data identification dual-task architecture that shares the Roberta model feature extraction layer. The explicit-implicit sensitive data identification dual-task architecture includes parallel explicit task branches and implicit task branches.

[0068] Among them, the explicit task branch uses the Bi-LSTM+CRF sequence labeling model to output field-level sensitive labels (B-SENSITIVE, I-SENSITIVE, O) for identifying sensitive data in known formats (such as ID card numbers and bank card numbers); the implicit task branch uses the TextCNN convolutional neural network to output sentence-level sensitivity probability values ​​of 0-1 for detecting potential sensitive information with contextual semantic associations (such as trade secrets and undisclosed financial data).

[0069] On this basis, we further design the joint loss function L = αL 显示 +βL 隐含 +γL reg Perform model training, where L 显示 is the focal loss to solve the problem of sample imbalance, L 隐含 A binary cross entropy loss is used and a contrastive learning mechanism is introduced. The dynamic weight parameters α and β are automatically adjusted according to the training rounds through the AdaGrad algorithm.

[0070] The dynamic Prompt engineering optimization module 150 in this system is associated with the joint task identification module 140 to build a Prompt library containing exclusive templates for multiple fields (such as 100+ fields), and use the T5 model to generate the initial Prompt. Through the reinforcement learning algorithm (such as PPO) with the F1 value as the reward function, the keyword weight in the Prompt is dynamically adjusted (such as increasing the trigger priority of "disease name" in medical data scenarios), forming a closed-loop optimization system.

[0071] The templates here are preferably stored in JSON format, supporting custom sensitive data type sets (such as {ID number, medical diagnosis code}) and industry keyword libraries.

[0072] The resulting structured sensitive data recognition system builds a multimodal feature fusion and joint task learning architecture. By fine-tuning the Roberta model in the field, it extracts multimodal features such as text, numbers, and dates, and further enhances features through cross-modal feature splicing and multi-head self-attention mechanism. It builds an explicit-implicit sensitive data recognition joint task model and combines it with dynamic prompt engineering to achieve adaptive recognition optimization.

[0073] The following further explains the structure of each functional module in this system.

[0074] See also Figure 2 In the specific implementation, the multimodal feature extraction module 120 in this system is mainly composed of a text feature extraction submodule 121, a numerical feature analysis submodule 122 and a date feature extraction submodule 123.

[0075] Among them, the text feature extraction submodule 121 is used to perform semantic modeling on the text field using the Roberta model of a 12-layer Transformer encoder, learn vocabulary-level semantic associations through the masked language model task, and combine dynamic word embedding technology to process mixed simplified and traditional Chinese and Chinese and English texts to generate high-dimensional vectors containing sentence-level semantics.

[0076] The numerical feature parsing submodule 122 performs three-level parsing on numerical fields: the first level matches the field format through regular expressions (such as the 18-digit rule of the ID card number), the second level uses pre-built field coding mapping tables (such as administrative division code tables, bank issuing bank prefix tables) to parse business meanings, and the third level uses the Roberta model to analyze the semantic association between the field and other fields (such as the logical correspondence between "date of birth" and "ID card number").

[0077] The date feature extraction submodule 123 extracts the local dependency features of the time series for date fields through TCN, and combines the contextual semantics generated by the Roberta model to construct a time series feature matrix containing seasons, weeks, and time intervals. The relative time feature accuracy is down to the day level (such as "30 days from the current date").

[0078] See also Figure 3 The cross-modal feature fusion module 130 in this system is mainly composed of a normalization sub-module 131, a position encoding sub-module 132 and a multi-head self-attention sub-module 133 in its specific implementation.

[0079] Among them, the normalization submodule 131 uses the normalization layer to perform Z-score normalization on the text (2048 dimensions), numerical value (128 dimensions), and date (64 dimensions) feature vectors, mapping the data of each dimension to the [-1,1] interval to eliminate the impact of numerical scale differences on model training.

[0080] The position coding submodule 132 generates a unique position identifier for each field through the position coding layer (such as "name" is 1, "ID number" is 2), and retains the field order information after splicing with the feature vector to avoid semantic dislocation during fusion.

[0081] The multi-head self-attention submodule 133 adopts an 8-head self-attention mechanism to calculate the attention weight of each modal feature. Each head independently calculates the dot product similarity of the Query, Key, and Value vectors. The output results of the 8 heads are concatenated and integrated through a linear layer to generate a 2048-dimensional fused feature vector. The attention weight matrix dynamically reflects the importance of each modal feature.

[0082] See also Figure 4 The joint task identification module 140 in this system is mainly composed of a shared feature extraction layer 141, an explicit sensitive data identification branch 142, an implicit sensitive data identification branch 143 and a joint loss calculation submodule 144 in its specific implementation.

[0083] Among them, the shared feature extraction layer 141 is a Roberta model consisting of 12 Transformer encoding layers and 768-dimensional hidden layers. It supports batch processing of 1,000 structured data items per time and outputs cross-modal fusion features.

[0084] After inputting the fused features, the explicit sensitive data identification branch 142 extracts the sequence dependency features through the Bi-LSTM layer (hidden layer dimension 256), and then generates field-level labels through the CRF layer, supporting parallel identification of up to 5 types of sensitive data (such as ID number, passport number, and driver's license number).

[0085] After inputting the fused features, the implicit sensitive data recognition branch 143 extracts local semantic features through the TextCNN layer (convolution kernels with three different window sizes: 3, 4, and 5), outputs the sentence-level sensitivity probability through the pooling layer and the fully connected layer, and is judged as sensitive with a threshold of 0.7.

[0086] The joint loss calculation submodule 144 calculates the joint loss function L = αL 显示 +βL 隐含 +γL reg , where L 显示 is the focal loss to solve the problem of sample imbalance, L 隐含 A binary cross-entropy loss is used, and a contrastive learning mechanism is introduced. The dynamic weight parameters α and β are automatically adjusted based on the number of training rounds using the AdaGrad algorithm. λ is the L2 regularization coefficient. In the joint loss function, α is initially set to 0.8 (focusing on explicit tasks) and is gradually reduced by 0.1 steps every five training rounds until α = β = 0.5. γ is the L2 regularization coefficient, ranging from 1e-4 to 1e-3, and its optimal value is determined through tuning on a validation set.

[0087] The joint task identification module 140 thus formed specifically trains and runs the following logic process:

[0088] (1) First, input preprocessing:

[0089] Structured data is preprocessed into a format acceptable to the model (e.g., tokenization or sub-tokenization and combined with structured features) to form a sequence of feature vectors containing cross-modal information.

[0090] The data is organized into batches (batch size supports 1000 records / batch).

[0091] (2) Next, feature extraction (Forward Pass):

[0092] Batch data is input into the shared feature extraction layer (RoBERTa); RoBERTa's 12-layer Transformer encoder processes the input layer by layer, learns the association and context information between words through the self-attention mechanism, and fuses cross-modal features; it outputs a shared feature vector.

[0093] (3) Next, parallel task processing:

[0094] Explicit branching:

[0095] The shared feature vector is input into the Bi-LSTM layer. The LSTM unit processes the features of each token sequentially, taking into account both the forward and backward hidden states, to obtain an enhanced contextual feature representation for each token.

[0096] The output of the Bi-LSTM is fed into the CRF layer. The CRF layer receives the predicted state score for each token (from the Bi-LSTM), considers the learned transition relationships between states (labels), and calculates the optimal label sequence with the highest overall probability through dynamic programming (such as the Viterbi algorithm).

[0097] Implicit branch:

[0098] The shared feature vector is input into the TextCNN layer (processing the feature matrix of the entire sentence simultaneously). Three different convolution kernel sizes (sizes 3, 4, and 5) slide on the feature matrix to extract semantic features of local regions (n-grams) and generate multiple feature maps.

[0099] The feature map passes through the maximum pooling layer, extracts the maximum value of each feature map (that is, retains the strongest feature signal), and forms three fixed-length feature vectors (one for each convolution kernel).

[0100] The pooled feature vectors are concatenated and fed into a fully connected layer. The fully connected layer learns the combined weights of these features and ultimately outputs a sentence-level sensitivity probability value S through an activation function (such as Sigmoid).

[0101] (4) Joint loss calculation (Backward Pass-backward propagation starting point):

[0102] Explicit task loss: calculated using FocalLoss.

[0103] Latent task loss: calculated using binary cross entropy loss (BCELoss).

[0104] Calculate the joint loss function:

[0105] L=αL 显示 +βL 隐含 +γL reg ;

[0106] Among them, the dynamic weight adjustment mechanism:

[0107] α and β are hyperparameters used to balance the weights of the two tasks, and α + β = 1;

[0108] Initial settings: α = 0.8, β = 0.2;

[0109] The dynamic weight parameters α and β are automatically adjusted based on the number of training epochs using the AdaGrad algorithm. The adjustment strategy is: every five training epochs, α is reduced by 0.1 and β is increased by 0.1 until both reach 0.5 (i.e., both tasks are equally important). This process simulates a more balanced learning of the two tasks by the model in the later stages of training.

[0110] L2 regularization (L_reg) penalizes all weight parameters of the model to prevent overfitting, with a coefficient of γ.

[0111] (5) Model parameter update (Backward Pass):

[0112] Calculate the final joint loss value L.

[0113] Backpropagation is used to calculate the gradient of the joint loss L with respect to all trainable model parameters (including RoBERTa parameters, Bi-LSTM parameters, CRF layer parameters, CNN convolution kernel parameters, and fully connected layer parameters). An optimizer is used to update the model parameters based on the calculated gradients to minimize the joint loss L. This process is repeated over batches of data.

[0114] See also Figure 5 The dynamic prompt engineering optimization module 150 in this system is mainly composed of a prompt template library management submodule 151, a prompt generation submodule 152, and a reinforcement learning submodule 153 in its specific implementation.

[0115] Among them, the Prompt template library management submodule 151 manages the domain-specific Prompt template library. The Prompt template library is divided into two categories: explicit templates and implicit templates. The explicit templates contain field-level detection instructions (such as "Please identify whether the following fields contain {sensitive identity information}: [field content], possible types include {ID number, passport number}"); implicit templates contain sentence-level semantic analysis instructions (such as "Analyze whether the following text involves {commercial secrets}: [text content], the judgment basis includes {core technology, undisclosed data}").

[0116] The prompt generation submodule 152 is implemented based on the T5 model. After inputting the data segment and the preliminary recognition result, it outputs an optimized prompt containing variable replacement (such as replacing "sensitive identity information" with the specific industry term "medical patient identification").

[0117] The reinforcement learning submodule 153 uses the F1 value as the reward function, performs a strategy update every time 100 prompts are generated, adjusts the weight distribution of keywords in the template through gradient descent, and supports dynamic expansion of custom sensitive data type sets (such as adding a "biometric data" category).

[0118] The structured sensitive data identification system based on the Roberta model formed based on the above solution has built a full-process technical system from data feature extraction to sensitive information identification through modular design, effectively improving the sensitive data detection capabilities in complex scenarios.

[0119] The following describes the process of identifying structured sensitive data based on this structured sensitive data identification system. The entire identification process includes the following steps:

[0120] (1) Perform multimodal feature extraction on structured data and use pre-trained Roberta model and other technologies to extract feature vectors of text, numerical value and date fields respectively.

[0121] (2) Perform cross-modal feature fusion on the multimodal feature vector and generate a fused feature vector through normalization, position encoding and multi-head self-attention mechanism.

[0122] (3) Use the joint task recognition model to identify sensitive data, perform recognition through the explicit-implicit dual-task architecture, and calculate the joint loss function to update the model parameters.

[0123] (4) Based on the dynamic prompt engineering optimization detection instructions, a prompt library is built, the initial prompt is generated using the T5 model, and the keyword weight is adjusted through the reinforcement learning algorithm to improve the model's adaptability to different scenarios.

[0124] Among them, step (1) is further implemented through the following sub-steps during multimodal feature extraction:

[0125] (11) The Roberta model is used to perform semantic modeling on text fields, and dynamic word embedding technology is used to process complex texts to generate semantic feature vectors;

[0126] (12) Perform three-layer parsing on the numeric field to extract format features, business meanings, and semantic associations, and generate a numeric feature vector;

[0127] (13) For date fields, TCN and Roberta models are combined to extract time series features and generate time series feature vectors.

[0128] Among them, step (4) is further implemented through the following sub-steps during the dynamic prompt engineering optimization:

[0129] (41) A dedicated prompt template library for the management field, divided into explicit and implicit templates, used according to different needs;

[0130] (42) Generate the initial prompt based on the T5 model, input the data fragment and the preliminary recognition results for optimization;

[0131] (43) Using F1 value as the reward function, the weight of keywords in Prompt is adjusted through reinforcement learning algorithm, and the set of sensitive data types is dynamically expanded.

[0132] The following further illustrates the implementation process of this solution and its corresponding performance characteristics through specific application examples.

[0133] The Roberta model-based structured sensitive data identification system solution of the present invention, when applied, can be configured as a corresponding software program, forming corresponding structured sensitive data identification software. When running, this software program will perform the functions of the structured sensitive data identification system described above and will be stored in a corresponding storage medium for access and execution by a processor.

[0134] See also Figure 6 , which shows the architecture of the structured sensitive data identification system software based on the Roberta model in this example.

[0135] This example is developed based on Python 3.9 and adopts a four-tier architecture of "frontend-backend-database-model service". The specific implementation is as follows:

[0136] Front-end layer:

[0137] Technology stack: Vue 3.0 + Element UI, providing data import (supports Excel / CSV batch upload, single file ≤ 2GB), visual reports (sensitive data distribution heat map, recognition result statistics table), template customization and other functions.

[0138] Interaction design: Supports field-level recognition result preview. Click on sensitive fields to view detailed information (such as "ID number recognition confidence 98.5%, matching regular R001 and semantic rule S002"). The interface response delay is ≤100ms.

[0139] Backend layer:

[0140] Model Serving: Roberta and joint task models are deployed using PyTorch, supporting GPU acceleration (NVIDIA A100). The single-card inference speed reaches 300 items per second, and the throughput exceeds 2000 items per second when multiple cards are used in parallel (8 cards).

[0141] Data processing: Structured data is processed through the Dask distributed framework, supporting parallel cleaning (deduplication and missing value filling) of files over 10GB in size. Processing 10GB of data takes ≤ 8 minutes.

[0142] API interface: Provides a RESTful API (such as / sensitive_detection), returns JSON format results, including the location, type, risk level and rectification suggestions of sensitive fields, and the interface response time is ≤50ms (95% of requests).

[0143] Database layer:

[0144] Main database: PostgreSQL 13, which stores raw data, model parameters, and historical recognition records. It supports second-level queries for 100,000 data records and optimizes storage capacity through table sharding (compression rate 60%).

[0145] Cache: Redis stores frequently accessed prompt templates, domain knowledge bases, and intermediate calculation results. The key-value pair read and write latency is less than 1ms, and the hit rate is over 95%.

[0146] Model serving layer:

[0147] It uses Docker containerized deployment and supports elastic expansion. The resource usage of a single container is: 4 CPU cores, 8GB of memory, and the container startup time is ≤30 seconds.

[0148] See also Figure 7 , this example specifically configures a structured sensitive data identification system 200 in the backend layer to complete data processing.

[0149] Based on the diagram, the structured sensitive data identification system 200 is composed of six core units, forming a complete technical chain of "data input - feature processing - intelligent identification - strategy optimization - result output":

[0150] Multimodal feature extraction module 210, which extracts semantic, format, and temporal features of text, numeric, and date fields;

[0151] Cross-modal feature fusion module 220, which implements standardization, position encoding, and attention-weighted fusion of multiple types of features;

[0152] Joint task identification module 230, which identifies sensitive data through a dual-task model of explicit sequence labeling and latent semantic classification;

[0153] Dynamic Prompt Engineering Module 240, based on domain template library and reinforcement learning dynamic optimization detection;

[0154] Data preprocessing module 250, supporting structured data cleaning, format conversion and sample balancing;

[0155] Result output module 260: generates sensitive data reports and supports multi-format export and system integration.

[0156] On this basis, the modules in this structured sensitive data identification system 200 interact through standardized data interfaces to ensure efficient data flow at the levels of feature vectors, identification results, and strategy parameters. The overall system throughput reaches 500 items per second (single-card GPU environment).

[0157] Furthermore, the multimodal feature extraction module 210 in the structured sensitive data identification system 200 designs differentiated feature extraction strategies for the three core types of structured data, specifically including:

[0158] The text semantic feature extraction unit 211 adopts the deeply optimized Roberta model (12-layer Transformer, hidden layer dimension 768), and supports GBK / UTF-8 multi-encoding format text processing. Through adversarial training (adding 15% random Token mask), the robustness of deformed text is improved, and the character-level recognition accuracy of typos such as "ID number" and "bank account number" reaches 95.2%. Furthermore, the model outputs a 2048-dimensional semantic vector, which contains vocabulary-level entity tags (such as "ID number" corresponds to B-IDENTITY) and sentence-level dependency features (such as "address field contains administrative division code"). After fine-tuning with financial and medical corpora, the entity recognition F1 value reaches 98.3%.

[0159] The numerical feature parsing unit 212 builds a two-layer architecture of "rule matching + semantic reasoning":

[0160] Rule layer: Pre-configured with 100+ regular expressions (e.g., ID card number regular expression: =^[1-9]\d{5}(18|19|20|21)\d{2}(0[1-9]|10|11|12)(0[1-9]|

[12] \d|30|31)\d{3}[0-9Xx]$), supports fault-tolerant matching (e.g., "110101-2000-01-01-123" containing the "-" separator), and achieves a format detection accuracy of 97.6%;

[0161] The semantic layer associates industry-specific knowledge bases (such as financial institution coding libraries and administrative division code libraries) and uses the Roberta model to parse the business meaning of numerical fields (such as "622848" corresponds to the Agricultural Bank of China debit card). It generates a 128-dimensional feature vector, of which the business sensitivity feature weight accounts for 70%, and the business meaning parsing accuracy rate reaches 99.1%.

[0162] The date feature time series unit 213 combines the TCN network and the Roberta model to extract three types of time features:

[0163] Absolute features: Year / month / day numeric values ​​(e.g., "2023-10-01" is parsed as [2023,10,1]), with 100% time formatting accuracy;

[0164] Relative feature: The time interval from the current date (e.g., "the contract expiration date is 30 days from now"), with a time difference calculation error of ≤ 1 day;

[0165] Periodic features: Quarterly / weekly periodic information (e.g., “the first day of each quarter” is marked as a sensitive node) with a recall rate of 96.8%.

[0166] Finally, a 64-dimensional time series feature matrix is ​​generated to support sensitive data association analysis within the time window (such as high-frequency trading time point detection delay ≤ 20ms).

[0167] The cross-modal feature fusion module 220 in the structured sensitive data identification system 200 specifically includes:

[0168] The feature normalization submodule 221 uses the Z-score normalization method to process the dimensional differences of different modal features. The formula is:

[0169]

[0170] Here, μ is the feature mean and σ is the standard deviation. This ensures that the data distribution variance of text, numeric, and date features is kept below 0.8, preventing vanishing gradients and improving the stability of subsequent model training. Furthermore, the normalization process supports parallel computing, with a single batch (1,000 data items) processing time of ≤5ms.

[0171] The position encoding submodule 222 generates a unique position embedding vector for each field (e.g., the "name" field is encoded as [1,0,0], and the "ID number" field is encoded as [0,1,0]). This vector preserves the field order using the sine-cosine position encoding algorithm. Experiments show that the fusion feature with position encoding improves sequence labeling accuracy by 3.2% compared to a scheme without position encoding (e.g., the recognition rate of the semantic difference between "ID number-name" and "name-ID number" increased from 85% to 88.2%).

[0172] The multi-head self-attention submodule 223 uses an 8-head self-attention mechanism (Single Head Dimension = 64) to calculate the attention weight of each modal feature:

[0173]

[0174] Single-head computation latency is ≤10μs, and overall processing speed reaches 1500 records / second (batch size = 32). In financial data scenarios, the weight of numerical features is automatically increased to 65%, the weight of text semantic features is 25%, and the weight of date features is 10%. The F1 value of sensitive data recognition is improved by 12% compared to the single-modal processing solution.

[0175] The joint task identification module 230 in the structured sensitive data identification system 200 specifically includes:

[0176] The shared semantic encoding layer reuses the first eight layers of the Roberta model's Transformer encoder, improving batch processing efficiency by 40%. With a batch size of 64, throughput reaches 1,200 items per second, saving 30% of computing resources compared to a full-layer fine-tuned solution. Furthermore, the encoding layer uses FP16 mixed-precision training, reducing memory usage by 50% (from 24GB to 12GB).

[0177] The explicit sensitive data identification branch implements field-level sequence labeling based on the Bi-LSTM+CRF architecture:

[0178] The Bi-LSTM layer (hidden layer dimension 256) captures long-range dependencies (such as the association between the region code and date of birth in the ID card number), and the sequence feature extraction latency is ≤15ms;

[0179] The CRF layer generates the optimal label sequence, supports more than five sensitive type annotations such as "B-ID, I-ID, O" and "B-CARD, I-CARD", and the single-field recognition accuracy of explicit data reaches 99.2% (financial dataset).

[0180] The implicit sensitive data recognition branch uses a TextCNN model with three convolution kernels (window sizes 3, 4, and 5). It extracts n-gram semantic features (such as "R&D costs" and "core technology"), processes them through global max pooling, and then feeds them into a fully connected layer, outputting a 0-1 sensitivity probability value (a threshold of 0.7 indicates sensitive). The recognition accuracy for text related to "undisclosed market strategies" reached 91.2%, with a Kappa consistency coefficient of 0.89 compared to manual annotations.

[0181] Joint training mechanism, design of dynamic weight loss function:

[0182] α starts at 0.8 (explicit tasks take precedence) and decreases by 0.1 every five iterations. β corresponds to the implicit task weight, and λ = 0.001 is the L2 regularization coefficient. This mechanism improves model convergence speed by 20% in scenarios where the proportion of sensitive data is less than 10%, reducing the total number of training rounds from 30 to 20, while maintaining a stable F1 score above 98%.

[0183] The dynamic prompt engineering module 240 in the structured sensitive data identification system 200 specifically includes:

[0184] The template library management unit maintains an industry-specific Prompt template library containing more than 100 templates, supporting explicit templates (such as "Detect whether the following fields contain {sensitive identity information}: {field value}, supported types: {ID number, passport number}") and implicit templates (such as "Analyze sensitive business information in the text: {text content}, focus on keywords: {commercial secrets, technical patents}").

[0185] Furthermore, the template library supports version management (such as financial version V1.0, medical version V2.0), template retrieval delay ≤ 5ms (Redis cache support), and can process 1,000 template calls per second.

[0186] The intelligent generation unit, based on the T5 model (finely tuned for NLI tasks), generates context-aware prompts. After entering "employee salary table" data, it automatically generates the following prompt: "Please identify whether the table contains {salary-sensitive information}: {field value}, judgment basis: {monthly salary, year-end bonus}." The template generation time is ≤ 80ms per time, and the adaptation efficiency for medical diagnosis record scenarios is 300% higher than manual configuration. The generated template has a business relevance accuracy rate of 91%.

[0187] Reinforcement learning optimization unit, using PPO algorithm to optimize Prompt parameters, the reward function is defined as: R = 0.6·F1+0.3·Precision+0.1·Recall

[0188] After 500 rounds of iterative training, the F1 value of implicit sensitive data recognition increased from 82% to 94%, and the strategy update frequency was once for every 100 data processed, forming an efficient closed-loop system of "data input-template generation-effect feedback-strategy optimization".

[0189] Combine Figure 6 and Figure 7 As shown, the structured sensitive data identification system software formed based on this has the following corresponding process in specific applications:

[0190] (1) Data preprocessing stage

[0191] Cleaning and transformation: Use pandas for data cleaning, remove invalid fields, and handle missing values ​​(fill numeric values ​​with the mean and mark text values ​​with "[unknown]"). The data cleaning accuracy rate reaches 99.5%.

[0192] Sample balancing: Using the SMOTE oversampling algorithm, the proportion of sensitive data was increased from 8% to 25%, class imbalance was reduced by 60%, and model training stability was improved.

[0193] (2) Model training phase

[0194] Pre-training loading: Load roberta-large pre-training weights through the Hugging Face library, freeze the parameters of the first 4 layers, and fine-tune the last 8 layers. The weight loading time is ≤ 2 minutes.

[0195] Joint training: Explicit tasks and implicit tasks are trained alternately, and shared layer parameters are updated synchronously in each iteration. During the training process, the GPU utilization rate is maintained above 85%, and the training cycle of a single A100 card is about 12 hours.

[0196] (3) Real-time recognition stage:

[0197] Multimodal processing: Text fields use Roberta to generate semantic vectors (500 items / second), numeric fields trigger regularization + semantic dual-layer parsing (800 items / second), and date fields extract time series features (600 items / second). The overall processing throughput reaches 300 items / second (three modes in parallel).

[0198] Dynamic Prompt Matching: Automatically selects corresponding templates based on data fields (such as "medical" and "finance") and adjusts detection strategies (for example, medical data prioritizes "medical records" and "medical insurance card numbers"). Strategy matching delay is ≤10ms.

[0199] (4) Result output stage:

[0200] Report generation: Generate Word / Excel / PDF reports based on a standardized template engine, including a list of sensitive data, risk level statistics, and rectification suggestions. The report generation time for 1,000 records is ≤ 20 seconds.

[0201] System integration: Integrate with data management systems (such as Hadoop and MySQL) through APIs to automatically trigger the desensitization process (such as masking the identified ID number to "110101********123"), with integration delay ≤ 30ms.

[0202] The following is a comparative experiment on various indicators between the structured sensitive data identification system software formed in this example and the traditional structured sensitive data identification method.

[0203] Here, we use the structured sensitive data identification system software developed in this example and traditional structured sensitive data identification methods to conduct sensitive data identification experiments on the same data sets in the financial and medical fields. The corresponding core technical indicators are compared in the table below:

[0204]

[0205]

[0206] The experimental comparison data above shows that this solution is suitable for scenarios with strict data security requirements:

[0207] Financial Industry: In the bank customer information system, the system achieved a 99.2% accuracy rate in identifying sensitive fields for 100,000 new data items added daily, with a missed detection rate of less than 0.1%;

[0208] Medical field: In electronic medical record systems, the recognition delay for implicitly sensitive information such as "cancer diagnosis" and "infectious disease history" is ≤30ms, supporting real-time compliance review;

[0209] Government data: In the census database, it supports batch sensitive screening of 200 data items per second, ensuring the security detection efficiency of tens of millions of data items.

[0210] Through modular design and intelligent optimization, this example solution can provide efficient and accurate structured sensitive data identification solutions for various industries, and promote the intelligent upgrade of data security protection technology.

[0211] With respect to the above-mentioned structured sensitive data identification system software solution based on the Roberta model, an embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the functions of the above-mentioned structured sensitive data identification system are realized.

[0212] An embodiment of the present invention further provides a processor, which is used to run a program, wherein the program executes the functions of the above-mentioned structured sensitive data identification system when running.

[0213] An embodiment of the present invention also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. The program code is loaded and executed by the processor to implement the functions of the above-mentioned structured sensitive data identification system.

[0214] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for performing the functions of the above-mentioned structured sensitive data identification system.

[0215] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0216] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0217] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0218] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0219] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0220] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0221] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0222] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0223] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0224] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0225] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0226] The aforementioned methods, or specific system units, or portions thereof, of the present invention are purely software-based and can be implemented as program code on physical media, such as a hard drive, optical disk, or any electronic device (e.g., a smartphone or computer-readable storage medium). When a machine loads and executes the program code (e.g., a smartphone), the machine becomes a device for implementing the present invention. The aforementioned methods and devices of the present invention can also be transmitted in program code form via some transmission medium, such as a cable, optical fiber, or any other transmission method. When the program code is received, loaded, and executed by a machine (e.g., a smartphone), the machine becomes a device for implementing the present invention.

[0227] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A structured sensitive data identification system based on the Roberta model, characterized in that: include: A multimodal feature extraction module configured to extract features from three fields in structured data, namely text, numeric, and date, using the Roberta model to generate corresponding semantic feature vectors, numeric feature vectors containing format and semantics, and time series feature vectors; a cross-modal feature fusion module, the cross-modal feature fusion module being configured to be associated with the multimodal feature extraction module and capable of processing the extracted multimodal feature vectors to generate a weighted fused feature vector; A joint task identification module is configured to be associated with a cross-modal feature fusion module to construct an explicit-implicit sensitive data identification dual-task architecture that shares the Roberta model feature extraction layer, wherein the explicit task branch adopts a Bi-LSTM+CRF sequence labeling model to output field-level sensitive labels for identifying sensitive data of known formats; the implicit task branch adopts a TextCNN convolutional neural network to output sentence-level sensitive probability values ​​for detecting potential sensitive information associated with contextual semantics; and a joint loss function L=αL is used to calculate the sensitivity of the data. 显示 +βL 隐含 +γL reg Perform model training, where L 显示 is the focal loss to solve the problem of sample imbalance, L 隐含 It adopts binary cross entropy loss and introduces contrastive learning mechanism. The dynamic weight parameters α and β are automatically adjusted according to the training rounds through the AdaGrad algorithm. A dynamic prompt engineering optimization module is associated with the joint task identification module to build a prompt library containing templates specific to multiple fields. The module uses the T5 model to generate the initial prompt. The module uses the reinforcement learning algorithm with the F1 value as the reward function to dynamically adjust the keyword weights in the prompt, forming a closed-loop optimization system.

2. The structured sensitive data identification system based on the Roberta model according to claim 1, characterized in that: The multimodal feature extraction module includes: A text feature extraction submodule, which uses a pre-trained Roberta model to perform bidirectional semantic enhancement fine-tuning on text fields in structured data, improves the model's robustness to deformed data through adversarial training strategies, and generates high-dimensional semantic feature vectors containing contextual dependencies; A numerical feature parsing submodule uses a regular expression-semantic fusion algorithm for numerical fields. It first extracts format features through regular expressions, then uses the Roberta model to parse the business meaning of the field, generating a high-dimensional numerical feature vector that contains both format and semantics. The date feature extraction submodule extracts the local dependency features of the time series through TCN, combines the contextual semantics generated by the Roberta model, extracts absolute features and relative features, and generates a multi-dimensional time series feature vector.

3. The structured sensitive data identification system based on the Roberta model according to claim 1, characterized in that: The cross-modal feature fusion module includes: The normalization submodule uses a normalization layer to perform Z-score normalization on text, numerical, and date feature vectors, mapping each dimension data to the interval [-1, 1] to eliminate the impact of numerical scale differences on model training; Position coding submodule: The position coding submodule generates a unique position identifier for each field through the position coding layer, and retains the field order information after splicing with the feature vector to avoid semantic dislocation during fusion; The multi-head self-attention submodule adopts an 8-head self-attention mechanism to calculate the attention weight of each modal feature. Each head independently calculates the dot product similarity of the query, key, and value vectors. The output results of the 8 heads are concatenated and integrated through a linear layer to generate a 2048-dimensional fused feature vector.

4. The structured sensitive data identification system based on the Roberta model according to claim 1, characterized in that: The joint task identification module includes: A shared feature extraction layer, which is a Roberta model consisting of 12 Transformer encoding layers and a 768-dimensional hidden layer, outputs cross-modal fusion features; The explicit sensitive data identification branch inputs cross-modal fusion features, extracts sequence dependency features through the Bi-LSTM layer, and then generates field-level labels through the CRF layer; The implicit sensitive data recognition branch inputs the cross-modal fusion features, extracts local semantic features through the TextCNN layer, and outputs sentence-level sensitivity probabilities through the pooling layer and the fully connected layer; A joint loss calculation submodule calculates the dynamic weighted joint loss function L = αL_explicit + βL_implicit + γL_reg, where L_explicit is the focal loss to solve the sample imbalance problem, L_implicit is the binary cross entropy loss and introduces a contrastive learning mechanism, and the dynamic weight parameters α and β are automatically adjusted according to the training rounds through the AdaGrad algorithm, and λ is the L2 regularization coefficient; and the parameters are updated after the losses of the explicit task and the implicit task are calculated.

5. The structured sensitive data identification system based on the Roberta model according to claim 1, characterized in that: The dynamic prompt engineering optimization module includes: A Prompt template library management submodule manages a domain-specific Prompt template library. The Prompt template library includes two types of templates: explicit templates and implicit templates. Explicit templates contain field-level detection instructions, while implicit templates contain sentence-level semantic analysis instructions. A prompt generation submodule, which generates an optimized prompt including variable replacement based on the T5 model, the input data segment, and the preliminary recognition result; The reinforcement learning submodule uses the F1 value as the reward function, performs a strategy update every time a preset number of prompts are generated, adjusts the weight distribution of keywords in the template through gradient descent, and supports dynamic expansion of custom sensitive data type sets.

6. A structured sensitive data identification method based on the Roberta model, characterized in that: The identification method comprises: Perform multimodal feature extraction on structured data, using the pre-trained Roberta model to extract feature vectors for text, numeric, and date fields respectively; Perform cross-modal feature fusion on multimodal feature vectors and generate fused feature vectors through normalization, position encoding and multi-head self-attention mechanism; For the generated fusion feature vector, sensitive data is identified through the explicit-implicit dual-task model, and the joint loss function is calculated to update the model parameters; Based on the dynamic prompt engineering, we optimized detection instructions, built a prompt library, used the T5 model to generate the initial prompt, and adjusted the keyword weights through the reinforcement learning algorithm to improve the model's adaptability to different scenarios.

7. The structured sensitive data identification method based on the Roberta model according to claim 6, characterized in that: The identification method includes the following steps when extracting multimodal features: The Roberta model is used to perform semantic modeling on text fields, and dynamic word embedding technology is used to process complex texts to generate semantic feature vectors. Perform three-layer parsing on numeric fields to extract format features, business meaning, and semantic associations, and generate numeric feature vectors. For date fields, TCN and Roberta models are combined to extract time series features and generate time series feature vectors.

8. The structured sensitive data identification method based on the Roberta model according to claim 6, characterized in that: The identification method, when optimizing the dynamic prompt project, includes: A dedicated prompt template library for the management field, divided into explicit and implicit templates, used according to different needs; Generate an initial prompt based on the T5 model, input data fragments and preliminary recognition results for optimization; Using the F1 value as the reward function, the keyword weights in Prompt are adjusted through a reinforcement learning algorithm, dynamically expanding the set of sensitive data types.

Images