A sensitive information extraction method and system based on natural language processing

Abstract

The present application provides a sensitive information extraction method and system based on natural language processing, which passes training text without supervision information into the algorithm for sensitive information extraction, assigns pseudo labels to the training text without supervision information in combination with the extracted basic character sequence set, determines the training text without supervision information and the pseudo labels as training samples of the algorithm, and trains the algorithm without sample annotation, saving costs. In the generation of pseudo character sequences, the basic character sequences obtained by executing different sensitive information extraction operators in the algorithm on the same training text are compared, and similar basic character sequences belong to the same sensitive information classification. The similar character sequences corresponding to the same basic character sequence are merged, and the obtained pseudo character sequence is more accurate, which makes the algorithm iteration update more accurate, thereby increasing the efficiency and accuracy of sensitive information extraction.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a sensitive information extraction method and system based on natural language processing. Background Art

[0002] With the rapid development of Internet technology, the amount of information is growing explosively. It is crucial to efficiently and accurately extract sensitive information from massive, unstructured text data. Sensitive information, such as personal identity information, financial information, and trade secrets, is of great significance to personal privacy protection, corporate security, and national security. Traditional sensitive information extraction methods mostly rely on manual rules or simple keyword matching. These methods are incapable of processing complex and changeable text data and are inefficient. With the continuous advancement of natural language processing (NLP) technology, especially the widespread application of deep learning and machine learning technologies, new solutions have been brought to sensitive information extraction. However, existing methods mostly rely on a large amount of well-annotated training data. The construction cost of these data sets is high and the cycle is long, which limits the widespread application of the technology. In addition, a single sensitive information extraction model is often difficult to fully cover all types of sensitive information. Different models show different advantages in processing specific types of sensitive information. How to integrate the advantages of multiple models to improve the accuracy and efficiency of sensitive information extraction has become an important research direction. Summary of the invention

[0003] In view of this, the present application provides a sensitive information extraction method and system based on natural language processing. The technical solution of the present application is implemented as follows:

[0004] On the one hand, the present application provides a sensitive information extraction method based on natural language processing, the method comprising: obtaining a text to be extracted, and passing the text to be extracted into a target sensitive information extraction algorithm to obtain a set of sensitive character sequences corresponding to the text to be extracted; the target sensitive information extraction algorithm is trained by the following operations: obtaining a first training text library; the first training text library is a training text library for unsupervised information; each first training text in the first training text library is passed into a basic sensitive information extraction algorithm to obtain a set of basic character sequences corresponding to each first training text; the set of basic character sequences corresponding to the first training text includes the basic character sequences obtained by respectively executing each sensitive information extraction operator in the basic sensitive information extraction algorithm and the sensitive character sequences corresponding to the basic character sequences. sensitive information classification; for the same basic character sequence set, combined with the character sequence similarity scores between the basic character sequences belonging to the same sensitive information classification, the basic character sequence obtained by executing the target sensitive information extraction operator is compared with the basic character sequences obtained by executing each other sensitive information extraction operator, and a similar character sequence corresponding to the basic character sequence obtained by executing the target sensitive information extraction operator is obtained; the basic character sequence and the corresponding similar character sequences are merged to obtain a false character sequence, and combined with the false character sequences corresponding to the same first training text, a false character sequence set corresponding to the first training text is obtained; combined with the first training text library and the corresponding false character sequence sets, the basic sensitive information extraction algorithm is iteratively updated to obtain a target sensitive information extraction algorithm.

[0005] On the other hand, the present application provides a computer system, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps in the above method when executing the program.

[0006] The beneficial effects of the present application include: the sensitive information extraction method and system based on natural language processing provided by the present application, by obtaining a first training text library, the first training text library is a training text library of unsupervised information, each first training text in the first training text library is passed into the basic sensitive information extraction algorithm, and the basic character sequence set corresponding to each first training text is obtained, the basic character sequence set corresponding to the first training text includes the basic character sequence obtained by each sensitive information extraction operator in the basic sensitive information extraction algorithm and the sensitive information classification corresponding to the basic character sequence, for the same basic character sequence set, combined with the character sequence similarity score between the basic character sequences belonging to the same sensitive information classification, the basic character sequence obtained by executing the target sensitive information extraction operator is compared with the basic character sequence obtained by executing each other sensitive information extraction operator, and the similar character sequence corresponding to the basic character sequence obtained by executing the target sensitive information extraction operator is obtained, the basic character sequence and the corresponding similar character sequences are merged to obtain a false character sequence, and the false character sequence set corresponding to the first training text is obtained by combining the false character sequences corresponding to the same first training text, and the basic sensitive information extraction algorithm is iteratively updated by combining the first training text library and the corresponding false character sequence sets to obtain the target sensitive information extraction algorithm. In this way, the training text without supervision information is passed into the algorithm for sensitive information extraction, and the training text without supervision information is assigned a pseudo label in combination with the extracted basic character sequence set. The training text without supervision information and the pseudo label are determined as the training samples of the algorithm, and the algorithm is trained without sample annotation, saving costs. In the generation of pseudo character sequences, the basic character sequences obtained by executing different sensitive information extraction operators in the algorithm on the same training text are compared. Similar basic character sequences belong to the same sensitive information classification. The similar character sequences corresponding to the same basic character sequence are merged, and the pseudo character sequence obtained is more accurate, which makes the algorithm iteration update more accurate, thereby increasing the efficiency and accuracy of sensitive information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A schematic diagram of the implementation flow of a sensitive information extraction method based on natural language processing provided in an embodiment of the present application.

[0008] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0009] Figure 1 A schematic diagram of an implementation flow of a sensitive information extraction method based on natural language processing provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0010] Step S1: Obtain the text to be extracted, and pass the text to be extracted into the target sensitive information extraction algorithm to obtain a set of sensitive character sequences corresponding to the text to be extracted.

[0011] In actual applications, a computer system obtains a piece of text to be extracted. This text may come from various channels, such as social media posts, emails, online chat records, user comments, or internal corporate documents. Taking emails as an example, suppose a financial institution's computer system monitors the content of communications between its customers to ensure that no sensitive information (such as personal identity information, bank account details, etc.) is improperly leaked. Once the computer system obtains an email to be checked from an email server or user device as the text to be extracted, the next task is to pass the text content of this email into a pre-trained target sensitive information extraction algorithm. This algorithm is obtained through a large amount of unsupervised learning and iterative optimization, and can automatically identify and extract sensitive information from the text. The target sensitive information extraction algorithm can be a complex neural network model that encodes and decodes the input text through a multi-layer network structure (such as a convolutional neural network CNN, a recurrent neural network RNN ​​or its variant LSTM) to understand the semantic information in the text. During the training process, the algorithm has learned to recognize various sensitive information patterns and associate these patterns with specific labels (such as "personal information", "financial information", etc.).

[0012] When the text content of the email is passed into the algorithm, the algorithm analyzes the text word by word or sentence by sentence, using the knowledge it has learned during the training process to identify fragments that may contain sensitive information. This process may involve the collaborative work of multiple sensitive information extraction operators, each of which is a network architecture in the sensitive information extraction algorithm that is specifically responsible for identifying a specific type of sensitive information. For example, one operator may be specifically used to identify email addresses and phone numbers in emails, while another operator focuses on detecting ID numbers or credit card numbers. These operators run in parallel within the algorithm, each generating a list of character sequences containing potential sensitive information. Subsequently, these lists are merged into a comprehensive set of sensitive character sequences, which lists in detail all the information fragments and their types in the email that are considered sensitive by the algorithm. Finally, the computer system not only obtains the original text to be extracted (i.e., the email content) through step S1, but also obtains a structured set of sensitive character sequences through the target sensitive information extraction algorithm. This set provides key data support for subsequent sensitive information processing steps (such as warning notifications, automatic deletion, or encrypted storage).

[0013] This application provides a training process for a target sensitive information extraction algorithm. Specifically, the target sensitive information extraction algorithm is trained using the following operations:

[0014] Step S100: Acquire a first training text library, where the first training text library is a training text library with unsupervised information.

[0015] In step S100 of the embodiment of the present application, the computer system obtains a training text library of unsupervised information, namely, the first training text library. This text library does not contain any pre-annotated sensitive information, that is, the system does not rely on manual annotation of sensitive information in the text during the learning phase, but discovers and learns these sensitive information through the algorithm's own mechanism. For example, assuming a scenario in the financial industry, a financial institution builds a system to automatically detect and analyze sensitive information that may be contained in customer communications, such as bank account numbers, credit card information, personal information, etc., to prevent data leakage and improper use. In this context, the construction of the first training text library is particularly important.

[0016] Computer systems can collect a large number of public data sources from the Internet, such as news articles, social media posts, blog posts, forum discussions, etc. These texts usually contain rich natural language content, but do not necessarily directly contain sensitive information that financial institutions are concerned about. In addition to public data sources, financial institutions can also use their internal communication data, such as emails between customers, chat records, internal documents, etc. Although the amount of this data may be relatively small, it is closer to actual business scenarios and may contain more context related to sensitive information. After obtaining the raw text data, the computer system usually performs a series of data cleaning and preprocessing steps to ensure the quality and consistency of the data. For example, it includes:

[0017] Noise removal: Remove non-text content such as HTML tags, special characters, advertising links, etc. from the text.

[0018] Word segmentation: Split the text into words or phrases so that subsequent algorithms can process them more efficiently. This step is especially important when processing Chinese text because Chinese does not have obvious spaces to separate words like English.

[0019] Normalization: Convert all texts into a unified format, such as unified uppercase and lowercase letters, number formats, etc.

[0020] The text data processed through the above steps is integrated into an unsupervised information training text library, namely the first training text library. Each record in this text library is an independent text sample, such as an email, a chat message, or a document paragraph. These text samples together constitute the "teaching materials" for algorithm learning. Although they themselves do not contain labels for sensitive information, the algorithm will identify sensitive information by learning the patterns and features in these texts. For example, the first training text library may contain the following types of text samples:

[0021] "Please confirm your bank account information as follows: Account number 123456789, account opening bank XX Bank."

[0022] "I just placed an order for a new mobile phone on the XX website. The order number is ABCD1234."

[0023] “Please respond as soon as possible with a detailed report on the project’s progress.”

[0024] Although these text samples are not clearly marked as sensitive information, the algorithm can gradually learn how to identify sensitive information such as account numbers and order numbers by analyzing the keywords, context and patterns in the text. In this way, the computer system obtains a training text library of unsupervised information, laying a data foundation for the training of the target sensitive information extraction algorithm in the subsequent steps.

[0025] Step S200: Pass each first training text in the first training text library into the basic sensitive information extraction algorithm to obtain a set of basic character sequences corresponding to each of the first training texts; the set of basic character sequences corresponding to the first training texts includes the basic character sequences obtained by respectively executing each sensitive information extraction operator in the basic sensitive information extraction algorithm and the sensitive information classification corresponding to the basic character sequences.

[0026] In an embodiment of the present application, a computer system accesses a first training text library, which contains a large amount of text data without manually annotated sensitive information. For each first training text in the library, the computer system passes it to the basic sensitive information extraction algorithm one by one. The "basic" here means that the algorithm is an initial version and has not been fully optimized and trained, but it already has basic sensitive information recognition capabilities, which is usually achieved through a pre-designed model architecture (such as a neural network).

[0027] The basic sensitive information extraction algorithm contains multiple sensitive information extraction operators, which can be understood as different parts or components of the algorithm, and each operator is responsible for identifying a specific type of sensitive information. For example, one operator may be dedicated to identifying email addresses, while another operator focuses on detecting phone numbers. When the first training text enters the algorithm, each sensitive information extraction operator will independently analyze the text and try to find character sequences that may contain sensitive information. These operators may make judgments based on multiple features in the text, such as vocabulary, grammatical structure, contextual relationships, etc. For example, an operator that identifies bank account numbers can look for continuous sequences of numbers that conform to the account format used by a specific bank.

[0028] For each first training text, the basic sensitive information extraction algorithm will output a basic character sequence set. This set includes all character sequences that the algorithm considers to be sensitive information, as well as the sensitive information classification corresponding to each sequence. A character sequence is a continuous section of the text that is marked by the algorithm as containing sensitive content, such as "123456789" may be considered a bank account number.

[0029] Suppose there is a first training text as follows: "Please remit the money to account 123456789 as soon as possible and attach your ID number 123456789012345678." When this text is passed into the basic sensitive information extraction algorithm, the different sensitive information extraction operators in the algorithm will work separately: the operator that recognizes the bank account number will output the character sequence "123456789" and classify it as "financial information."

[0030] The operator that recognizes the ID card number will output the character sequence "123456789012345678" and classify it as "personal information". Therefore, for this text, the basic sensitive information extraction algorithm will generate a basic character sequence set that contains the above two character sequences and their corresponding classifications.

[0031] Through step S200, the computer system can use the basic sensitive information extraction algorithm to process each text in the first training text library to generate corresponding basic character sequence sets. These sets not only contain character sequences that the algorithm considers to be sensitive information, but also provide classification information of these sequences. This information will be used as input for subsequent training steps to further optimize the algorithm and improve its accuracy in identifying sensitive information.

[0032] Step S300: For the same set of basic character sequences, combined with the character sequence similarity scores between the basic character sequences belonging to the same sensitive information classification, the basic character sequence obtained by executing the target sensitive information extraction operator is compared with the basic character sequences obtained by executing other sensitive information extraction operators to obtain similar character sequences corresponding to the basic character sequence obtained by executing the target sensitive information extraction operator.

[0033] In step S200, the computer system has generated a basic character sequence set for each text in the first training text library. Each set contains basic character sequences and their classifications respectively identified by multiple sensitive information extraction operators. In step S300, the computer system will further process the basic character sequence set of each text. For the basic character sequence set of the same text, the computer system identifies those basic character sequences that belong to the same sensitive information classification. For example, if multiple operators have identified a bank account number, then these recognition results belong to the same sensitive information classification. Next, the computer system calculates the similarity scores between these basic character sequences belonging to the same classification. The calculation of the similarity score can be based on a variety of factors, such as the edit distance of the character sequence, the longest common subsequence length, Jaccard similarity, etc. Which method is used depends on the choice of the algorithm designer and the needs of the specific application scenario.

[0034] After obtaining the similarity score, the computer system compares the basic character sequence generated by the target sensitive information extraction operator (the operator currently being optimized or evaluated) with the basic character sequences generated by other operators in the set. If the output of some other operator exceeds a preset threshold in the similarity score with the output of the target operator, the two character sequences are considered similar. Through this process, the computer system can find character sequences generated by other operators that are similar to the output of the target operator, and these sequences are considered similar character sequences. These similar character sequences may provide a supplement or verification to the output of the target operator, which helps to improve the overall performance of the algorithm.

[0035] For example, suppose there are two sensitive information extraction operators: operator A and operator B. For a certain text, operator A recognizes a bank account number "123456789", while operator B recognizes "123-456-789". If the similarity score of these two character sequences is high (for example, based on the calculation of the edit distance or the longest common subsequence length), then they can be considered similar.

[0036] In step S300, if the target sensitive information extraction operator is operator A, the computer system will regard the character sequence recognized by operator B as a similar character sequence corresponding to the output of operator A. In this way, even if operators A and B are different in format (such as whether they contain separators), they can still be regarded by the algorithm as identifying the same sensitive information.

[0037] Through step S300, the computer system can utilize the complementarity between different sensitive information extraction operators and improve the algorithm's ability to identify sensitive information through similarity scoring and comparison fusion mechanisms. This process not only enhances the robustness of the algorithm, but also helps to discover potential recognition errors or omissions, providing valuable feedback for subsequent training and optimization.

[0038] Step S400: merging the basic character sequence and the corresponding similar character sequences to obtain a pseudo character sequence, and combining the pseudo character sequences corresponding to the same first training text to obtain a pseudo character sequence set corresponding to the first training text.

[0039] In the embodiment of the present application, step S400 is a key step, which involves effectively merging the basic character sequence with the similar character sequence to generate a pseudo character sequence as a pseudo label. These pseudo labels are crucial for the training and optimization of subsequent algorithms because they provide supervisory information for unsupervised training texts.

[0040] In step S300, the computer system has found basic character sequences similar to the output of the target sensitive information extraction operator. In step S400, these similar character sequences are merged with the basic character sequence of the target operator to generate a more accurate and reliable false character sequence.

[0041] There are many ways to merge. For example, if the character sequence in the text can be located by the character sequence number (such as the starting position and the ending position), then the mean of the sequence numbers of all similar character sequences can be calculated as the sequence number of the pseudo character sequence. For example, the sequence number of the basic character sequence is (10, 20), and the sequence numbers of the similar character sequences are (11, 19) and (9, 21), then the sequence number of the pseudo character sequence can be the mean of these three coordinates, that is, ((10+11+9) / 3, (20+19+21) / 3). Another way is to perform a weighted sum based on the support of the character sequence (that is, the number of times the sequence is simultaneously identified as sensitive information by multiple operators). The character sequence with higher support has a greater weight in the merging process. Assuming that the support of the basic character sequence is 3, and the support of the two similar character sequences is 2 and 1 respectively, the coordinates of the pseudo character sequence can be the weighted sum of the coordinates, where the weight is determined by the support, and the sum of the weights is 1. This method can better reflect the reliability of different character sequences.

[0042] For example, suppose that in a first training text, the target sensitive information extraction operator identifies a bank account number "123456789" with coordinates (50, 60). At the same time, other sensitive information extraction operators identify two similar character sequences: "123-456-789" (coordinates (51, 62)) and "12345678" (coordinates (49, 58)), and their support degrees are 2 and 1 respectively.

[0043] Through step S400, the computer system can effectively merge the basic character sequence and the similar character sequence into a pseudo character sequence as a pseudo label for subsequent training. This method not only improves the accuracy of the pseudo label, but also provides more supervision information for the training of the algorithm, which helps to improve the algorithm's ability to identify sensitive information. In practical applications, the appropriate merging method can be selected according to specific scenarios and needs.

[0044] Step S500: combining the first training text library and the corresponding sets of pseudo character sequences, iteratively updating the basic sensitive information extraction algorithm to obtain a target sensitive information extraction algorithm.

[0045] In step S400, the computer system has generated a corresponding set of pseudo character sequences for each text in the first training text library. These pseudo character sequences serve as pseudo labels and provide supervisory information for unsupervised text data. In step S500, the computer system will use this information to guide the iterative update of the basic sensitive information extraction algorithm. The iterative update process involves the following key steps:

[0046] Forward propagation: For each text in the first training text library, the computer system inputs it into the current version of the basic sensitive information extraction algorithm, and the algorithm outputs a predicted basic character sequence set. These prediction results are compared with the pseudo character sequence set generated in step S400.

[0047] Loss calculation: The computer system calculates the difference between the predicted result and the set of false character sequences, i.e., the loss. The choice of loss function depends on the specific implementation of the algorithm, but it is usually aimed at minimizing the prediction error. For example, the cross entropy loss function can be used to measure classification error, or the mean square error loss function can be used to measure the prediction bias of regression problems.

[0048] Backpropagation: Based on the loss value, the computer system adjusts the parameters in the algorithm (such as weights and biases in a neural network) through the backpropagation algorithm. Backpropagation uses the chain rule to calculate the gradient of the loss function with respect to each parameter and updates the parameters according to the gradient descent method (or its variants, such as the Adam optimizer) to reduce the loss.

[0049] Parameter update: At each iteration, the computer system updates the parameters in the algorithm based on the gradients calculated by backpropagation. The magnitude of the update is controlled by the learning rate, which is a hyperparameter set before training.

[0050] Iteration and Convergence: The above process (forward propagation, loss calculation, backpropagation, and parameter update) is repeated for multiple iterations until the stopping condition is met (such as the loss value is lower than a certain threshold, the performance on the validation set no longer improves, or the preset number of iterations is reached).

[0051] For example, suppose that in an application scenario of financial text analysis, a computer system is training a sensitive information extraction algorithm to identify bank account numbers and credit card numbers in text. In step S500, the system uses a first training text library containing a large amount of financial-related text, and a set of pseudo character sequences corresponding to these texts (considered to be a pseudo label). In each iteration, the system inputs the text in the text library into the current version of the algorithm, and the algorithm attempts to identify sensitive information (such as bank account numbers and credit card numbers) in the text. Then, the system calculates the difference between the basic character sequence predicted by the algorithm and the pseudo character sequence, that is, the loss value. Based on this loss value, the system adjusts the parameters in the algorithm through the back propagation algorithm to reduce the prediction error. As the iteration proceeds, the algorithm gradually learns how to extract sensitive information from the text more accurately. When the loss value is reduced to a certain level or the performance of the algorithm on the validation set is no longer significantly improved, the iterative process stops, and the algorithm obtained at this time is the optimized target sensitive information extraction algorithm.

[0052] Through the iterative update process of step S500, the computer system can gradually optimize the performance of the basic sensitive information extraction algorithm by using the unsupervised first training text library and the generated set of false character sequences. This process not only improves the accuracy of the algorithm in identifying sensitive information, but also enhances the generalization ability of the algorithm, so that it can also perform well on text data that has not been seen before.

[0053] In one implementation, the method further includes a pre-debugging process of the sensitive information extraction algorithm, specifically including:

[0054] Step PS1: Acquire a second training text library, wherein the second training text library is a training text library with supervised information.

[0055] In the pre-training process of the sensitive information extraction algorithm, step PS1 is a key step at the beginning, which involves obtaining a specific training resource - the second training text library. This text library is different from ordinary unsupervised training data. It contains supervised information, that is, the sensitive information in each text sample has been accurately labeled manually. Such a data set is crucial to the learning process of the algorithm because it provides a clear guiding direction for the algorithm to help the algorithm understand what is the "correct" sensitive information extraction result.

[0056] Specifically, in step PS1, the computer system retrieves and loads the second training text library from a storage medium (such as a hard drive, a cloud storage service, etc.). Each text sample in this library is a natural language text that may come from various practical application scenarios, such as financial transaction records, social media posts, email communications, etc. Importantly, the sensitive information in these texts (such as personal identity information, financial account numbers, key terms in privacy policies, etc.) has been carefully annotated by professionals, including the specific location of the sensitive information (such as the start and end position), type (such as name, address, phone number, etc.) and possible additional information (such as the confidence or importance level of the sensitive information).

[0057] Taking the application in the financial field as an example, the second training text library may contain a series of loan application forms submitted by customers. In these application forms, the customer's name, ID number, bank account information, etc. have been marked as sensitive information. The marking process may involve the use of specific text annotation tools that allow annotators to highlight sensitive information directly in the text and assign corresponding types and attributes to it.

[0058] After obtaining the second training text library, the computer system will use this labeled text data to train and adjust the sensitive information extraction algorithm. By comparing the differences between the algorithm prediction results and the manual labeling results, the system can evaluate the performance of the algorithm and optimize it accordingly. This supervised learning method can significantly improve the accuracy and efficiency of the algorithm in extracting sensitive information, laying a solid foundation for subsequent unsupervised learning or practical applications.

[0059] Step PS2: Pass each second training text in the second training text library into the sensitive information extraction algorithm to be trained to obtain a set of forward propagation character sequences corresponding to each second training text; the forward propagation character sequence set includes a set of sub-forward propagation character sequences obtained by respectively executing each sensitive information extraction operator in the sensitive information extraction algorithm to be trained.

[0060] In the pre-debugging process of the sensitive information extraction algorithm, step PS2 is one of the core links of algorithm learning. It involves inputting text data in the second training text library with precise annotations into the sensitive information extraction algorithm to be trained, and obtaining the set of forward propagation character sequences predicted by the algorithm. This process is a key step for the algorithm to gradually optimize its performance through supervised learning. Specifically, the computer system plays the role of the execution subject in this step. It first takes out each second training text in the second training text library one by one as the input data of the algorithm. These text data may come from various practical application scenarios, such as emails, social media posts, contract documents, etc., which contain sensitive information that is accurately extracted.

[0061] Next, the computer system feeds the text data into the sensitive information extraction algorithm to be trained. This algorithm can be a complex machine learning model, such as a deep neural network (DNN), a convolutional neural network (CNN), or a recurrent neural network (RNN), which integrates multiple sensitive information extraction operators. These operators are specifically designed to identify different types of sensitive information in text, such as names, phone numbers, email addresses, bank account numbers, etc.

[0062] After receiving the text data, each sensitive information extraction operator will independently perform forward propagation calculations. Forward propagation is the process of information flowing from the input layer to the output layer in a neural network, which simulates the way the human brain processes information. In this process, the operator will perform a series of operations on the text, such as word segmentation, embedding representation, and feature extraction, and finally generate a predicted forward propagation character sequence.

[0063] For each second training text, the sensitive information extraction algorithm to be trained will output a forward propagation character sequence set, which contains the sensitive information character sequences predicted by all operators. Each operator will generate a sub-forward propagation character sequence set, which is a subset of the forward propagation character sequence set and consists exclusively of the results predicted by the operator.

[0064] Each sub-forward propagation character sequence set contains three key elements: the forward propagation character sequence itself, the sensitive information category corresponding to the sequence, and the support (such as confidence or probability). Taking the recognition of phone numbers as an example, a sub-forward propagation character sequence can be "13812345678", and its corresponding sensitive information category is "phone number", while the support can be a value between 0 and 1, indicating the algorithm's confidence in the prediction result.

[0065] Through this step, the computer system provides a large number of supervised learning samples for the sensitive information extraction algorithm to be trained, so that the algorithm can continuously optimize its own parameters by comparing the differences between the predicted results and the actual annotations, thereby improving the accuracy and efficiency of extracting sensitive information. This process is the key to improving the performance of the algorithm and lays a solid foundation for subsequent unsupervised learning and practical applications.

[0066] Step PS3: combining the differences between each prior character sequence set corresponding to the same second training text and each corresponding sub-forward propagation character sequence set, obtaining the sub-errors corresponding to the second training text on each sensitive information extraction operator, wherein the sub-errors include character sequence error, category error and support error;

[0067] In the pre-debugging process of the sensitive information extraction algorithm, step PS3 compares the real annotation information (i.e., the prior character sequence) of each second training text with the set of forward propagation character sequences predicted by the algorithm in detail, so as to calculate the specific errors generated in each sensitive information extraction operator. This process not only helps developers understand the current performance of the algorithm, but also provides a key basis for subsequent parameter optimization.

[0068] Specifically, for each second training text, the computer system accesses its corresponding set of prior character sequences. These prior character sequences are manually annotated sensitive information, including the exact location and type of sensitive content (such as name, phone number, etc.) and possible support information (such as confidence score). The system then compares these prior character sequences with each sub-forward propagation character sequence set predicted by the algorithm. During the comparison process, the system calculates three types of sub-errors: character sequence error, category error, and support error.

[0069] Character sequence error is the mismatch between the predicted sensitive information character sequence and the actual annotated character sequence in terms of position or content in the text. This error can be measured by calculating the edit distance (such as the Levenshtein distance). When the predicted sensitive information type does not match the actual annotation, a category error will occur. For example, if the actual annotation is "ID number" and the algorithm predicts "telephone number", this constitutes a category error. The calculation of category error is relatively straightforward and is usually done by comparing the similarities and differences between the predicted category and the actual category. For each predicted sensitive information, the algorithm will provide a support (or confidence) score, indicating the confidence of the algorithm in the prediction result. Support error measures the difference between the support predicted by the algorithm and the actual situation (or preset standard). For example, if the algorithm's prediction result with high confidence is actually wrong, a large support error will be generated. The calculation of support error may involve a direct comparison of support scores, or converting the support into probability and then using a loss function such as cross entropy to calculate.

[0070] Taking financial text analysis as an example, suppose there is a second training text containing a sensitive information "123456789" labeled as "bank account number". In step PS3, the computer system will compare this labeled information with the result predicted by the algorithm. If the sub-forward propagation character sequence set predicted by the algorithm contains a character sequence with a similar position but slightly different content (such as "12345678"), then a character sequence error will occur; if the algorithm mistakenly classifies the sequence as "credit card number", a category error will occur; if the support score given by the algorithm is far below the preset threshold or is unreasonable compared with other high-confidence but wrong prediction results, then a support error will occur.

[0071] Through the calculation and analysis of these errors, the computer system can fully understand the performance of the algorithm on each sensitive information extraction operator, and adjust the algorithm parameters accordingly to optimize its prediction accuracy.

[0072] Step PS4: combining the sub-errors corresponding to the second training texts on the same sensitive information extraction operator to obtain the operator errors corresponding to the sensitive information extraction operators;

[0073] Step PS4 involves synthesizing the sub-errors generated by each second training text calculated in step PS3 on the same sensitive information extraction operator to evaluate the overall performance of each operator. This step is crucial for the subsequent optimization of algorithm parameters and improvement of algorithm accuracy. When executing step PS4, the computer system collects the sub-errors calculated in step PS3 for each combination of second training text and sensitive information extraction operator. These sub-errors include character sequence errors, category errors, and support errors, which respectively reflect the position accuracy, classification correctness, and confidence rationality of the algorithm when predicting sensitive information. Then, the computer system summarizes all sub-errors generated by each sensitive information extraction operator. This processing may include calculating the average, median, or other statistics of each type of sub-error to obtain an indicator that can represent the overall performance of the operator. For example, for character sequence errors, the computer system can calculate the average edit distance of all sub-errors; for category errors, the proportion of samples with classification errors can be counted; for support errors, the average difference between the predicted support and the true support can be calculated.

[0074] Through this process, the computer system can generate an operator error for each sensitive information extraction operator, which comprehensively reflects the multi-faceted performance of the operator in predicting sensitive information. These operator errors not only provide intuitive performance evaluation results for algorithm developers, but also point out the direction for subsequent parameter optimization.

[0075] Taking financial text analysis as an example, assume that the sensitive information extraction algorithm contains an operator specifically used to identify "bank account number". In step PS4, the computer will summarize the sub-errors generated by all second training texts on this operator. If most of the sub-errors are concentrated on the character sequence error, and are mainly manifested as a small number of character differences between the predicted bank account number and the real number, then it can be inferred that the operator needs to be improved in the precise matching of character sequences. On the contrary, if the category error and support error are small, it means that the performance of the operator in classification and confidence assessment is relatively good.

[0076] Step PS5: Optimize the operator composition parameters of the corresponding sensitive information extraction operator in the sensitive information extraction algorithm to be trained in combination with the operator error until a first stable state is reached to obtain a basic sensitive information extraction algorithm.

[0077] At this stage, the computer system will use the operator error calculated in step PS4 as feedback to adjust and optimize the component parameters of each sensitive information extraction operator in the sensitive information extraction algorithm to be trained. This process aims to enable the algorithm to more accurately identify sensitive information in the text by iteratively updating the parameters. Specifically, the computer system analyzes the operator error and determines which sensitive information extraction operators need to be improved. For each optimized operator, the computer will design a set of parameter adjustment strategies. These strategies may include changing the weights and biases of the neural network layer, adjusting the learning rate, introducing regularization terms to reduce overfitting, etc. The specific method of parameter adjustment depends on the specific implementation of the algorithm and the machine learning framework used.

[0078] Taking neural networks as an example, if the operator error of a sensitive information extraction operator is high, the computer system can use the back propagation algorithm to optimize the component parameters of the operator. The back propagation algorithm calculates the gradient of the loss function with respect to the network parameters and uses these gradients to update the parameter values, thereby reducing the loss function value. In this process, the learning rate is an important hyperparameter that controls the step size of the parameter update. A learning rate that is too large will cause the algorithm to diverge, while a learning rate that is too small will make the algorithm converge too slowly. The optimization process is usually iterative. In each iteration, the computer system inputs the text data in the second training text library into the sensitive information extraction algorithm to be trained, calculates the difference between the predicted result and the true annotation (i.e., the loss), and then uses the back propagation algorithm to calculate the gradient and update the parameters. This process is repeated until the algorithm reaches a certain stable state, such as the parameter change becomes very small, the gradient is close to zero, or the number of iterations reaches a preset threshold.

[0079] In one implementation, step S300, for the same set of basic character sequences, combines the character sequence similarity scores between the basic character sequences belonging to the same sensitive information classification, compares the basic character sequences obtained by executing the target sensitive information extraction operator with the basic character sequences obtained by executing each other sensitive information extraction operator, and obtains similar character sequences corresponding to the basic character sequences obtained by executing the target sensitive information extraction operator, including:

[0080] Step S310: for the current basic character sequence set, a first character sequence is determined in the basic character sequences obtained by executing the target sensitive information extraction operator, and a second character sequence is determined in the basic character sequences obtained by executing other sensitive information extraction operators; the first character sequence and the second character sequence correspond to the same sensitive information category;

[0081] Step S320: combining the character sequence overlap between the first character sequence and the second character sequence to obtain a character sequence similarity score between the first character sequence and the second character sequence;

[0082] Step S330: obtaining a second character sequence corresponding to a character sequence similarity score greater than a preset similarity score as a candidate character sequence, and obtaining a candidate character sequence corresponding to a maximum character sequence similarity score as a similar character sequence corresponding to the first character sequence.

[0083] In step S310, the goal of the computer system is to match the basic character sequence generated by the target sensitive information extraction operator (i.e., operator A) in the currently processed basic character sequence set with the basic character sequences generated by other sensitive information extraction operators (such as operator B, operator C, etc.) to find out those character sequences that belong to the same sensitive information classification.

[0084] In step S310, it is assumed that the task of operator A is to identify the email address in the email. For a specific email text, operator A may generate a basic character sequence "example@email.com", which is classified as "email address". At the same time, other operators (such as operator B identifying phone numbers and operator C identifying bank account numbers) are also processing the same text and each generates their own basic character sequence. The first step taken by the computer system is to find out the sequence that belongs to the "email address" classification with "example@email.com" among these sequences. However, in this particular step, the main focus is on the comparison between the output of operator A and the output of other operators, so it is temporarily assumed that other operators do not generate sequences belonging to the "email address" classification (in actual situations, there may be misidentification or overlap, but this is not considered here for the sake of simplicity). It is important that the computer system knows which sequences are generated by operator A and which are generated by other operators, and knows their respective classifications.

[0085] To implement step S310, the computer system may maintain a data structure to store the basic character sequences generated by each operator and their corresponding categories. This data structure may be a dictionary (or hash table), where the key is the sensitive information category and the value is a list containing all the basic character sequences belonging to the category and their source operator identifiers.

[0086] In step S320, the computer system calculates the similarity between the basic character sequence (first character sequence) generated by the target sensitive information extraction operator (operator A) and the basic character sequence (second character sequence) generated by other sensitive information extraction operators. This similarity is usually evaluated based on the overlap of the character sequences, but other factors such as semantic similarity or contextual consistency may also be considered.

[0087] When calculating the similarity score, the character sequence overlap calculation method can be used to calculate the proportion or number of identical characters in the two character sequences. For example, if the first character sequence is "example@email.com" and the second character sequence is "sample@email.com", there is a certain overlap between them (especially the domain name part), and a similarity score can be given based on the number or proportion of overlapping characters. Alternatively, the edit distance calculation is used. The edit distance (such as Levenshtein distance) is another commonly used string similarity measurement method. It calculates the minimum number of single-character edits (insertion, deletion, or substitution) required to convert one string into another. The smaller the edit distance, the more similar the two strings are. Alternatively, the Jaccard similarity calculation is used. For set-type data, the Jaccard similarity is an effective similarity measurement method. Although it may not be intuitive to apply directly to strings, the strings can be regarded as sets of characters and the ratio of the intersection and union of the two sets can be calculated to approximate the similarity. However, this method may not be the best choice in character sequence similarity calculation because it ignores the order of characters. Alternatively, for semantic-based similarity calculations, character overlap alone may not be accurate enough for some applications. For example, "123 Main St" and "123 E MainStreet" are semantically similar even though they do not have high character overlap. In this case, more sophisticated natural language processing techniques (such as word embeddings, sentence embeddings, or pre-trained language models) may be used to evaluate semantic similarity.

[0088] A computer system can calculate the similarity of two sequences using either character sequence overlap or edit distance. For example, if edit distance is used, the calculation process may involve comparing the differences of each character in the two sequences and accumulating the number of necessary edit operations. Since the two sequences have a certain similar structure before and after the "@" symbol (although the domain names are different), it can be expected that their edit distance will not be too large, giving a relatively high similarity score.

[0089] In step S330, the computer system selects the most similar sequence to the basic character sequence generated by the target sensitive information extraction operator from the basic character sequences generated by other sensitive information extraction operators as a similar character sequence based on the similarity score calculated in step S320. In order to determine which sequences are similar, the computer system sets a preset similarity score threshold. Only when the similarity score of two sequences exceeds this threshold are they considered similar. The setting of the threshold depends on the specific application scenario and the characteristics of the data set, and the optimal value may be determined through experiments. After determining the similarity score and the preset threshold, the computer system traverses the basic character sequences generated by all other sensitive information extraction operators and calculates their similarity scores with the target sequence. Then, the sequence with a score greater than the preset threshold is selected as a candidate similar character sequence. Finally, the sequence with the highest score is selected from these candidate sequences as the final similar character sequence.

[0090] Through the above three sub-steps (S310, S320, S330), the computer system can effectively filter out character sequences similar to the target operator output from the outputs of multiple sensitive information extraction operators. This process not only helps to improve the accuracy of the algorithm in identifying sensitive information, but also provides valuable pseudo-label data for subsequent unsupervised learning.

[0091] In one implementation, the step S500, combining the first training text library and the corresponding pseudo character sequence sets, iteratively updates the basic sensitive information extraction algorithm to obtain a target sensitive information extraction algorithm, specifically including:

[0092] Step S510: combining the same first training text, obtaining the target error corresponding to the selected sensitive information extraction operator according to the difference between the sub-basic character sequence set and the corresponding pseudo character sequence set obtained by executing the same selected sensitive information extraction operator, and obtaining the target error corresponding to each selected sensitive information extraction operator;

[0093] Step S520: optimizing the basic operator composition parameters corresponding to the selected sensitive information extraction operator in the basic sensitive information extraction algorithm in combination with the target error corresponding to the selected sensitive information extraction operator, and obtaining the optimized operator composition parameters corresponding to the selected sensitive information extraction operator;

[0094] Step S530: combining the optimized operator composition parameters corresponding to each of the selected sensitive information extraction operators, optimizing the basic operator composition parameters of other sensitive information extraction operators other than the selected sensitive information extraction operator in the basic sensitive information extraction algorithm, and obtaining the optimized operator composition parameters corresponding to the other sensitive information extraction operators;

[0095] Step S540: Combine the optimization operator composition parameters corresponding to each sensitive information extraction operator to obtain a target sensitive information extraction algorithm.

[0096] In step S510, the primary task of the computer system is to evaluate the performance of the selected sensitive information extraction operator (such as sensitive information extraction operator 1, 2, 3, etc.) for each text in the first training text library, and calculate its corresponding target error. The target error is an indicator that measures the difference between the operator prediction result and the actual (pseudo) label, which guides the direction of subsequent parameter optimization.

[0097] Suppose you are processing a dataset containing a large number of unlabeled social media posts, and your goal is to extract sensitive information (such as personal information, insults, etc.) from the posts. The basic sensitive information extraction algorithm consists of four sensitive information extraction operators, each of which is responsible for identifying one or a category of sensitive information. For example, operator 1 identifies personal information (such as name, address), operator 2 identifies insults, and so on.

[0098] In step S510, the computer system selects one or more sensitive information extraction operators for evaluation. Taking operator 1 as an example, it will traverse each text in the first training text library and generate a corresponding sub-basic character sequence set (i.e., the sensitive information character sequence set predicted by operator 1). At the same time, each text also corresponds to a pseudo character sequence set, which is a pseudo label set generated by step S400.

[0099] In order to calculate the target error of operator 1, the computer system compares the sub-basic character sequence set generated by operator 1 with the corresponding pseudo character sequence set. The content of the comparison may include multiple dimensions such as the position, content, and classification of the character sequence. By performing the above comparison and error calculation process on all texts in the first training text library, the computer system can obtain the average error of operator 1 on the entire data set, that is, the target error of operator 1. This process will be repeated until all selected sensitive information extraction operators have been evaluated.

[0100] In step S520, the computer system will optimize the component parameters of the selected sensitive information extraction operator according to the target error calculated in step S510. These parameters can be weights and biases in a neural network, or adjustable parameters in other machine learning models.

[0101] Taking neural networks as an example, the optimization process usually involves the back-propagation algorithm. The back-propagation algorithm calculates the gradient of the loss function with respect to the network parameters and uses these gradients to update the parameter values, thereby reducing the loss function value. In the scenario of sensitive information extraction, the loss function may contain multiple parts such as classification error, position error, and possible semantic error.

[0102] In step S530, the computer system considers the possible impact of the selected sensitive information extraction operator (such as operator 1, 2, 3, etc.) on the performance of other operators after optimization, and adjusts the composition parameters of other operators accordingly. This process is one of the key steps of collaborative optimization, which ensures the stability and consistency of the entire sensitive information extraction algorithm during the iterative update process.

[0103] Since different sensitive information extraction operators may overlap or affect each other when processing text data (for example, there may be a correlation between personal information and contact information), optimizing one operator alone can have an adverse effect on the performance of other operators. To avoid this, the computer system adopts a collaborative optimization strategy to adjust the parameters of multiple operators at the same time. The specific implementation of the collaborative optimization strategy may vary depending on the algorithm and model. A common approach is to use a multi-task learning framework to train multiple sensitive information extraction operators simultaneously. In multi-task learning, each operator is regarded as a task branch, and all branches share part of the network structure (such as embedding layer, convolution layer, etc.). By sharing parameters, different tasks can learn from each other and share useful feature information. However, directly applying the backpropagation algorithm in the multi-task learning framework may encounter the problem of gradient conflict (that is, the gradient directions of different tasks are inconsistent, resulting in overall optimization difficulties). To solve this problem, some advanced optimization techniques can be used, such as gradient clipping, dynamic weight adjustment, etc. If a multi-task learning framework is used for collaborative optimization, a special loss function may be designed to consider the performance of multiple operators at the same time. This loss function can be a weighted sum of the loss functions of a single operator or a more complex combination. By adjusting the weights of different operators, their importance in the overall optimization process can be balanced. Another method of collaborative optimization is to adopt an alternating optimization strategy. That is, first fix the parameters of other operators and only optimize the parameters of the selected operator; then switch to other operators for optimization after the parameters of the selected operator are optimized to a certain extent; and alternate in this way until all operators reach a better state. Although this method is simple, it may be inefficient and easy to fall into a local optimal solution.

[0104] After multiple iterations, the computer system integrates the optimization results of all sensitive information extraction operators to obtain the final target sensitive information extraction algorithm in step S540. This algorithm should be superior to the initial basic sensitive information extraction algorithm in performance and can more accurately extract sensitive information from text data.

[0105] During the integration process, the computer system replaces the optimized parameters of each sensitive information extraction operator into the original algorithm. At the same time, it also checks whether the overall structure of the algorithm remains unchanged and whether there are potential parameter conflicts or inconsistencies. After the integration is completed, the computer system will use the validation set (or test set) to evaluate the performance of the target sensitive information extraction algorithm. The validation set is a data set independent of the training set that is used to evaluate the generalization ability of the model during the training process and prevent overfitting. By calculating indicators such as accuracy, recall, and F1 score on the validation set, you can fully understand the performance of the algorithm.

[0106] If the performance of the algorithm on the validation set reaches the expected level, the training process can be considered to be over; otherwise, it may return to the previous iterative update step to continue optimizing the algorithm parameters until the performance meets the requirements.

[0107] In one implementation, step S510, in combination with the same first training text, obtains the target error corresponding to the selected sensitive information extraction operator according to the difference between the sub-basic character sequence set and the corresponding pseudo character sequence set obtained by executing the same selected sensitive information extraction operator, and obtains the target error corresponding to each selected sensitive information extraction operator, specifically including:

[0108] Step S511: For the same set of basic character sequences, combined with the character sequence similarity scores between the basic character sequences belonging to the same sensitive information classification, the basic character sequences obtained by executing the target selected sensitive information extraction operator are compared with the basic character sequences obtained by executing each other selected sensitive information extraction operator to obtain the similarity evaluation results corresponding to the basic character sequences obtained by executing the target selected sensitive information extraction operator.

[0109] In step S511, the core task of the computer system is to conduct an in-depth analysis of the basic character sequence set in the same first training text to evaluate the similarity of character sequences of different sensitive information extraction operators when identifying the same category of sensitive information.

[0110] For the same first training text (such as a financial document containing personal information), operators A, B, and C will execute and generate their own basic character sequence sets respectively. These sets contain the sensitive information character sequences they identify from the text. In step S511, the computer system focuses on the basic character sequence set generated by operator A. For each basic character sequence in the set (such as a name "Zhang San" identified by operator A), the system will try to find a similar character sequence in the basic character sequence set generated by other selected operators (operator B and operator C). The similarity is evaluated based on the similarity score between the character sequences, which can be calculated by a variety of algorithms, such as edit distance (Levenshtein distance), Jaccard similarity, or cosine similarity based on character embedding. Taking edit distance as an example, if the edit distance between "Zhang San" identified by operator A and "Zhang Sanfeng" misidentified by operator B (assuming it is due to format or context misunderstanding) is small, then the two character sequences are considered similar.

[0111] For each basic character sequence generated by operator A, the computer system will compare it with all basic character sequences generated by operators B and C, and calculate the similarity scores between them. In this process, the system only cares about those character sequences that belong to the same sensitive information classification (although the classification is different in this example, this step assumes that the classification difference is ignored for the sake of illustration) or that may be mistakenly identified as the same information.

[0112] After the comparison is completed, the system generates a similarity evaluation result for each base character sequence generated by operator A. This evaluation result can be a list containing all character sequences with similarity scores exceeding a preset threshold and their corresponding similarity and source operators. If no similar character sequences are found, the similarity evaluation result may be empty or contain a flag indicating that no similarity was found.

[0113] Through this step, the computer system provides important reference information for the target error of the sensitive information extraction operator selected for subsequent calculation targets, that is, which prediction results can be accurate (if there are similar results with other operators) and which can be misidentified (if there are no similar results or the similarity is very low).

[0114] Step S512: For the target first training text, a sub-basic character sequence set is obtained by executing the target selected sensitive information extraction operator, and the basic sub-errors corresponding to each basic character sequence in the sub-basic character sequence set are obtained by combining the difference between the basic character sequence in the sub-basic character sequence set and the corresponding control character sequence; wherein, if the similarity evaluation result is that there is a similar sequence, the control character sequence corresponding to the basic character sequence is a similar character sequence with a similar sequence; if the similarity evaluation result is that there is no similar sequence, the control character sequence corresponding to the basic character sequence is a corresponding false character sequence in the false character sequence set.

[0115] In step S512, the computer system traverses the sub-basic character sequence set generated by the target operator. For each basic character sequence in the set, the system determines the control character sequence based on its similarity evaluation result: if the similarity evaluation result is that there is a similar sequence, this means that a basic character sequence of the target operator is similar to a character sequence generated by other operators. In this case, the control character sequence is one of these similar character sequences (usually the one with the highest similarity is selected). The calculation of the basic sub-error will be based on the difference between the basic character sequence of the target operator and this similar character sequence. The difference may include character errors, position offsets or classification errors, etc. If the similarity evaluation result is that there is no similar sequence, this means that the basic character sequence of the target operator is not similar to the character sequence of any other operator. In this case, the control character sequence will be selected from the pseudo character sequence set. Pseudo character sequences are obtained through the pseudo-label generation process in the unsupervised learning stage, and they serve as substitutes for the "real" labels in the training data. The calculation of the basic sub-error will be based on the difference between the basic character sequence of the target operator and the corresponding pseudo-character sequence.

[0116] The error calculation method can be designed according to the specific application scenario and requirements. The following is a simplified error calculation example that combines character-level differences and position errors:

[0117] Assume that a basic character sequence of the target operator is St={c1,c2,...,cn}, and its control character sequence is Sc={c1′,c2′,...,cm′}. The basic sub-error E can be expressed as the weighted sum of character difference and position offset:

[0118] ;

[0119] Among them, 1[·] is an indicator function that returns 1 when the condition is true and 0 otherwise; α and β are weight coefficients used to balance the contribution of character differences and position shifts to the error; N is min(n,m), that is, the length of the shorter of the two sequences; |n−m| represents the absolute difference between the lengths of the two sequences, which is used to measure the degree of position shift. Through step S512, the computer system can generate a basic sub-error for each basic character sequence of the target operator, which reflects the difference between the prediction result of the target operator and the control character sequence. These basic sub-errors will serve as the basis for the subsequent calculation of the target error of the target operator on the entire training set.

[0120] Step S513: Determine the error impact corresponding to the basic character sequence in combination with the similarity evaluation result corresponding to the basic character sequence, wherein the error impact corresponding to the similar sequence is greater than the error impact corresponding to the non-similar sequence.

[0121] In step S513, the computer system assigns an error impact to each basic character sequence according to the similarity evaluation results. Specifically, if a basic character sequence is evaluated to match a similar character sequence generated by other operators, then its corresponding error impact (weight) should be set relatively high. On the contrary, if a basic character sequence does not find any similar sequence, then its corresponding error impact should be relatively low. The specific value of the error impact can be designed according to the actual situation, but generally follows the following principles: For basic character sequences with similar sequences, the error impact (weight) can be set to a value greater than 0.5, such as 0.7 or 0.8, to indicate that these prediction results are more accurate, so a higher weight should be given when calculating the total error. For basic character sequences without similar sequences, the error impact (weight) should be set to a value less than 0.5, such as 0.3 or 0.2, to indicate that these prediction results can be misidentified or noise, so a lower weight should be given when calculating the total error.

[0122] In step S514, the task of the computer system is to combine the basic sub-errors corresponding to each basic character sequence in the sub-basic character sequence set generated by the target selected sensitive information extraction operator in the target first training text according to their error influence (i.e., weight), so as to calculate the target sub-error of the text on the target selected sensitive information extraction operator. This step is a key link in evaluating the performance of the operator on a single text, and provides a basis for subsequent overall performance evaluation.

[0123] In step S514, the computer system weights and combines the corresponding basic sub-errors according to the error impact of each basic character sequence. This process can be regarded as a weighted average calculation of all basic sub-errors, where the weight is the error impact determined in step S513.

[0124] Specifically, assuming that the sub-basic character sequence set contains n basic character sequences, the basic sub-error corresponding to each sequence i is ei, and the error influence is wi, then the target sub-error E can be calculated by the following formula:

[0125] ;

[0126] Among them, the sum of all error influences is 1, which ensures the rationality of weighted merging.

[0127] In step S515, the task of the computer system is to summarize the target sub-errors of each first training text on the target selected sensitive information extraction operator calculated in step S514, so as to obtain the overall performance of the operator on the entire training set, that is, the target error corresponding to the target selected sensitive information extraction operator. This process is a key link in evaluating operator performance and guiding subsequent parameter optimization.

[0128] In the previous steps, the computer system has calculated the corresponding target sub-errors for each first training text (i.e., each social media post) in the training set according to the output of the target selected sensitive information extraction operator. These target sub-errors reflect the performance of the operator on a single text.

[0129] In step S515, the system aggregates all these target sub-errors to obtain a target error that fully reflects the performance of the operator on the entire training set. This is usually achieved by calculating the average or weighted average of all target sub-errors. Considering that the importance of each text in the training set may be different (for example, some texts may contain more types of sensitive information, or their annotation quality is higher), the weighted average may be an option. In order to simplify the calculation and maintain fairness, it is also a common practice to directly use the average value.

[0130] Through step S515, the computer system can comprehensively evaluate the performance of the target selected sensitive information extraction operator on the entire training set and obtain a quantified target error value. This value not only reflects the accuracy of the operator in identifying sensitive information, but also provides a clear guiding direction for its subsequent optimization.

[0131] In one implementation, the step S530, in combination with the optimized operator composition parameters corresponding to each of the selected sensitive information extraction operators, optimizes the basic operator composition parameters of other sensitive information extraction operators other than the selected sensitive information extraction operator in the basic sensitive information extraction algorithm to obtain the optimized operator composition parameters corresponding to the other sensitive information extraction operators, specifically including:

[0132] Step S531: obtaining the operator influence corresponding to each sensitive information extraction operator; the operator influence corresponding to the selected sensitive information extraction operator is less than the operator influence corresponding to the other sensitive information extraction operators;

[0133] Step S532: In combination with the operator influences corresponding to the respective sensitive information extraction operators, the optimized operator composition parameters corresponding to the respective selected sensitive information extraction operators and the basic operator composition parameters corresponding to the other sensitive information extraction operators are merged to obtain the optimized operator composition parameters corresponding to the other sensitive information extraction operators.

[0134] In step S531, the primary task of the computer system is to evaluate the importance or "influence" of each sensitive information extraction operator in the algorithm. This influence not only reflects the ability of the operator in identifying a specific type of sensitive information, but also reflects the contribution of the operator to the overall algorithm performance.

[0135] Among these operators, some may be particularly important in business scenarios due to the type of sensitive information they process (e.g., identification of bank account numbers is crucial for financial fraud detection), while some may process more common but less important sensitive information (e.g., identification of general personal information). Therefore, different operators are given different levels of importance during the optimization process.

[0136] The evaluation of operator influence can be based on a variety of factors, including but not limited to the following:

[0137] Business importance: The impact of an operator is determined based on the importance of the sensitive information type in the business scenario. For example, an operator that identifies a bank account number may have a higher impact than an operator that identifies general personal information.

[0138] Data distribution: Analyze the frequency and diversity of various types of sensitive information in the training data. Frequently occurring and diverse types of sensitive information may require more powerful operators to accurately identify.

[0139] Performance: In the early stages of training or in previous iterations, the performance of an operator is evaluated by indicators such as prediction accuracy, recall, and F1 score. Operators with better performance often have a higher impact.

[0140] In actual operation, the computer system can assign an influence weight to each sensitive information extraction operator based on the above factors, and the weight will be used in the subsequent parameter optimization process.

[0141] In step S532, the optimized operator composition parameters and the basic operator composition parameters are merged in combination with the operator influence. After obtaining the influence of each sensitive information extraction operator, the computer system enters step S532. The goal of this step is to merge the optimization results of the selected sensitive information extraction operator (i.e., the optimized operator composition parameters) with the basic operator composition parameters of other sensitive information extraction operators to obtain the optimized operator composition parameters corresponding to these operators.

[0142] Parameter merging is a complex process because it involves the interactions and dependencies between multiple sensitive information extraction operators. In order to effectively merge parameters, computer systems may adopt the following strategies:

[0143] 1. Weighted merging: The optimized operator components and basic operator components are weightedly merged according to the operator influence. Specifically, a weight can be assigned to each parameter (which is related to the operator influence), and then the optimized parameters and basic parameters are weighted averaged according to their respective weights. This method ensures that the importance of different operators is fully considered during the merging process.

[0144] 2. Parameter migration: In some cases, if there are significant similarities between the selected sensitive information extraction operator and other sensitive information extraction operators (for example, they identify different subcategories under the same large category of sensitive information), you can consider directly migrating some of the optimization parameters of the selected operator to other operators. This method can speed up the optimization process of other operators, but be careful to avoid negative migration (i.e. migrating inappropriate parameters to irrelevant operators, resulting in performance degradation).

[0145] 3. Collaborative optimization: Multi-task learning or joint optimization methods are used to optimize multiple sensitive information extraction operators simultaneously. In this method, all operators are optimized in the same training process, and they influence each other through shared parameters or constraints. This method can ensure that the operators in the algorithm remain consistent during the optimization process and make full use of the potential connections between them.

[0146] During the merging process, the computer system assigns weights to each operator based on its influence. For example, if an operator that identifies personal information is equally important (or even more important) in a financial fraud detection scenario, it can get a higher weight in the merging process. On the contrary, some operators that process relatively less sensitive information can get lower weights.

[0147] In this way, the computer system can gradually extend the optimization results of the selected sensitive information extraction operator to other operators in the algorithm, thereby improving the overall performance of the algorithm. This parameter merging method not only takes into account the differences between different operators, but also ensures the consistency and stability of the algorithm during the optimization process.

[0148] In another implementation, the step S500, combining the first training text library and the corresponding pseudo character sequence sets, iteratively updates the basic sensitive information extraction algorithm to obtain a target sensitive information extraction algorithm, specifically including:

[0149] Step S500A: combining the first training text library and the corresponding pseudo character sequence sets, optimizing the algorithm parameters of the basic sensitive information extraction algorithm to obtain a transition sensitive information extraction algorithm;

[0150] Step S500B: determine the transition sensitive information extraction algorithm as the basic sensitive information extraction algorithm, return to the operation of obtaining the first training text library and repeat the operation until a second stable state is reached to obtain a target sensitive information extraction algorithm.

[0151] In step S500A, the core task of the computer system is to optimize the parameters of the basic sensitive information extraction algorithm using the first training text library (training set of unsupervised information) and its corresponding pseudo character sequence set (as pseudo labels). This process essentially combines unsupervised learning with pseudo-supervised learning, and approaches the optimal solution by iteratively adjusting the algorithm parameters.

[0152] During the training process, the computer system calculates the loss function (such as cross entropy loss) based on the difference between the pseudo-label and the algorithm's prediction results, and uses the back-propagation algorithm to optimize the weights and biases of the neural network. Specifically, the system updates the network parameters based on the gradient information of the loss function to minimize the loss value. This process is repeated for multiple iterations until the loss value stabilizes or reaches the preset number of iterations.

[0153] The specific implementation of parameter optimization depends on the selected machine learning model and optimization algorithm. Taking neural networks as an example, the stochastic gradient descent algorithm can be used to update network parameters. By applying the above update rules multiple times, the computer system can gradually adjust the parameters of the neural network to better adapt it to the training data and generate a transition sensitive information extraction algorithm. The performance of this algorithm in identifying sensitive information has been significantly improved compared to the basic algorithm.

[0154] In step S500B, the computer system uses the transition sensitive information extraction algorithm as a new basic sensitive information extraction algorithm and repeats the entire training process starting from obtaining the first training text library. The purpose of this step is to further consolidate and optimize the algorithm performance through multiple iterations until the algorithm reaches a more stable state (i.e., the second stable state).

[0155] The second stable state can refer to the first stable state and can be the same or different. If different, the difference can be set at a specific value. In practical applications, in order to speed up the training process and reduce computing resource consumption, some advanced technologies can be used to optimize the iterative update process. For example:

[0156] Early Stopping: Stop training early when the performance on the validation set no longer improves.

[0157] Learning rate decay: Gradually reduce the learning rate as the number of iterations increases to stabilize the training process and promote the convergence of the algorithm.

[0158] Data enhancement: By transforming the training data (such as random cropping, rotation, flipping, etc.), the diversity of the data can be increased and the generalization ability of the algorithm can be improved.

[0159] Through the iterative update process of steps S500A and S500B, the computer system can gradually transform the basic sensitive information extraction algorithm into an efficient target sensitive information extraction algorithm. This process not only improves the accuracy of the algorithm in identifying sensitive information, but also enhances its robustness and generalization ability in practical applications.

[0160] In one implementation, the step S500A, combining the first training text library and the corresponding pseudo character sequence sets, optimizing the algorithm parameters of the basic sensitive information extraction algorithm to obtain a transition sensitive information extraction algorithm, may include:

[0161] Step S500A1: Acquire a third training text library, wherein the third training text library is a training text library with supervised information;

[0162] Step S500A2: combining the first training text library and the corresponding pseudo character sequence sets and the third training text library, optimizing the algorithm parameters of the basic sensitive information extraction algorithm to obtain a transition sensitive information extraction algorithm.

[0163] In step S500A1, the computer system obtains a training text library containing supervised information, namely, the third training text library. Different from the first training text library (a training set without supervised information), each text sample in the third training text library has been manually and accurately labeled with the specific location, type and attribute of sensitive information. This labeled information provides valuable supervisory signals for the algorithm, which helps the algorithm to adjust parameters more accurately during the optimization process.

[0164] When acquiring the third training text library, the computer system considers the following factors:

[0165] Relevance of data: Select datasets that are highly relevant to the target task to ensure the effectiveness of supervised information.

[0166] Data diversity: Ensure that the dataset contains multiple types of sensitive information (such as personal information, financial information, privacy policies, etc.) and variants to improve the generalization ability of the algorithm.

[0167] Data annotation quality: Check the annotation quality of the dataset to ensure that the annotation information is accurate to avoid introducing noisy data.

[0168] After obtaining the third training text library, the computer system performs some preprocessing operations on the data, such as removing duplicate data, correcting annotation errors, and unifying data formats to ensure the neatness and consistency of the data.

[0169] In step S500A2, the computer system combines the first training text library and its corresponding pseudo character sequence set with the third training text library to optimize the parameters of the basic sensitive information extraction algorithm. This process is essentially to introduce supervised information on the basis of unsupervised learning to further guide the optimization direction of the algorithm.

[0170] There are many strategies for combining the first training text library and the third training text library to optimize parameters. The following are some possible implementation methods:

[0171] Phased optimization: First, the algorithm is preliminarily optimized using the first training text library and the set of pseudo character sequences (as described in step S500 above) to obtain a preliminarily optimized algorithm model. Then, the model is fine-tuned using the third training text library to further improve its performance on supervised information.

[0172] Joint optimization: The first training text library (and its set of pseudo-character sequences) and the third training text library are combined into a larger training set, and the basic sensitive information extraction algorithm is jointly optimized. During the optimization process, different loss function weights or sampling strategies can be set according to the characteristics of different data sets to ensure that the algorithm can make full use of both types of data.

[0173] Multi-task learning: Decompose the sensitive information extraction task into multiple subtasks (such as identifying different types of sensitive information) and set a corresponding loss function for each subtask. Then, use the multi-task learning framework to optimize these subtasks simultaneously, where the first training text library and the set of fake character sequences are used to train some subtasks (unsupervised or weakly supervised), while the third training text library is used to train all subtasks (supervised).

[0174] Taking phased optimization as an example, the computer system can first perform preliminary optimization on the basic sensitive information extraction algorithm according to step S500 described above (i.e., all steps before step S500A). In this process, the system will use the first training text library and its pseudo character sequence set to train the algorithm model, and obtain a preliminary optimized transition sensitive information extraction algorithm through iterative updates.

[0175] Then, in the second stage of optimization, the computer system will use the third training text library to fine-tune the algorithm model after preliminary optimization. During the fine-tuning process, the system will calculate the loss function value (such as supervised cross entropy loss) of the algorithm model on the third training text library, and use the back propagation algorithm to update the model parameters. Since the third training text library contains accurate supervised information, the fine-tuning process can more directly guide the algorithm to approach the optimal solution. The loss function calculation in the fine-tuning process may involve multiple aspects, such as classification error, position error, semantic similarity error, etc. The fine-tuned algorithm model will have stronger generalization ability and higher accuracy, thus becoming the required transition sensitive information extraction algorithm. This algorithm can not only generate a high-quality set of false character sequences on unsupervised data as pseudo labels for subsequent iterative training processes; it can also be directly applied to actual scenarios to accurately extract sensitive information from texts to meet business needs.

[0176] In one implementation, the step S500A1, obtaining a third training text library, includes:

[0177] Step S500A11: Acquire a fourth training text library, wherein the fourth training text library is a training text library with unsupervised information.

[0178] Step S500A12: Pass each fourth training text in the fourth training text library into the basic sensitive information extraction algorithm to obtain a basic character sequence set corresponding to each fourth training text, wherein the basic character sequence set corresponding to the fourth training text includes a sub-basic character sequence set obtained by respectively executing each sensitive information extraction operator in the basic sensitive information extraction algorithm, and the sub-basic character sequence set includes a basic character sequence obtained by executing the same sensitive information extraction operator and a support corresponding to the basic character sequence.

[0179] In step S500A12, the core task of the computer system is to input each unsupervised training text (i.e., the fourth training text) in the fourth training text library into the basic sensitive information extraction algorithm to generate a basic character sequence set corresponding to each text. This process is the basis for subsequent sensitive information screening and pseudo-label generation, and is crucial to improving the effect of combining unsupervised learning with semi-supervised learning.

[0180] In step S500A12, the computer system will traverse each text in the fourth training text library and input it into the basic sensitive information extraction algorithm one by one. This algorithm can be a complex model that integrates multiple sensitive information extraction operators, each of which is responsible for identifying a specific type of sensitive information. For example, operator A may be specifically used to identify personal information (such as name, address), operator B focuses on detecting insults or hate speech, and operator C may be responsible for identifying illegal content. When the fourth training text is input into the algorithm, each sensitive information extraction operator will analyze and process the text independently. The operator will use natural language processing technology (such as word segmentation, part-of-speech tagging, dependency syntax analysis, etc.) to parse the text structure and identify potential sensitive information fragments based on preset rules or learned patterns. These fragments are then organized into basic character sequences and attached with the algorithm's confidence score for the sequence, i.e., support.

[0181] For each fourth training text, the algorithm generates a basic character sequence set consisting of multiple sub-basic character sequence sets. Each subset corresponds to the output of a sensitive information extraction operator, which contains all the basic character sequences identified by the operator and their support. For example, for the above text, if there are two operators (one to detect bad behavior and one to detect abusive speech), two subsets can be obtained: one contains sequences such as "super rude people", and the other may contain an assessment of the overall emotional tendency of the text (although this may not be directly expressed as a character sequence, it can be used for subsequent analysis). Step S500A12 is the key link to connect the unsupervised training text with the sensitive information extraction algorithm. Through this process, the computer system can convert a large amount of unlabeled text into a data set containing potential sensitive information fragments and their support, providing rich materials for subsequent data screening and pseudo-label generation. This process not only makes full use of the advantages of unsupervised learning, but also lays the foundation for subsequent semi-supervised learning or weakly supervised learning, which helps to improve the overall performance of the sensitive information extraction system.

[0182] Step S500A13: Perform random calculation on the support corresponding to each basic character sequence in the same sub-basic character sequence set to obtain the random calculation results corresponding to each sub-basic character sequence set, and merge the random calculation results corresponding to the same fourth training text to obtain the random merge results corresponding to each fourth training text.

[0183] In step S500A13, the task of the computer system is to further analyze the basic character sequence set generated by the basic sensitive information extraction algorithm, especially to perform random calculations on the support in each sub-basic character sequence set. This process is intended to evaluate the consistency or uncertainty of the algorithm in identifying sensitive information between different texts and different sensitive information extraction operators, and to provide a basis for subsequent screening of high-quality training samples.

[0184] Randomness calculation here refers to the inconsistency or uncertainty of the evaluation algorithm when identifying sensitive information. Since the basic sensitive information extraction algorithm may contain multiple sensitive information extraction operators, and these operators may be affected by multiple factors such as text content, format, context, etc. when identifying sensitive information, their performance on different texts may be different. The purpose of randomness calculation is to quantify this difference in order to identify texts with more consistent recognition results between operators (i.e., lower randomness), which are more likely to contain real and consistent sensitive information.

[0185] For each sub-basic character sequence set (i.e., the output set of each sensitive information extraction operator), the computer system calculates the variability of the support of each basic character sequence therein. This can be achieved by calculating statistics such as the standard deviation, variance, or entropy of the support. For example, if an operator identifies similar sensitive information fragments on multiple texts but the support fluctuates greatly, it means that the performance of the operator on these texts has a high randomness. After obtaining the randomness calculation results of each sub-basic character sequence set, the computer system combines these results into a single randomness indicator to reflect the randomness of the entire fourth training text in sensitive information recognition. This can be achieved by weighted averaging the randomness calculation results of each subset, taking the maximum value, or adopting a more complex fusion strategy. For example, if the randomness of a text on all sensitive information extraction operators is low, the overall randomness of the text is also low.

[0186] To quantify this randomness, a computer system can calculate the standard deviation or coefficient of variation of the support of each operator output set and combine these values ​​into an overall randomness indicator. For example, if the output set of operator A is less random (i.e., has a smaller standard deviation) and the output set of operator B is more random, then the overall randomness of text T may be somewhere in between the two.

[0187] Step S500A13 generates a random merging result for each text in the fourth training text library through random calculation, which reflects the uncertainty of the algorithm in identifying sensitive information of the text. This process provides an important basis for the subsequent screening of high-quality training samples, helps to reduce pseudo-label noise and improve the effect of semi-supervised learning. In practical applications, the specific implementation of random calculation may vary depending on factors such as algorithm design and data characteristics, but the core idea is the same: to evaluate the quality of the text by quantifying the inconsistency of the algorithm in identifying sensitive information.

[0188] Step S500A14: performing random value segmentation based on the randomness merging results corresponding to each fourth training text to obtain a plurality of random value ranges, and determining a selected random value range from the plurality of random value ranges.

[0189] In step S500A14, the core task of the computer system is to further process the merged results obtained after the randomness calculation of each fourth training text, so as to screen out high-quality texts suitable for subsequent training. This process is achieved by segmenting the randomness value and determining the selected randomness value range, aiming to select those texts with relatively consistent sensitive information recognition results and low randomness from a large amount of unsupervised text data.

[0190] In this step, the computer system will sort the randomness merging results of all the fourth training texts, and divide the range of randomness values ​​according to actual needs or preset rules. For example, the randomness value can be divided into several intervals from low to high, such as "very low", "low", "medium", "high" and "very high". Such a division helps to understand the randomness level of each text more intuitively and provide a basis for subsequent screening. After obtaining the randomness value range, the computer system selects one or more randomness value ranges as the source of subsequent training samples according to training requirements and text quality. Usually, texts with lower randomness are selected because the sensitive information recognition results in these texts are more consistent and are more likely to contain true and accurate sensitive information. However, in practical applications, the diversity and generalization ability of text data may also be considered. Therefore, texts containing a certain degree of randomness can sometimes be selected as training samples to increase the robustness and generalization ability of the model. This depends on the specific application scenario and training objectives.

[0191] For example, suppose there are 1000 fourth training texts, and their random merge results are obtained after random calculation. Now, sort these results from small to large and divide them into five intervals: [0, 0.1), [0.1, 0.3), [0.3,0.6), [0.6, 0.9), [0.9, 1]. After analysis, it is found that the text quality with randomness values ​​in the interval [0, 0.3) is high, and the sensitive information recognition results are more consistent; while the text quality with randomness values ​​in the interval [0.9, 1] is poor, and there may be a lot of misrecognition or noise. Based on the above analysis, the computer system selects [0, 0.3) as the selected randomness value range, and selects all texts in this range from the original text data as the sample set for subsequent training. In this way, it is possible to ensure that the text data used in the training process has high quality and consistency, thereby improving the accuracy and robustness of the model.

[0192] Step S500A14 provides an effective method for screening high-quality training samples from a large amount of unsupervised text data by segmenting the randomness value and determining the selected randomness value range. This process not only helps to reduce pseudo-label noise and improve training effects, but also can flexibly adjust the screening strategy according to specific application scenarios and training goals. In practical applications, the randomness value range and screening conditions can be reasonably set according to the characteristics of the text data and the needs of model training to achieve the best training effect.

[0193] Step S500A15: Determine a selected training text for extracting sensitive information from the fourth training text whose random merging result falls within the selected randomness value range, extract sensitive information from the selected training text, and obtain a priori character sequence set corresponding to the selected training text.

[0194] In step S500A15, the task of the computer system is to further refine the high-quality training sample set selected from the large amount of unsupervised text data. Specifically, the system will further determine which texts do contain valuable sensitive information among the fourth training texts whose random merging results fall within the selected randomness value range, and perform accurate sensitive information extraction on these texts to generate the corresponding prior character sequence set. This process is one of the key steps in constructing a supervised training set, and is crucial to the accuracy and efficiency of subsequent model training.

[0195] In step S500A15, the computer system will traverse the fourth training texts whose random merging results fall within the selected random value range. For each text, the system will perform a preliminary analysis and judgment to determine whether it actually contains valuable sensitive information. This judgment process may be based on a variety of factors, such as the topic relevance of the text content, the frequency of occurrence of sensitive words, the standardization of the text structure, etc.

[0196] For example, the system can first check whether the text contains words from a predefined list of sensitive words. If a large number of words related to financial sensitive information (such as "account", "card number", "password", etc.) appear in the text, the system can consider the text as potential selected training text. In addition, the system may also use natural language processing technology to perform a deeper analysis of the text, such as identifying possible sensitive information fragments and verifying their contextual consistency.

[0197] Once the selected training texts are determined, the computer system will perform accurate sensitive information extraction on these texts. This process may involve complex natural language processing techniques and machine learning algorithms. For example, the system can use rule-based matching methods, regular expression matching, or more advanced machine learning models (such as conditional random fields, deep learning models, etc.) to identify sensitive information fragments in the text. After extracting the sensitive information fragments, the system will organize these fragments into a set of prior character sequences. Each prior character sequence contains the precise location, content, and type information of the sensitive information. For example, a prior character sequence may be represented as a four-tuple: (starting position, ending position, sensitive information content, sensitive information type), such as (100, 113, "1234567890123456", "credit card number").

[0198] Step S500A15 provides high-quality sample data for constructing a supervised training set through further text screening and accurate sensitive information extraction. This process not only ensures the accuracy and relevance of the training samples, but also provides rich supervisory information for subsequent model training by generating a priori character sequence sets.

[0199] Step S500A16: Combine each selected training text with the corresponding prior character sequence set to obtain a third training text library.

[0200] In step S500A16, the task of the computer system is to integrate the selected training texts carefully screened and extracted in the previous steps and their corresponding prior character sequence sets to construct a third training text library. This library is a training data set containing supervised information, which is crucial for the optimization and training of the subsequent sensitive information extraction algorithm. The following is a detailed explanation of this step, which is explained in conjunction with specific application scenarios.

[0201] In step S500A16, the computer system pairs each selected training text with its corresponding prior character sequence set to form a complete training sample. Each training sample contains the original text content and the precise location and type information of sensitive information in the text.

[0202] As all selected training texts have completed pairing with the prior character sequence, the computer system aggregates these training samples to form a third training text library. Each sample in this library has complete supervisory information and can be directly used for optimization and training of subsequent sensitive information extraction algorithms. Step S500A16 successfully constructs a third training text library containing supervisory information by integrating the selected training texts and the corresponding prior character sequence set. This library provides valuable data resources for the optimization and training of subsequent sensitive information extraction algorithms. In practical applications, constructing such a training data set is one of the key steps to improve algorithm performance and ensure the accuracy and efficiency of sensitive information extraction.

[0203] In one implementation, the step S1, passing the text to be extracted into a target sensitive information extraction algorithm to obtain a set of sensitive character sequences corresponding to the text to be extracted, includes:

[0204] The text to be extracted is passed into multiple target sensitive information extraction algorithms to obtain multiple sensitive character sequence sets corresponding to the text to be extracted. Different target sensitive information extraction algorithms correspond to different sensitive information classification sets. Any target sensitive information extraction algorithm is used to determine the target sensitive information classification corresponding to the text to be extracted in the corresponding sensitive information classification set and extract it.

[0205] In step S1, the computer system does not only rely on a single target sensitive information extraction algorithm to process the text to be extracted, but adopts a more flexible and comprehensive approach: the text to be extracted is simultaneously passed into multiple target sensitive information extraction algorithms to fully utilize the advantages of different algorithms in classifying specific sensitive information.

[0206] Suppose the task is to automatically extract various types of sensitive information from a large number of user comments, including personal privacy, insults, malicious promotion, etc. Since each type of sensitive information has its own unique language characteristics and context, a single algorithm may not be able to fully cover all situations. Therefore, the computer system has designed multiple target sensitive information extraction algorithms specifically for different sensitive information classifications.

[0207] For example, Algorithm A is specifically designed to identify private personal information, such as name, address, and phone number. It has been trained on a large amount of text data containing private personal information and can accurately identify and extract this sensitive information. Algorithm B focuses on detecting insults and inappropriate content. The algorithm identifies potential sensitive information by analyzing the emotional color, insulting words, and context in the text. Algorithm C is optimized for malicious promotion and spam. It can identify features such as advertising links and promotional statements in the text, thereby helping to filter out unnecessary interference information.

[0208] When the computer system receives a text to be extracted (for example, a user comment: "This product is terrible and not worth the price at all! Don't buy it, it's a waste of money! Also, stop advertising, XX brand XX product is garbage!"), it will pass the text to algorithm A, algorithm B, and algorithm C at the same time.

[0209] Output of Algorithm A: Identify possible personal privacy information from the text (although there may be no direct private information in this example, the algorithm will try to find and mark any strings that match the private information pattern).

[0210] Algorithm B output: Abusive language and negative reviews (“terrible”, “don’t buy”, “waste of money”) identified in the text, which are marked as sensitive information.

[0211] Output of Algorithm C: Detects promotional statements and advertising hints in the text ("Stop advertising", "XX product of XX brand"), which are also considered sensitive information.

[0212] Each algorithm generates a set of sensitive character sequences, which contains all sensitive information fragments and their classifications identified by the algorithm on the text. Ultimately, the computer system will aggregate these sets to form a comprehensive sensitive information report, which will be displayed to the end user or used for subsequent processing.

[0213] This multi-algorithm parallel processing approach ensures the comprehensiveness of the extraction process and reduces the possibility of missing information by covering different sensitive information categories with multiple algorithms. Each algorithm is optimized for its specific sensitive information category, so the recognition accuracy on that category is usually higher. When new sensitive information types emerge, new algorithms can be easily added to deal with them without large-scale modifications to existing algorithms.

[0214] An embodiment of the present application provides a computer system, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0215] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

Claims

1. A sensitive information extraction method based on natural language processing, characterized in that: The method comprises: Obtaining a text to be extracted, and passing the text to be extracted into a target sensitive information extraction algorithm to obtain a set of sensitive character sequences corresponding to the text to be extracted; The target sensitive information extraction algorithm is trained using the following operations: Acquire a first training text library; the first training text library is a training text library with unsupervised information; Passing each first training text in the first training text library into the basic sensitive information extraction algorithm to obtain a set of basic character sequences corresponding to each first training text; the set of basic character sequences corresponding to the first training text includes basic character sequences obtained by respectively executing each sensitive information extraction operator in the basic sensitive information extraction algorithm and sensitive information classifications corresponding to the basic character sequences; For the same set of basic character sequences, combined with the character sequence similarity scores between basic character sequences belonging to the same sensitive information classification, the basic character sequence obtained by executing the target sensitive information extraction operator is compared with the basic character sequences obtained by executing other sensitive information extraction operators to obtain similar character sequences corresponding to the basic character sequence obtained by executing the target sensitive information extraction operator; The basic character sequence and the corresponding similar character sequences are combined to obtain a pseudo character sequence, and the pseudo character sequences corresponding to the same first training text are combined to obtain a set of pseudo character sequences corresponding to the first training text; In combination with the first training text library and the corresponding sets of pseudo character sequences, the basic sensitive information extraction algorithm is iteratively updated to obtain a target sensitive information extraction algorithm; The step of merging the basic character sequence and the corresponding similar character sequences to obtain a pseudo character sequence includes: Calculate the average of the character sequence coordinates of the basic character sequence and its corresponding similar character sequences as the character sequence coordinates of the false character sequence; Alternatively, the character sequence coordinates of the basic character sequence and its corresponding similar character sequences are weightedly summed to obtain the character sequence coordinates of the pseudo character sequence, wherein the greater the support corresponding to the character sequence, the greater the corresponding weight of the character sequence, and the sum of the corresponding weights of each character sequence is equal to 1.

2. The method according to claim 1, characterized in that The method also includes a process of pre-debugging the sensitive information extraction algorithm, including: Acquire a second training text library, wherein the second training text library is a training text library with supervised information; Passing each second training text in the second training text library into the sensitive information extraction algorithm to be trained, obtaining a forward propagation character sequence set corresponding to each second training text; the forward propagation character sequence set includes a sub-forward propagation character sequence set obtained by respectively executing each sensitive information extraction operator in the sensitive information extraction algorithm to be trained, and the sub-forward propagation character sequence set includes a forward propagation character sequence obtained by executing the same sensitive information extraction operator, and a sensitive information classification and support corresponding to the forward propagation character sequence; Combining the differences between each prior character sequence set corresponding to the same second training text and each corresponding sub-forward propagation character sequence set, obtaining the sub-errors corresponding to the second training text on each sensitive information extraction operator, wherein the sub-errors include character sequence error, category error and support error; The sub-errors corresponding to the second training texts on the same sensitive information extraction operator are combined to obtain the operator errors corresponding to the sensitive information extraction operators respectively; The operator composition parameters of the corresponding sensitive information extraction operator in the sensitive information extraction algorithm to be trained are optimized in combination with the operator error until a first stable state is reached, thereby obtaining a basic sensitive information extraction algorithm.

3. The method according to claim 1, characterized in that For the same basic character sequence set, in combination with the character sequence similarity scores between the basic character sequences belonging to the same sensitive information classification, the basic character sequence obtained by executing the target sensitive information extraction operator is compared with the basic character sequences obtained by executing each other sensitive information extraction operator to obtain a similar character sequence corresponding to the basic character sequence obtained by executing the target sensitive information extraction operator, including: For the current basic character sequence set, a first character sequence is determined in the basic character sequences obtained by executing the target sensitive information extraction operator, and a second character sequence is determined in the basic character sequences obtained by executing other sensitive information extraction operators; the first character sequence and the second character sequence correspond to the same sensitive information category; combining the character sequence overlap between the first character sequence and the second character sequence to obtain a character sequence similarity score between the first character sequence and the second character sequence; A second character sequence corresponding to a character sequence similarity score greater than a preset similarity score is obtained as a candidate character sequence, and a candidate character sequence corresponding to a maximum character sequence similarity score is obtained as a similar character sequence corresponding to the first character sequence.

4. The method according to claim 1, characterized in that: The step of iteratively updating the basic sensitive information extraction algorithm by combining the first training text library and the corresponding sets of pseudo character sequences to obtain a target sensitive information extraction algorithm includes: In combination with the same first training text, according to the difference between the sub-basic character sequence set obtained by executing the same selected sensitive information extraction operator and the corresponding pseudo character sequence set, the target error corresponding to the selected sensitive information extraction operator is obtained, and the target error corresponding to each selected sensitive information extraction operator is obtained respectively; In combination with the target error corresponding to the selected sensitive information extraction operator, the basic operator composition parameters corresponding to the selected sensitive information extraction operator in the basic sensitive information extraction algorithm are optimized to obtain the optimized operator composition parameters corresponding to the selected sensitive information extraction operator; In combination with the optimized operator composition parameters corresponding to each of the selected sensitive information extraction operators, the basic operator composition parameters of other sensitive information extraction operators other than the selected sensitive information extraction operator in the basic sensitive information extraction algorithm are optimized to obtain the optimized operator composition parameters corresponding to the other sensitive information extraction operators; The target sensitive information extraction algorithm is obtained by combining the optimization operator composition parameters corresponding to each sensitive information extraction operator.

5. The method according to claim 4, characterized in that The method combines the same first training text, obtains the target error corresponding to the selected sensitive information extraction operator according to the difference between the sub-basic character sequence set and the corresponding pseudo character sequence set obtained by executing the same selected sensitive information extraction operator, and obtains the target error corresponding to each selected sensitive information extraction operator, including: For the same set of basic character sequences, combined with the character sequence similarity scores between basic character sequences belonging to the same sensitive information classification, the basic character sequence obtained by executing the target selected sensitive information extraction operator is compared with the basic character sequences obtained by executing each other selected sensitive information extraction operator to obtain the similarity evaluation result corresponding to the basic character sequence obtained by executing the target selected sensitive information extraction operator; For the target first training text, a sub-basic character sequence set is obtained by executing the target selected sensitive information extraction operator, and the basic sub-errors corresponding to each basic character sequence in the sub-basic character sequence set are obtained by combining the difference between the basic character sequence in the sub-basic character sequence set and the corresponding control character sequence; wherein, if the similarity evaluation result is that there is a similar sequence, the control character sequence corresponding to the basic character sequence is a similar character sequence with a similar sequence, and if the similarity evaluation result is that there is no similar sequence, the control character sequence corresponding to the basic character sequence is a corresponding false character sequence in the false character sequence set; Determine the error impact degree corresponding to the basic character sequence in combination with the similarity evaluation result corresponding to the basic character sequence, wherein the error impact degree corresponding to the similar sequence is greater than the error impact degree corresponding to the non-similar sequence; Combine the error influence corresponding to the basic character sequence, merge the basic sub-errors corresponding to each basic character sequence in the sub-basic character sequence set, and obtain the target sub-error corresponding to the target first training text on the target selected sensitive information extraction operator; Combining the target sub-errors corresponding to the target selected sensitive information extraction operator of each first training text, a target error corresponding to the target selected sensitive information extraction operator is obtained; The step of combining the optimized operator composition parameters corresponding to each of the selected sensitive information extraction operators, optimizing the basic operator composition parameters of other sensitive information extraction operators other than the selected sensitive information extraction operator in the basic sensitive information extraction algorithm, and obtaining the optimized operator composition parameters corresponding to other sensitive information extraction operators includes: Obtaining the operator influence corresponding to each sensitive information extraction operator; the operator influence corresponding to the selected sensitive information extraction operator is less than the operator influence corresponding to the other sensitive information extraction operators; In combination with the operator influences corresponding to the respective sensitive information extraction operators, the optimized operator composition parameters corresponding to the respective selected sensitive information extraction operators and the basic operator composition parameters corresponding to the other sensitive information extraction operators are merged to obtain the optimized operator composition parameters corresponding to the other sensitive information extraction operators.

6. The method according to any one of claims 1 to 5, characterized in that The step of iteratively updating the basic sensitive information extraction algorithm by combining the first training text library and the corresponding sets of pseudo character sequences to obtain a target sensitive information extraction algorithm includes: In combination with the first training text library and the corresponding sets of pseudo character sequences, the algorithm parameters of the basic sensitive information extraction algorithm are optimized to obtain a transition sensitive information extraction algorithm; The transition sensitive information extraction algorithm is determined as the basic sensitive information extraction algorithm, and the operation of obtaining the first training text library is returned to and repeatedly executed until a second stable state is reached to obtain a target sensitive information extraction algorithm.

7. The method according to claim 6, characterized in that The step of optimizing the algorithm parameters of the basic sensitive information extraction algorithm by combining the first training text library and the corresponding sets of pseudo character sequences to obtain a transition sensitive information extraction algorithm includes: Acquire a third training text library, wherein the third training text library is a training text library with supervised information; In combination with the first training text library and the corresponding sets of pseudo character sequences and the third training text library, the algorithm parameters of the basic sensitive information extraction algorithm are optimized to obtain a transition sensitive information extraction algorithm.

8. The method according to claim 7, characterized in that The step of obtaining a third training text library comprises: Acquire a fourth training text library, wherein the fourth training text library is a training text library of unsupervised information; Each fourth training text in the fourth training text library is passed into the basic sensitive information extraction algorithm to obtain a basic character sequence set corresponding to each fourth training text, wherein the basic character sequence set corresponding to the fourth training text includes a sub-basic character sequence set obtained by respectively executing each sensitive information extraction operator in the basic sensitive information extraction algorithm, and the sub-basic character sequence set includes a basic character sequence obtained by executing the same sensitive information extraction operator and a support corresponding to the basic character sequence; Performing random calculation on the support corresponding to each basic character sequence in the same sub-basic character sequence set to obtain random calculation results corresponding to each sub-basic character sequence set, and merging the random calculation results corresponding to the same fourth training text to obtain random merging results corresponding to each fourth training text; Perform random value segmentation based on the randomness merging results corresponding to each fourth training text to obtain multiple random value ranges, and determine a selected random value range from the multiple random value ranges; Determine a selected training text for extracting sensitive information from the fourth training text whose random merging result falls within the selected randomness value range, extract sensitive information from the selected training text, and obtain a priori character sequence set corresponding to the selected training text; The third training text library is obtained by combining each selected training text with the corresponding prior character sequence set.

9. The method according to claim 1, characterized in that: The step of passing the text to be extracted into the target sensitive information extraction algorithm to obtain a set of sensitive character sequences corresponding to the text to be extracted includes: The text to be extracted is passed into multiple target sensitive information extraction algorithms to obtain multiple sensitive character sequence sets corresponding to the text to be extracted. Different target sensitive information extraction algorithms correspond to different sensitive information classification sets. Any target sensitive information extraction algorithm is used to determine the target sensitive information classification corresponding to the text to be extracted in the corresponding sensitive information classification set and extract it.

10. A computer system comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 9 are implemented.

Images