A sensitive information processing method, program product, device and storage medium

Abstract

This invention discloses a sensitive information processing method, program product, device, and storage medium, comprising: acquiring a code file to be processed and determining the scannable text content corresponding to the code file; performing line-by-line code scanning on the scannable text content according to preset sensitive word regularization rules to determine the sensitive content of the code file; performing semantic classification on sensitive variable names based on a pre-determined semantic classifier to obtain the target sensitive category and category semantic similarity of the sensitive variable names; determining the sensitive information entropy of the sensitive variable values, and determining the confidence level of the sensitive content based on the category semantic similarity, sensitive information entropy, and contextual information; performing desensitization processing on the sensitive content based on the confidence level, and generating a detection report based on the desensitized sensitive content and the confidence level of the sensitive content. This invention can accurately identify and process raw data without keywords, achieving high-precision and low-false-positive sensitive content detection.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent data processing technology, and in particular to a sensitive information processing method, program product, device and storage medium. Background Technology

[0002] Currently, in software development, code hosting, and open-source collaboration scenarios, source code files often contain various sensitive information, such as passwords, keys, phone numbers, card numbers, and email addresses. If such information is uploaded or made public without proper detection and anonymization, it can easily lead to security risks such as data leakage, account theft, and system intrusion.

[0003] Currently, common methods for detecting sensitive information are based on string matching using keywords or variable names, such as identifying variables containing names like "password," "secret," and "token." The results are then used to determine the sensitive content and its category within the code. However, in practice, keywords and variable names in code are not always standard naming conventions. For example, a key name might not be "password" but rather "pwd" or "pass." This method cannot identify sensitive content with semantic variations or ambiguities. Furthermore, it relies too heavily on keyword triggering mechanisms, becoming completely ineffective when a sensitive value appears alone without a corresponding variable name, resulting in high false positive and false negative rates. Summary of the Invention

[0004] This invention provides a sensitive information processing method, program product, device, and storage medium, which can perform multi-dimensional joint sensitivity detection on code files, accurately identify and process raw data without keywords, and achieve high-precision and low-false-positive sensitive content detection.

[0005] In a first aspect, embodiments of the present invention provide a method for processing sensitive information, including: Obtain the code file to be processed and determine the scannable text content corresponding to the code file; The scannable text content is scanned line by line according to preset sensitive word regular expression rules to determine the sensitive content of the code file; wherein, the sensitive content includes sensitive variable names, sensitive variable values ​​of the sensitive variable names, text content of the line where the sensitive variable names are located, and context information of the line where the sensitive variable names are located; Based on a pre-determined semantic classifier, the sensitive variable names are semantically classified to obtain the target sensitive category and category semantic similarity of the sensitive variable names; Determine the sensitive information entropy of the sensitive variable value, and determine the confidence level of the sensitive content based on the category semantic similarity, the sensitive information entropy, and the context information; The sensitive content is de-identified based on the confidence level, and a detection report is generated based on the de-identified sensitive content and the confidence level of the sensitive content.

[0006] In a second aspect, embodiments of the present invention provide a sensitive information processing apparatus, the apparatus comprising: The data acquisition module is used to acquire the code file to be processed and determine the scannable text content corresponding to the code file; The first analysis module is used to perform line-by-line code scanning on the scannable text content according to preset sensitive word regular expression rules to determine the sensitive content of the code file; wherein, the sensitive content includes sensitive variable names, sensitive variable values ​​of the sensitive variable names, text content of the line where the sensitive variable names are located, and context information of the line where the sensitive variable names are located; The second analysis module is used to perform semantic classification on the sensitive variable name based on a pre-determined semantic classifier, and obtain the target sensitive category and category semantic similarity of the sensitive variable name; The third analysis module is used to determine the sensitive information entropy of the sensitive variable value, and to determine the confidence level of the sensitive content based on the category semantic similarity, the sensitive information entropy and the context information. The report generation module is used to de-identify the sensitive content based on the confidence level, and generate a detection report based on the de-identified sensitive content and the confidence level of the sensitive content.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the sensitive information processing method as described in any of the embodiments of the present invention.

[0008] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the sensitive information processing method as described in any of the embodiments of the present invention.

[0009] Fifthly, embodiments of the present invention provide a computer program product, including a computer program, which, when executed by a processor, implements an emergency response method as described in any of the embodiments of the present invention.

[0010] In this embodiment of the invention, code files are initially screened based on sensitive names and sensitive data formats, covering the blind spot of identifying raw data without keywords. This allows for the rapid preliminary screening of sensitive content from massive amounts of code. Semantic classification of sensitive variable names using a semantic classifier identifies their true meaning and identifies synonyms, abbreviations, or variants of variable names through semantic similarity. Information entropy is used to randomly calculate the values ​​of sensitive variable names. Multi-dimensional joint detection of sensitive content is performed based on semantic similarity, sensitive information entropy, and contextual information, improving the comprehensiveness and accuracy of sensitive information detection. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a first flowchart of a sensitive information processing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the sensitive information processing device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all structures. The acquisition, storage, use, and processing of data in the technical solutions of this application comply with relevant national laws and regulations. It should be noted that, in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used the relevant content of such solutions.

[0014] Figure 1This is a first flowchart of a sensitive information processing method provided in an embodiment of the present invention. The method of this embodiment can perform multi-dimensional joint sensitive detection on code files, accurately identify and process raw data without keywords, and achieve high-precision and low-false-positive sensitive content detection. This method can be executed by a sensitive information processing device provided in this embodiment, which can be implemented in software and / or hardware. The following embodiments will illustrate this using the example of the device integrated into an electronic device. The electronic device can be a server or computer device, etc., used to implement a sensitive information processing method. (Refer to...) Figure 1 The method may specifically include the following steps:

[0015] Step 101: Obtain the code file to be processed and determine the scannable text content corresponding to the code file.

[0016] The code file refers to the source code file that needs to be checked for sensitive information. This code file can be uploaded by the user or automatically obtained by the server. The scannable text content is used for subsequent sensitive content detection. Specifically, the code file to be processed may not necessarily conform to standards and be the actual file requiring sensitive information detection. Therefore, after obtaining the code file, its format and content can be validated first, and the next step can be determined based on the validation results. If the validation passes, the code file can be preprocessed to obtain the scannable text content, thereby standardizing the input format of the code file and reducing subsequent complexity. Optionally, in this solution, obtaining the code file to be processed and determining the corresponding scannable text content includes the following steps A1-A3:

[0017] Step A1: Perform format and content validation on the code file and obtain the validation results.

[0018] The format and content validation checks the attributes of the code file itself. Format validation includes checking if the file extension meets the requirements, such as whether it supports formats like .zip, .jar, .tar.gz, .java, or .py. Content validation includes checking the size of the code file, such as whether it's within a preset limit (e.g., a single package cannot exceed 100 megabytes). If both the format and content of the code file meet the corresponding standards, the validation result is considered passed. If either the format or content of the code file fails, the validation result is considered failed. If the validation result is failed, the code file is not processed further to avoid wasting computational resources.

[0019] Step A2: If the verification result is passed and the code file is in compressed format, then the code file is traversed and decompressed based on the compressed entries of the code file. When the file corresponding to the decompressed compressed entry is in text format, the file corresponding to the compressed entry is determined to be scannable text content.

[0020] The format of code files is not fixed; different formats correspond to different analysis methods. Code file formats include compressed formats, text file formats, binary formats, and other unknown file formats. Specifically, the server can determine the format of a code file based on its extension or header information. If the code file is in a compressed format (e.g., ZIP, JAR, or TAR.GZ), the server can iterate and decompress the file based on its compression entries. For example, the `src` directory of a code file might contain multiple sub-packages with configuration files scattered across different paths. The server can use recursive decompression to expand nested compressed packages and multi-level directories, ensuring that no files potentially containing sensitive information are missed. For instance, a Spring Boot project's JAR package, after decompression, might contain `BOOT-INF / classes / application.properties`, which could contain database password configuration files.

[0021] In one optional implementation, when the server reads the compressed package, it identifies each file one by one. If the decompressed file is a text file, it is identified as scannable text content. Its content is then immediately extracted, and the process proceeds to the subsequent line-by-line code scanning step. After the text file is processed, the memory resources it occupies are released before decompressing the next file. That is, in this scheme, at any given time, only the content of the currently processed file is retained in memory, rather than the entire compressed package. This decouples memory usage from the total size of the compressed package, relating only to the size of a single file. Even with a large compressed package, the peak memory usage remains at a low level. Furthermore, decompression, scanning, and result output can be performed in parallel without waiting for complete decompression before starting detection, resulting in a shorter overall server response time.

[0022] Step A3: If the verification result is passed and the code file is in binary format, then filter the code file to obtain scannable text content.

[0023] In practical applications, code files may contain binary content, such as compiled .class files, image resources, or executable files. This type of content is difficult to process with subsequent regular expression matching and semantic analysis. Therefore, if the code file is in binary format, the server can automatically filter out the corresponding content, retaining only text files such as .java, .py, .properties, .yaml, or .json, reducing unnecessary calculations and preventing binary content from interfering with the detection results. If the code file is in text format, it is directly identified as scannable text content.

[0024] Through the aforementioned format adaptation and content extraction, diverse code files are uniformly converted into standardized text file streams. This ensures that subsequent processing steps only need to process the plain text content, without needing to concern themselves with the original file's encapsulation format. In other words, these steps provide a clean, unified, and controllable input data source for the entire detection process, improving information processing efficiency.

[0025] Step 102: Perform line-by-line code scanning on the scannable text content according to the preset sensitive word regular expression rules to determine the sensitive content of the code file.

[0026] Sensitive content includes sensitive variable names, sensitive variable values, the text content of the line containing the sensitive variable name, and the context information of the line containing the sensitive variable name. Specifically, the server can use preset sensitive word regular expression rules to scan each line of code and quickly filter out lines that may contain sensitive information. Sensitive word regular expression rules include sensitive names and sensitive data formats. Optionally, in this solution, the scannable text content is scanned line by line according to preset sensitive word regular expression rules to determine the sensitive content of the code file, including the following steps B1-B2:

[0027] Step B1: For each line of the scannable text content, perform name matching and data format matching on each code in the current line based on sensitive names and sensitive data formats.

[0028] Sensitive names are pre-defined keywords that may be included in various sensitive content sections. For example, sensitive names may include password-related sensitive names, mobile phone number-related sensitive names, ID card-related sensitive names, account-related sensitive names, email-related sensitive names, and address-related sensitive names. For example, password-related sensitive names may include Chinese and English identifiers such as password, passwd, pwd, secret, and key, as well as its variants such as db_password, apiSecret, and user_pwd. Sensitive data formats are pre-defined data formats that may be included in various sensitive content sections. For example, the data format for mobile phone number-related sensitive content is an eleven-digit number.

[0029] Specifically, after obtaining the scannable text content, each line of code is validated based on a combination of sensitive names and sensitive data formats. The sensitive name regular expression identifies whether variable names, attribute names, or configuration item names contain sensitive semantic identifiers, while the sensitive data format regular expression identifies whether the actual content on the right side of the equals sign, inside quotation marks, or in parameter positions conforms to the format characteristics of sensitive data. Only when both the sensitive name and sensitive data format meet the matching conditions, or if one of them is a strict match, is the code determined to be sensitive content.

[0030] Step B2: For each code in the current line, if the current code matches a sensitive name or sensitive data format, determine the sensitive variable name based on the current code; determine the sensitive content based on the sensitive variable name and the scannable text content.

[0031] Here, the sensitive variable name is the name of the code containing sensitive content. Specifically, for each piece of code in the current line, name matching and data format matching are performed one by one to determine whether the code is a sensitive name or whether the code format conforms to the format characteristics of sensitive data. In this scheme, when matching sensitive names and sensitive data formats for code, only the first successfully matched pattern for each line is determined. For example, if a line contains both a password and a phone number, such as "userPwd": "12345678901", the server can classify it only as the first successfully matched sensitive type according to the order of the regular expression rules. That is, based on "userPwd", the sensitive variable name corresponding to this line of code is determined to be a password, not a phone number. In this way, the same line is avoided from being repeatedly marked as multiple categories, reducing redundancy in subsequent processing.

[0032] Using regular expression rules to filter sensitive content can quickly identify potentially sensitive code from massive amounts of code text with minimal computational cost, thus narrowing down the target range for subsequent semantic analysis and entropy calculation.

[0033] Furthermore, the contextual information of the sensitive variable name is extracted. In this scheme, the two lines preceding and following the line containing the sensitive variable name are determined as the contextual information of the sensitive variable name. That is, the sensitive content corresponding to the sensitive variable name includes a five-line code snippet. This sensitive content provides sufficient context to help understand the purpose and environment of the sensitive information, while avoiding unnecessary storage and transmission overhead due to excessively long context.

[0034] Step 103: Based on a pre-determined semantic classifier, perform semantic classification on the sensitive variable names to obtain the target sensitive category and category semantic similarity of the sensitive variable names.

[0035] Among them, the semantic classifier includes an embedding model and standard vectors corresponding to each sensitive category. The target sensitive category is the sensitive category to which the sensitive variable name belongs. The category semantic similarity is used to measure the matching degree between the sensitive variable name and sensitive variable names of various types in the semantic classifier. Specifically, after obtaining the sensitive variable name, the server can perform semantic analysis on it to confirm whether it truly belongs to a sensitive category. For example, "key" can be a cryptographic key or a database primary key. The sensitive variable name can be converted into an embedding vector through the semantic classifier, and then the similarity between the calculator and the standard vectors of each sensitive category can be calculated. In this solution, optionally, based on a pre-determined semantic classifier, semantic classification is performed on the sensitive variable name to obtain the target sensitive category and category semantic similarity of the sensitive variable name, including: generating a variable name vector for the sensitive variable name through the embedding model; calculating the cosine similarities between the variable name vector and each standard vector, and determining the cosine similarity with the largest value as the category semantic similarity; if the category semantic similarity is greater than a preset threshold, determining the sensitive category corresponding to the category semantic similarity as the target sensitive category.

[0036] Among them, the embedding model is used to convert the sensitive variable name into a vector. The embedding model in this solution can be a lightweight pre-trained model, such as the Sentence-BERT model or the Text2Vec model. The embedding model has a moderate size and can be deployed locally on the server without relying on an external network. The standard vector is a vector pre-constructed according to various sensitive vocabulary. Taking the password category as an example, a group of keywords that can represent the concept of "password" and can cover Chinese and English, full names, abbreviations, and common variants are selected: password, pwd, passwd, pass, 密码, 密钥, secret, secretKey, and credential, etc. Each vocabulary is converted into a high-dimensional vector through the embedding model, and then the element-wise average of all vectors is taken to obtain the standard vector of the password category. The preset threshold is pre-determined based on historical data and big data in the field, etc.

[0037] Specifically, after obtaining the sensitive content, the sensitive variable names within the sensitive content are input into the embedding model to obtain the variable name vector output by the embedding model. Further, the cosine similarity between the variable name vector and each standard vector is calculated. The cosine similarity value ranges from -1 to 1; the closer the value is to 1, the more consistent the directions of the two vectors are, and the closer their semantics are. The highest cosine similarity is selected from all cosine similarities and determined as the category semantic similarity of the variable name vector. If the category semantic similarity is greater than a preset threshold, it indicates that the variable name semantically belongs to the corresponding sensitive category, and the corresponding sensitive category is determined as the target sensitive category. If the category semantic similarity is less than or equal to the preset threshold, it is determined that the sensitive content corresponding to the sensitive variable name has been misjudged, and the server can filter it to avoid misjudgment. For example, with a preset threshold of 0.85, the cosine similarity between the vector of `myPassword` and the standard vector of the password category is 0.92, thus confirming that `myPassword` belongs to the password category. The similarity between `passwordField` and the standard vector of the password category is 0.78, so it can be filtered.

[0038] By converting variable names into vectors and calculating their cosine similarity with standard vectors of sensitive categories, the server gains a deeper semantic understanding of variable names. This allows for more accurate identification of sensitive words such as synonyms, abbreviations, variant words, and cross-language naming, preventing false negatives caused by developers' diverse expressions. Furthermore, threshold filtering avoids false positives for literally similar but semantically unrelated words, improving the accuracy and generalization ability of sensitive information identification.

[0039] Step 104: Determine the sensitive information entropy of the sensitive variable values, and determine the confidence level of the sensitive content based on the category semantic similarity, sensitive information entropy, and contextual information.

[0040] Sensitive information entropy represents the randomness of the string value of a sensitive variable. A higher sensitive information entropy value indicates stronger randomness and a more genuine key or password. In real-world development scenarios, code may contain pseudo-sensitive information, such as strings that formally conform to the characteristics of sensitive data but are not genuine credentials. Examples include sample passwords like admin123, password123, and 123456, test keys like test_key and dummy_secret, and hard-coded default values ​​like changeme and default. This pseudo-sensitive information may consist of meaningful words, consecutive numbers, keyboard sequences, or repetitive patterns, and its corresponding information entropy value may be low. Therefore, sensitive information entropy can, to some extent, reflect whether a sensitive variable value is truly sensitive. In this solution, the Shannon information entropy formula can be used to calculate the sensitive information entropy of sensitive variable values. By calculating the randomness of the variable value of the sensitive variable name using Shannon information entropy, the uniformity of the string's character distribution is quantified into a specific numerical value, effectively distinguishing between highly random genuine sensitive values ​​and low-random sample passwords or test data, laying the foundation for accurate subsequent confidence calculations.

[0041] Confidence score is used to quantify the sensitivity of sensitive content. In this scheme, optionally, the confidence score of sensitive content is determined based on category semantic similarity, sensitive information entropy, and contextual information, including the following steps C1-C2:

[0042] Step C1: When the category semantic similarity is less than the preset similarity and the sensitive information entropy is less than the preset entropy value, the confidence level is determined to be low confidence; when the category semantic similarity is greater than or equal to the preset similarity and the sensitive information entropy is greater than or equal to the preset entropy value, the confidence level is determined to be high confidence.

[0043] The preset entropy value and preset similarity are predetermined based on historical data and domain big data. The preset similarity in this scheme can be the same as the preset threshold in step 103. When the category semantic similarity is less than the preset similarity and the sensitive information entropy is less than the preset entropy value, it indicates that the sensitive content is not actually very "sensitive," and its confidence level is further determined to be low. When the category semantic similarity is greater than or equal to the preset similarity and the sensitive information entropy is greater than or equal to the preset entropy value, it indicates that the sensitive content is indeed very "sensitive," and its confidence level is further determined to be high.

[0044] Step C2: When the category semantic similarity is greater than or equal to the preset similarity and the sensitive information entropy is less than the preset entropy value; or, when the category semantic similarity is less than the preset similarity and the sensitive information entropy is greater than the preset entropy value, determine the confidence level based on the context information.

[0045] Specifically, when the category semantic similarity is greater than or equal to a preset similarity and the sensitive information entropy is less than a preset entropy value; or when the category semantic similarity is less than a preset similarity and the sensitive information entropy is greater than a preset entropy value, it indicates a contradiction in the sensitivity judgment of the variable name and variable value, and the confidence level can be further determined based on the contextual information of the sensitive content. In an optional implementation, when the category semantic similarity is greater than or equal to a preset similarity and the sensitive information entropy is less than a preset entropy value; or when the category semantic similarity is less than a preset similarity and the sensitive information entropy is greater than a preset entropy value, feature matching is performed on each code of the contextual information based on preset enhancement features and suppression features to obtain matching enhancement features or matching suppression features; the matching enhancement features and matching suppression features are weighted and summed according to preset feature weight information to obtain a matching score; if the matching score is less than a first preset score, the confidence level is determined to be low confidence; if the matching score is greater than a second preset score, the confidence level is determined to be high confidence; if the matching score is greater than or equal to the first preset score and less than or equal to the second preset score, the confidence level is determined to be medium confidence.

[0046] The enhancement and suppression features are pre-defined. Enhancement features increase the similarity of sensitive content, while suppression features decrease it. When an identifier (enhanced feature) strongly correlated with a sensitive scenario appears in the context information, that identifier is identified as a matching enhancement feature. For example, enhancement features for network request libraries may include: axios, requests, fetch, curl, HttpClient, and RestTemplate. Enhancement features for encryption operations may include: encrypt, decrypt, cipher, hash, salt, and sign. Suppression mode (reducing confidence or filtering): When an identifier unrelated to a sensitive scenario or clearly harmless appears in the context, the system adjusts the candidate downwards or discards it directly. Suppression features related to game development may include: game_token, playerToken, rewardToken, and game_id. Suppression features related to security mechanisms may include: csrf_token, xsrfToken, antiForgeryToken, and nonce.

[0047] When there is a contradiction between the sensitivity judgment of variable names and variable values, enhancement and suppression features can be matched in their context information to obtain matching enhancement features and / or matching suppression features. This scheme can adopt a multi-feature weighted combination strategy, where each enhancement or suppression feature is assigned a certain weight score. After completing feature matching for the context information of sensitive variable names, the matching enhancement features and matching suppression features are weighted and summed to obtain the matching score. Taking token = "abc123xyz" as an example, if its context information contains headers["Authorization"] = f"Bearer{token}", the server recognizes two strongly matching enhancement features: Authorization and Bearer. If its context information also includes: csrf_token = generate_nonce(), the server can recognize two matching suppression features: the csrf_prefix and the nonce function.

[0048] For example, the context information of token = "xyz789" includes both the "Authorization: Bearer" matching enhancement feature and the " / / local test" matching suppression feature. The matching enhancement and suppression features are weighted and summed according to preset feature weights: the "Authorization: Bearer" matching enhancement feature is +3 points, and the " / / local test" matching suppression feature is -1 point, thus determining the matching score as 2 points. If the first preset score is 3 points and the second preset score is 5 points, then the matching score is determined to be less than the first preset score, and the confidence level of the sensitive content is determined to be low.

[0049] By analyzing contextual information, we can assist in identifying sensitive content where semantic similarity and information entropy are contradictory, improving classification accuracy and the granularity of risk stratification in contradictory scenarios. Through multi-dimensional cross-validation (category semantic similarity, sensitive information entropy, and contextual information), we can more accurately determine the confidence level of sensitive content, significantly reducing false positives and false negatives. Simultaneously, we provide a data foundation for subsequent de-identification processing and the generation of detection reports.

[0050] Step 105: De-identify sensitive content based on confidence level, and generate a detection report based on the de-identified sensitive content and the confidence level of the sensitive content.

[0051] Anonymization is a data protection technique that partially hides or replaces sensitive information, ensuring it retains certain identifiable characteristics while preventing direct reading or reconstruction into the original, complete data. This solution allows for anonymization of highly sensitive content. For example, passwords may retain only the first and last two characters, replacing the rest with asterisks; phone numbers may retain the first three and last four characters, hiding the middle four asterisks; and email addresses may retain the first two characters and the domain name suffix, replacing the username portion with asterisks.

[0052] The detection report in this solution is the final report presented to the user, including structured data such as sensitive content, de-identified sensitive content, and confidence levels. The report may specifically include: the file path and line number of the sensitive content (indicating its location and allowing developers to quickly pinpoint its location in the code without searching the entire project); the type of sensitive information; the confidence level; the original sensitive content; and the de-identified sensitive content.

[0053] The detection report in this solution also includes manual review information. The report is generated based on the anonymized sensitive content and its confidence level, including generating manual review information for sensitive content with medium confidence levels. Specifically, the server can anonymize high-confidence sensitive content and filter low-confidence sensitive content. For medium-confidence sensitive content, the server can generate manual review information based on its original content, contextual information, and confidence level, and add this information to the detection report. Through confidence-based anonymization and report generation, the detection results from the previous stage can be transformed into operable and transferable presentation documents. Anonymization ensures the effectiveness of information protection, and the structured report provides clear judgment criteria for human decision-making.

[0054] The technical solution of this embodiment involves acquiring a code file to be processed and determining the scannable text content corresponding to the code file; scanning the scannable text content line by line according to preset sensitive word regularization rules to determine the sensitive content of the code file; wherein, the sensitive content includes sensitive variable names, sensitive variable values ​​of sensitive variable names, text content of the line where the sensitive variable name is located, and contextual information of the line where the sensitive variable name is located; semantically classifying the sensitive variable names based on a pre-determined semantic classifier to obtain the target sensitive category and category semantic similarity of the sensitive variable names; determining the sensitive information entropy of the sensitive variable values, and determining the confidence level of the sensitive content based on the category semantic similarity, sensitive information entropy, and contextual information; de-identifying the sensitive content based on the confidence level, and generating a detection report based on the de-identified sensitive content and the confidence level of the sensitive content. The technical solution of this embodiment performs initial screening of code files through sensitive names and sensitive data formats, covering the blind spot of identification without keywords in raw data, and can quickly and initially screen out sensitive content from massive amounts of code. Semantic classification of sensitive variable names using a semantic classifier can identify their true meaning. Semantic similarity is used to identify synonyms, abbreviations, or variants of the variable names. Information entropy is used to randomly calculate the values ​​of sensitive variable names. Multi-dimensional joint detection of sensitive content is performed based on semantic similarity, sensitive information entropy, and contextual information, improving the comprehensiveness and accuracy of sensitive information detection.

[0055] Figure 2 This is a schematic diagram of a sensitive information processing apparatus provided in an embodiment of the present invention. This apparatus is suitable for executing the sensitive information processing method provided in an embodiment of the present invention. Figure 2 As shown, the device may specifically include:

[0056] The data acquisition module 201 is used to acquire the code file to be processed and determine the scannable text content corresponding to the code file; The first analysis module 202 is used to perform line-by-line code scanning on the scannable text content according to preset sensitive word regular expression rules to determine the sensitive content of the code file; wherein, the sensitive content includes sensitive variable names, sensitive variable values ​​of the sensitive variable names, text content of the line where the sensitive variable names are located, and context information of the line where the sensitive variable names are located; The second analysis module 203 is used to perform semantic classification on the sensitive variable name based on a pre-determined semantic classifier, and obtain the target sensitive category and category semantic similarity of the sensitive variable name; The third analysis module 204 is used to determine the sensitive information entropy of the sensitive variable value, and to determine the confidence level of the sensitive content based on the category semantic similarity, the sensitive information entropy and the context information. The report generation module 205 is used to de-identify the sensitive content based on the confidence level, and generate a detection report based on the de-identified sensitive content and the confidence level of the sensitive content.

[0057] Optionally, the data acquisition module 201 is specifically used to: perform format and content verification on the code file and obtain the verification result; If the verification result is passed and the code file is in compressed format, then the code file is traversed and decompressed based on the compressed entries of the code file. When the file corresponding to the decompressed compressed entry is in text format, the file corresponding to the compressed entry is determined to be the scannable text content. If the verification result is successful and the code file is in binary format, then the code file is filtered to obtain the scannable text content.

[0058] Optionally, the sensitive word regular expression rule includes sensitive names and sensitive data formats. The first analysis module 202 is specifically used to: perform name matching and data format matching on each code in the current line based on the sensitive names and sensitive data formats for each line of the scannable text content. For each code in the current line, if the current code successfully matches the sensitive name or the sensitive data format, then the sensitive variable name is determined based on the current code; The sensitive content is determined based on the sensitive variable name and the scannable text content.

[0059] Optionally, the semantic classifier includes an embedding model and standard vectors corresponding to each sensitive category; the second analysis module 203 is specifically used to: generate vectors for the sensitive variable names through the embedding model to obtain variable name vectors; Calculate the cosine similarity between the variable name vector and each standard vector, and determine the cosine similarity with the largest value as the category semantic similarity; If the semantic similarity of the categories is greater than a preset threshold, then the sensitive category corresponding to the semantic similarity of the categories is determined to be the target sensitive category.

[0060] Optionally, the third analysis module 204 is specifically used to: determine the confidence level as low confidence level when the semantic similarity of the category is less than a preset similarity and the sensitive information entropy is less than a preset entropy value; When the semantic similarity of the category is greater than or equal to the preset similarity and the sensitive information entropy is greater than or equal to the preset entropy value, the confidence level is determined to be high confidence. When the category semantic similarity is greater than or equal to the preset similarity and the sensitive information entropy is less than the preset entropy value; or, when the category semantic similarity is less than the preset similarity and the sensitive information entropy is greater than the preset entropy value, the confidence level is determined based on the context information.

[0061] Optionally, the third analysis module 204 is further configured to: perform feature matching on each code of the context information based on preset enhancement features and suppression features to obtain matching enhancement features or matching suppression features; The matching enhancement feature and the matching suppression feature are weighted and summed according to the preset feature weight information to obtain the matching score; If the matching score is less than a first preset score, the confidence level is determined to be low confidence; if the matching score is greater than a second preset score, the confidence level is determined to be high confidence; if the matching score is greater than or equal to the first preset score and less than or equal to the second preset score, the confidence level is determined to be medium confidence.

[0062] Optionally, the test report may also include manual review information. The report generation module 205 is specifically used to generate the manual review information based on the sensitive content corresponding to the medium confidence level.

[0063] The sensitive information processing apparatus provided in this embodiment of the invention can execute the sensitive information processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Content not described in detail in this embodiment can be referred to the description in any method embodiment of the invention.

[0064] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, with reference to... Figure 3 , Figure 3 The electronic device 12 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application. Figure 3 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0065] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0066] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0067] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 3 Not shown; usually referred to as a "hard drive"). Although Figure 3 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0068] A program / utility 40 having a set (at least one) of program modules 46 may be stored, for example, in system memory 28. Such program modules 46 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 46 typically perform the functions and / or methods described in the embodiments of this application.

[0069] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 3 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0070] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a sensitive information processing method provided in this embodiment of the invention: obtaining a code file to be processed and determining the scannable text content corresponding to the code file; scanning the scannable text content line by line according to preset sensitive word regularization rules to determine the sensitive content of the code file; wherein, the sensitive content includes a sensitive variable name, a sensitive variable value of the sensitive variable name, the text content of the line where the sensitive variable name is located, and the context information of the line where the sensitive variable name is located; performing semantic classification on the sensitive variable name based on a predetermined semantic classifier to obtain the target sensitive category and category semantic similarity of the sensitive variable name; determining the sensitive information entropy of the sensitive variable value, and determining the confidence level of the sensitive content based on the category semantic similarity, the sensitive information entropy, and the context information; performing desensitization processing on the sensitive content based on the confidence level, and generating a detection report based on the desensitized sensitive content and the confidence level of the sensitive content.

[0071] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a sensitive information processing method as provided in all embodiments of this invention: acquiring a code file to be processed and determining the scannable text content corresponding to the code file; scanning the scannable text content line by line according to preset sensitive word regularization rules to determine the sensitive content of the code file; wherein the sensitive content includes a sensitive variable name, a sensitive variable value of the sensitive variable name, the text content of the line containing the sensitive variable name, and the context information of the line containing the sensitive variable name; performing semantic classification on the sensitive variable name based on a predetermined semantic classifier to obtain the target sensitive category and category semantic similarity of the sensitive variable name; determining the sensitive information entropy of the sensitive variable value, and determining the confidence level of the sensitive content based on the category semantic similarity, the sensitive information entropy, and the context information; performing desensitization processing on the sensitive content based on the confidence level, and generating a detection report based on the desensitized sensitive content and the confidence level of the sensitive content. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, semiconductor electronic devices, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an electronic device, apparatus, or device that can be executed by instructions.

[0072] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in conjunction with an electronic device, apparatus, or device that executes instructions.

[0073] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0074] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0075] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for processing sensitive information, characterized in that, The method includes: Obtain the code file to be processed and determine the scannable text content corresponding to the code file; The scannable text content is scanned line by line according to preset sensitive word regular expression rules to determine the sensitive content of the code file; wherein, the sensitive word regular expression rules include sensitive names and sensitive data formats, and the sensitive content includes sensitive variable names, sensitive variable values ​​of the sensitive variable names, text content of the line where the sensitive variable names are located, and context information of the line where the sensitive variable names are located; Based on a pre-determined semantic classifier, the sensitive variable names are semantically classified to obtain the target sensitive category and category semantic similarity of the sensitive variable names; Determine the sensitive information entropy of the sensitive variable value, and determine the confidence level of the sensitive content based on the category semantic similarity, the sensitive information entropy, and the context information; The sensitive content is de-identified based on the confidence level, and a detection report is generated based on the de-identified sensitive content and the confidence level of the sensitive content.

2. The method according to claim 1, characterized in that, Determining the scannable text content corresponding to the code file includes: The code file is formatted and its contents are validated to obtain the validation results. If the verification result is passed and the code file is in compressed format, then the code file is traversed and decompressed based on the compressed entries of the code file. When the file corresponding to the decompressed compressed entry is in text format, the file corresponding to the compressed entry is determined to be the scannable text content. If the verification result is successful and the code file is in binary format, then the code file is filtered to obtain the scannable text content.

3. The method according to claim 1, characterized in that, The scannable text content is scanned line by line according to preset sensitive word regular expression rules to determine the sensitive content of the code file, including: For each line of the scannable text content, name matching and data format matching are performed on each code in the current line based on the sensitive name and the sensitive data format; For each code in the current line, if the current code successfully matches the sensitive name or the sensitive data format, then the sensitive variable name is determined based on the current code; The sensitive content is determined based on the sensitive variable name and the scannable text content.

4. The method according to claim 1, characterized in that, The semantic classifier includes an embedding model and standard vectors corresponding to each sensitive category; based on the pre-determined semantic classifier, the sensitive variable names are semantically classified to obtain the target sensitive category and category semantic similarity of the sensitive variable names, including: The sensitive variable names are vectorized using the embedding model to obtain a variable name vector. Calculate the cosine similarity between the variable name vector and each standard vector, and determine the cosine similarity with the largest value as the category semantic similarity; If the semantic similarity of the categories is greater than a preset threshold, then the sensitive category corresponding to the semantic similarity of the categories is determined to be the target sensitive category.

5. The method according to claim 1, characterized in that, Determining the confidence level of the sensitive content based on the target sensitive category, the category semantic similarity, the sensitive information entropy, and the context information includes: When the semantic similarity of the categories is less than a preset similarity and the sensitive information entropy is less than a preset entropy value, the confidence level is determined to be low confidence. When the semantic similarity of the category is greater than or equal to the preset similarity and the sensitive information entropy is greater than or equal to the preset entropy value, the confidence level is determined to be high confidence. When the category semantic similarity is greater than or equal to the preset similarity and the sensitive information entropy is less than the preset entropy value; or, when the category semantic similarity is less than the preset similarity and the sensitive information entropy is greater than the preset entropy value, the confidence level is determined based on the context information.

6. The method according to claim 5, characterized in that, Determining the confidence level based on the context information includes: Based on preset enhancement and suppression features, feature matching is performed on each code in the context information to obtain matching enhancement features or matching suppression features; The matching enhancement feature and the matching suppression feature are weighted and summed according to the preset feature weight information to obtain the matching score; If the matching score is less than a first preset score, the confidence level is determined to be low confidence; if the matching score is greater than a second preset score, the confidence level is determined to be high confidence; if the matching score is greater than or equal to the first preset score and less than or equal to the second preset score, the confidence level is determined to be medium confidence.

7. The method according to claim 6, characterized in that, The test report also includes manual review information. The test report is generated based on the de-identified sensitive content and the confidence level of the sensitive content, including: generating the manual review information based on the sensitive content corresponding to the medium confidence level.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the sensitive information processing method as described in any one of claims 1-7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the sensitive information processing method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the sensitive information processing method as described in any one of claims 1-7.

Images