A non-intrusive method for desensitizing Java application logs

Abstract

This invention proposes a non-intrusive Java application log de-identification method, belonging to the field of data de-identification technology. The method involves rewriting and de-identifying the Java framework's logging module methods using a de-identification program to obtain additional data information; acquiring log record information; performing a preliminary judgment on the log record information; obtaining pass-through data based on the preliminary judgment information; parsing the pass-through information; performing a loop judgment on the k value to obtain loop judgment information; performing de-identification processing based on the loop judgment results to obtain the final processing result; calculating the de-identification efficiency coefficient and judging the de-identification efficiency; calculating the de-identification accuracy coefficient and judging the de-identification accuracy; updating the de-identification process data in real time based on the judgment results; and finally, after all loop judgments have completed de-identification, obtaining the de-identification completion result and outputting the de-identified or un-de-identified log record information.

Description

Technical Field

[0001] This invention proposes a non-intrusive method for de-identifying logs in Java applications, relating to the field of data de-identification technology, specifically to the field of non-intrusive Java application log de-identification technology. Background Technology

[0002] The application outputs logs during operation, which record detailed information about the application's operation, including but not limited to application interface dependency call records, application status, error messages, user operation traces, etc.

[0003] Application logs may not be properly formatted due to problem analysis or other reasons, resulting in sensitive user information, including but not limited to user ID numbers / phone numbers, being recorded in the logs. According to the requirements of the National Personal Information Protection Law, it is illegal to collect, use, process, transmit, provide, or disclose other people's personal information. Therefore, applications should obfuscate / encrypt or desensitize personal privacy information before outputting it to avoid public display.

[0004] All applications need to implement their own de-identification logic, calling the de-identification method to process logs when each application outputs logs. This is intrusive to existing applications and increases the development workload. Summary of the Invention

[0005] This invention provides a non-intrusive method for de-identifying logs in Java applications to solve the above-mentioned problems:

[0006] This invention proposes a non-intrusive method for desensitizing Java application logs, the method comprising:

[0007] S1. Rewrite the Java framework logging module methods using a desensitization program, add desensitization logic to the rewritten methods to obtain additional data information;

[0008] S2. Obtain log record information through a desensitization procedure, make a preliminary judgment on the log record information, obtain preliminary judgment information, and obtain judgment pass data based on the preliminary judgment information;

[0009] S3. The judgment information is parsed by the desensitization program according to the preset single-line log regular expression matching, and then the k value is judged in a loop to obtain the loop judgment information. The desensitization process is performed according to the loop judgment result to obtain the final processing result.

[0010] S4. Calculate the desensitization efficiency coefficient, judge the desensitization efficiency, then calculate the desensitization accuracy coefficient, judge the desensitization accuracy, and update the desensitization process data in real time based on the judgment result.

[0011] S5. After all loop checks have completed the desensitization process, obtain the desensitization completion result and output the log information after desensitization or without desensitization.

[0012] Further, S1 includes:

[0013] Obtain the log information output by the application during runtime, rewrite the method data of the Java framework logging module through a de-identification program, and obtain the rewritten method data;

[0014] By rewriting the method, the desensitization rules for log records are added to obtain the added data information.

[0015] The added data information is preprocessed to obtain preprocessed added data information.

[0016] Further, S2 includes:

[0017] A preliminary assessment is made of the rewritten method data; the preliminary assessment includes assessment of log length and regular expression rules;

[0018] Retrieve the log length judgment result and the regular expression rule judgment result;

[0019] Based on the log length determination result, obtain method data that conforms to the preset range of log length;

[0020] Then, based on the regular expression rule judgment result, method data that conforms to the preset range of the regular expression rule is obtained;

[0021] Then, the data obtained after the initial judgment is used to make the judgment.

[0022] Further, S3 includes:

[0023] Obtain preset single-line log regular expression matching rule information, and use a de-identification program to split the judgment data according to the preset single-line log regular expression matching information to obtain multiple k or v fields;

[0024] Iterate through each field k. When field k is within the preset range of fields to be desensitized, desensitize the v value corresponding to k to obtain the desensitization result.

[0025] If k is not within the preset range of fields to be desensitized, the loop continues to process the data and obtain the final processed data.

[0026] Further, S4 includes:

[0027] Acquire data on the desensitization process and calculate the desensitization efficiency coefficient based on the data.

[0028] The desensitization efficiency coefficient is compared with a preset efficiency coefficient threshold to obtain the efficiency comparison result;

[0029] When the desensitization efficiency coefficient is greater than the preset efficiency coefficient threshold, the desensitization accuracy coefficient is calculated.

[0030] The desensitization accuracy coefficient is compared with a preset accuracy threshold to obtain the accuracy comparison result;

[0031] Based on the accuracy comparison results, determine whether to update the desensitization process.

[0032] Furthermore, the desensitization system includes:

[0033] The rewrite module is used to rewrite the methods of the Java framework logging module through a de-identification program. The rewritten methods add de-identification logic for the log content to obtain additional data information.

[0034] The preliminary judgment module is used to obtain log record information through a de-identification program, perform a preliminary judgment on the log record information, obtain preliminary judgment information, and obtain judgment pass data based on the preliminary judgment information.

[0035] The desensitization and parsing module is used to parse the judgment pass information according to the preset single-line log regular expression matching through the desensitization program, and then perform loop judgment on the k value to obtain loop judgment information, perform desensitization processing according to the loop judgment result, and obtain the final processing result.

[0036] The performance calculation module is used to calculate the desensitization efficiency coefficient, judge the desensitization efficiency, calculate the desensitization accuracy coefficient, judge the desensitization accuracy, and update the desensitization process data in real time based on the judgment result.

[0037] The information output module, after all loop judgments have completed de-sensitization, obtains the de-sensitization completion result and outputs the log record information after de-sensitization or without de-sensitization.

[0038] Furthermore, the rewriting module includes:

[0039] Obtain the log information output by the application during runtime, rewrite the method data of the Java framework logging module through a de-identification program, and obtain the rewritten method data;

[0040] By rewriting the method, the desensitization rules for log records are added to obtain the added data information.

[0041] The added data information is preprocessed to obtain preprocessed added data information.

[0042] Furthermore, the preliminary judgment module includes:

[0043] A preliminary assessment is made of the rewritten method data; the preliminary assessment includes assessment of log length and regular expression rules;

[0044] Retrieve the log length judgment result and the regular expression rule judgment result;

[0045] Based on the log length determination result, obtain method data that conforms to the preset range of log length;

[0046] Then, based on the regular expression rule judgment result, method data that conforms to the preset range of the regular expression rule is obtained;

[0047] Then, the data obtained after the initial judgment is used to make the judgment.

[0048] Furthermore, the desensitization and parsing module includes:

[0049] Obtain preset single-line log regular expression matching rule information, and use a de-identification program to split the judgment data according to the preset single-line log regular expression matching information to obtain multiple k or v fields;

[0050] Iterate through each field k. When field k is within the preset range of fields to be desensitized, desensitize the v value corresponding to k to obtain the desensitization result.

[0051] If k is not within the preset range of fields to be desensitized, the loop continues to process the data and obtain the final processed data.

[0052] Furthermore, the performance calculation module includes:

[0053] Acquire data on the desensitization process and calculate the desensitization efficiency coefficient based on the data.

[0054] The desensitization efficiency coefficient is compared with a preset efficiency coefficient threshold to obtain the efficiency comparison result;

[0055] When the desensitization efficiency coefficient is greater than the preset efficiency coefficient threshold, the desensitization accuracy coefficient is calculated.

[0056] The desensitization accuracy coefficient is compared with a preset accuracy threshold to obtain the accuracy comparison result;

[0057] Based on the accuracy comparison results, determine whether to update the desensitization process.

[0058] The beneficial effects of this invention are as follows: Through de-identification processing, sensitive information in logs is effectively protected from unauthorized acquisition and use, reducing the risk of data leakage. While protecting privacy, the validity and readability of log records are maintained, which is helpful for subsequent log analysis and troubleshooting. By calculating the de-identification efficiency coefficient, the system can continuously optimize the processing flow and improve the speed of de-identification processing. By calculating the de-identification accuracy coefficient, the system can ensure the accuracy of de-identification processing and avoid accidental or missed deletion of sensitive information. Furthermore, the system supports real-time updates of data during the de-identification process, enabling rapid adjustments to processing strategies based on the latest evaluation results, improving the system's flexibility and adaptability. Attached Figure Description

[0059] Figure 1 This is a diagram illustrating a non-intrusive method for desensitizing Java application logs. Detailed Implementation

[0060] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0061] One embodiment of the present invention provides a non-intrusive Java application log desensitization method, the method comprising:

[0062] S1. Rewrite the Java framework logging module methods using a desensitization program, add desensitization logic to the rewritten methods to obtain additional data information;

[0063] S2. Obtain log record information through a desensitization procedure, make a preliminary judgment on the log record information, obtain preliminary judgment information, and obtain judgment pass data based on the preliminary judgment information;

[0064] S3. The judgment information is parsed by the desensitization program according to the preset single-line log regular expression matching, and then the k value is judged in a loop to obtain the loop judgment information. The desensitization process is performed according to the loop judgment result to obtain the final processing result.

[0065] S4. Calculate the desensitization efficiency coefficient, judge the desensitization efficiency, then calculate the desensitization accuracy coefficient, judge the desensitization accuracy, and update the desensitization process data in real time based on the judgment result.

[0066] S5. After all loop checks have completed the desensitization process, obtain the desensitization completion result and output the log information after desensitization or without desensitization, such as... Figure 1 As shown.

[0067] The working principle of the above technical solution is as follows: Raw log records are acquired. A de-identification program is used to initially assess this log information, distinguishing which information requires de-identification (i.e., sensitive information) and which information can remain unchanged (i.e., non-sensitive information). After the initial assessment, a set of data that passes the assessment is obtained, containing log records that require further processing (i.e., de-identification). The de-identification program uses preset single-line log regular expression matching rules to parse the data that passes the assessment. These regular expression matching rules are used to identify specific formats or content in the logs, such as sensitive data like personal identification information and account information. The program enters a loop assessment process, repeatedly assessing the identified sensitive information using a k-value (which may represent a loop or iteration condition, such as the type or quantity of sensitive information). This ensures that all sensitive information is correctly identified and processed. During the loop assessment process, sensitive information is de-identified based on the assessment results, such as replacing real data with placeholders, hash values, or obfuscating it, to protect privacy and security. Simultaneously or after the de-identification process, the system calculates the de-identification efficiency coefficient and the de-identification accuracy coefficient to evaluate the speed and accuracy of the de-identification process. The accuracy of data masking can be evaluated by comparing the differences between the data before and after masking, and by assessing whether necessary non-sensitive information has been retained. Based on the evaluation results, the system may update the data during the masking process in real time to optimize subsequent processing efficiency and accuracy. Once all loop checks are completed and all sensitive information has been masked, the system obtains the final masking result. The system will output the masked log information, or, for some information that does not require masking, directly output the original record. This protects sensitive information from leakage while maintaining the integrity and readability of the logs.

[0068] The technical effects of the above solution are as follows: Through de-identification, sensitive information in logs is effectively protected from unauthorized acquisition and use, reducing the risk of data leakage. While protecting privacy, the validity and readability of log records are maintained, which is helpful for subsequent log analysis and troubleshooting. By calculating the de-identification efficiency coefficient, the system can continuously optimize the processing flow and improve the speed of de-identification. By calculating the de-identification accuracy coefficient, the system can ensure the accuracy of de-identification and avoid accidental or missed deletion of sensitive information. Furthermore, the system supports real-time updates of data during the de-identification process, enabling rapid adjustments to processing strategies based on the latest evaluation results, improving the system's flexibility and adaptability.

[0069] In one embodiment of the present invention, S1 includes:

[0070] Obtain the log information output by the application during runtime, rewrite the method data of the Java framework logging module through a de-identification program, and obtain the rewritten method data;

[0071] By rewriting the method, the desensitization rules for log records are added to obtain the added data information.

[0072] The added data information is preprocessed to obtain preprocessed added data information.

[0073] The working principle of the above technical solution is as follows: During operation, the system or framework continuously generates and outputs log information. This log information contains various types of data, including application running status, user behavior, and system exceptions. To achieve de-identification of the log information, the methods responsible for logging in the Java framework need to be rewritten. This typically involves modifying or extending the API calls of logging frameworks (such as Log4j, SLF4J, etc.). New logging methods are created using inheritance or decorator patterns. These methods de-identify the log information before internally calling the original logging methods. De-identification rules are added to the rewritten method data. These rules, based on regular expressions, keyword matching, or other custom logic, are used to identify and mark sensitive data that needs to be de-identified. De-identification rules can be flexibly configured to adapt to different data protection needs and application scenarios. Log records containing de-identification rule information are preprocessed. Preprocessing steps may include: identifying and extracting sensitive data from the log information; and applying the corresponding de-identification rules to replace the sensitive data with a de-identified form (such as placeholders, hash values, etc.). Ensure that non-sensitive data remains unchanged to maintain log integrity and readability. Preprocessed log information will no longer contain the original sensitive data, but will retain sufficient information for subsequent analysis and troubleshooting.

[0074] The technical effects of the above solution are as follows: By anonymizing log information, the leakage of sensitive data (such as user personal information and system passwords) is effectively prevented, enhancing data protection security. Although sensitive data is anonymized, the preprocessing steps ensure the integrity and readability of non-sensitive data, making the anonymized logs still highly usable and facilitating subsequent analysis and troubleshooting. Anonymization rules can be flexibly configured and extended according to actual needs to adapt to different data protection requirements and application scenarios. This makes the entire anonymization process more flexible and controllable. Anonymization of log information is achieved by rewriting and extending the log framework without modifying business code. This reduces interference with business logic and lowers maintenance costs. As part of the system security strategy, anonymization helps improve the overall system security. By reducing the risk of sensitive data exposure, the possibility of the system suffering malicious attacks and data leaks is reduced.

[0075] In one embodiment of the present invention, S2 includes:

[0076] A preliminary assessment is made of the rewritten method data; the preliminary assessment includes assessment of log length and regular expression rules;

[0077] Retrieve the log length judgment result and the regular expression rule judgment result;

[0078] Based on the log length determination result, obtain method data that conforms to the preset range of log length;

[0079] Then, based on the regular expression rule judgment result, method data that conforms to the preset range of the regular expression rule is obtained;

[0080] Then, the data obtained after the initial judgment is used to make the judgment.

[0081] The working principle of the above technical solution is as follows: A preliminary judgment is made on the rewritten log recording method data. This step is to filter out data that meets both the log length requirements and regular expression constraints to ensure the efficiency and accuracy of subsequent processing. The length of each log record is judged. This typically involves comparing whether the number of bytes or characters in the log record is within a preset range. The preset range can be set according to actual needs, such as excluding excessively long or short log records, as they may not contain valuable information or do not conform to specific log specifications. The log length judgment result is obtained, i.e., which log records meet the preset length range. Regular expression rule judgment is performed on the log records that pass the length judgment. This step is to identify whether the log records contain data with specific patterns, which may be sensitive information or content requiring special processing. Predefined regular expressions are used to match the log records, obtaining the regular expression rule judgment result, i.e., which log records meet the preset range of regular expression rules. The log length judgment result and the regular expression rule judgment result are combined, retaining only those method data that simultaneously meet both preset ranges. This data is considered the preliminary judgment pass data and will be used for subsequent de-identification processing or other analysis operations.

[0082] The technical effects of the above solution are as follows: By using both log length and regular expression rules for judgment, the log records that need to be processed can be filtered more accurately, avoiding the processing of irrelevant or invalid data and improving the accuracy and efficiency of data processing. The preliminary judgment process reduces the amount of data that needs to be processed in subsequent steps, thereby alleviating the system's processing burden and helping to improve overall system performance. The preliminary judgment ensures that only log records that meet specific requirements are retained for subsequent processing; these log records usually contain more valuable information, helping to improve the usability of log data. The preset ranges for log length and regular expression rules can be flexibly configured and expanded according to actual needs to adapt to different application scenarios and data protection requirements. This flexibility makes the entire preliminary judgment process more adaptable to changes and easier to maintain and update. Through preliminary judgment, some log records that may contain sensitive information but do not meet specific specifications can be excluded in advance, further improving data security. This helps reduce the risk of sensitive information leakage and protects user privacy and system security.

[0083] In one embodiment of the present invention, S3 includes:

[0084] Obtain preset single-line log regular expression matching rule information, and use a de-identification program to split the judgment data according to the preset single-line log regular expression matching information to obtain multiple k or v fields;

[0085] Iterate through each field k. When field k is within the preset range of fields to be desensitized, desensitize the v value corresponding to k to obtain the desensitization result.

[0086] If k is not within the preset range of fields to be desensitized, the loop continues to process the data and obtain the final processed data.

[0087] The working principle of the above technical solution is as follows: The system or program needs to read preset single-line log regular expression matching rules from the configuration file. These rules define how to identify key information (i.e., key or value fields) in the log, which is achieved through regular expressions. Using a de-identification program, based on the obtained regular expression matching rule information, the data that passes the initial judgment (i.e., log records that meet the preset range of log length and regular expression rules) is split. The splitting process extracts information from the log records, forming a set of multiple key (k) or value (v) fields. The program iterates through all the split key fields. The purpose of this iteration is to check whether each key field belongs to the preset range of fields that need to be de-identified. When a key field is found to be within the preset range of fields that need to be de-identified, the program de-identifies the corresponding value of that key field. The specific de-identification method may include replacement, encryption, truncation, etc., depending on the de-identification strategy and business requirements. After de-identification, the original value is replaced with the de-identified result, while key fields that are not within the range of fields that need to be de-identified and their corresponding values ​​remain unchanged. After the above steps, all fields requiring data masking have been masked, while fields that do not require masking remain unchanged. At this point, the program will output or store this final processed data.

[0088] The technical effects of the above solution are as follows: By de-identifying sensitive fields, the risk of sensitive information leakage can be effectively reduced, protecting user privacy and system security. This process is automated, reducing the need for manual intervention and improving processing efficiency and accuracy. Simultaneously, by using regular expression matching rules to split and identify log data, the fields requiring de-identification can be quickly and accurately located. The preset range of fields to be de-identified and the de-identification strategy can be flexibly configured and expanded according to actual needs. When business requirements change, only the regular expression matching rules and de-identification strategies in the configuration file need to be updated, without requiring large-scale modifications to the program. During the de-identification process, only the fields requiring de-identification are modified; other non-sensitive fields remain unchanged. This helps maintain data integrity and consistency, ensuring that the de-identified data still has usability. Since the entire de-identification process is automated through configuration files and the program, maintenance and management become relatively simple. Administrators only need to focus on updating the configuration files and de-identification strategies, without needing to delve into the specific implementation details of the program.

[0089] In one embodiment of the present invention, S4 includes:

[0090] Acquire data on the desensitization process and calculate the desensitization efficiency coefficient based on the data.

[0091] The formula for calculating the desensitization efficiency coefficient is as follows:

[0092]

[0093] Among them, T xl R is the desensitization efficiency coefficient. cg R represents the number of successfully anonymized logs. zs M represents the total number of de-identified logs. tm M represents the duration of the desensitization process. zm Q represents the total processing time. t Z is the preset time weight value. js For system resource savings, Z max Q represents the highest resource saving rate in history. z Preset resource weight values;

[0094] The desensitization efficiency coefficient is compared with a preset efficiency coefficient threshold to obtain the efficiency comparison result;

[0095] When the desensitization efficiency coefficient is greater than the preset efficiency coefficient threshold, the desensitization accuracy coefficient is calculated.

[0096] The formula for calculating the desensitization accuracy coefficient is as follows:

[0097]

[0098] Among them, D tm Z represents the desensitization accuracy coefficient. xx To properly de-identify sensitive information data, O zx The total sensitive information data is represented by Wbv, which represents the percentage reduction in false alarm rate. max Q is the percentage reduction in the maximum false alarm rate. w To preset the false alarm weight, L bv Z represents the false negative rate. jc Q represents the overall detection rate. lb Preset the underreporting weight;

[0099] The desensitization accuracy coefficient is compared with a preset accuracy threshold to obtain the accuracy comparison result;

[0100] Based on the accuracy comparison results, determine whether to update the desensitization process.

[0101] The working principle of the above technical solution is as follows: the system collects data on the de-identification process, including the number of successful de-identification logs (R). cg ), Total number of desensitized logs (R) zs Desensitization treatment duration (M) tm ) and total processing time (M zm The desensitization efficiency coefficient (T) is calculated based on the provided formula. xl At the same time, a preset time weight value (Q) is considered. t ) and system resource savings rate (Z js ) and the historical maximum resource saving rate (Z maxThe ratio of ) and multiplied by the preset resource weight value (Q) z To comprehensively evaluate the efficiency of the desensitization process, the calculated desensitization efficiency coefficient (T) will be used. xl ) is compared with a preset efficiency coefficient threshold. If T xl If the value is greater than the preset threshold, it indicates that the desensitization process has reached the expected or better level in terms of efficiency. At this point, the desensitization accuracy coefficient (D) is further calculated. tm The desensitization accuracy coefficient is calculated by taking into account the correctly desensitized sensitive information data (Z). xx Total Sensitive Information Data (O) zx ), the percentage reduction in false alarm rate (W) bv ), Maximum false alarm rate reduction ratio (W) max ), false negative rate (L) bv ), total detection rate (Z) jc ) and the corresponding preset weights (Q) w and Q lb The calculated desensitization accuracy coefficient (D) tm The value is compared with a preset accuracy threshold. Based on the accuracy comparison result, the system determines whether the desensitization process needs to be updated. If D tm If the result is higher than the preset threshold, it indicates that the desensitization process is performing well in terms of accuracy and may not require immediate updates; otherwise, it may be necessary to optimize the desensitization strategy, algorithm, or parameters.

[0102] The technical effects of the above solution are as follows: By comprehensively considering multiple dimensions such as de-identification success rate, processing speed, system resource utilization, false positive rate, and false negative rate, a comprehensive evaluation of the de-identification process is achieved. Based on the evaluation results, the system can intelligently determine whether the de-identification process needs to be updated, providing strong support for optimization decisions. Through regular evaluation and adjustment, the de-identification strategy and algorithm are continuously optimized, thereby improving de-identification efficiency and accuracy and reducing the risk of sensitive information leakage. Considering system resource saving rate during the evaluation process helps optimize resource allocation and improve resource utilization efficiency. Evaluation indicators and thresholds can be flexibly configured and expanded according to actual needs to adapt to different business scenarios and security requirements. A continuous improvement cycle is established, ensuring that the de-identification process remains efficient and accurate through continuous evaluation, optimization, and updates.

[0103] In one embodiment of the present invention, the desensitization system includes:

[0104] The rewrite module is used to rewrite the methods of the Java framework logging module through a de-identification program. The rewritten methods add de-identification logic for the log content to obtain additional data information.

[0105] The preliminary judgment module is used to obtain log record information through a de-identification program, perform a preliminary judgment on the log record information, obtain preliminary judgment information, and obtain judgment pass data based on the preliminary judgment information.

[0106] The desensitization and parsing module is used to parse the judgment pass information according to the preset single-line log regular expression matching through the desensitization program, and then perform loop judgment on the k value to obtain loop judgment information, perform desensitization processing according to the loop judgment result, and obtain the final processing result.

[0107] The performance calculation module is used to calculate the desensitization efficiency coefficient, judge the desensitization efficiency, calculate the desensitization accuracy coefficient, judge the desensitization accuracy, and update the desensitization process data in real time based on the judgment result.

[0108] The information output module, after all loop judgments have completed de-sensitization, obtains the de-sensitization completion result and outputs the log record information after de-sensitization or without de-sensitization.

[0109] The working principle of the above technical solution is as follows: Raw log records are acquired. A de-identification program is used to initially assess this log information, distinguishing which information requires de-identification (i.e., sensitive information) and which information can remain unchanged (i.e., non-sensitive information). After the initial assessment, a set of data that passes the assessment is obtained, containing log records that require further processing (i.e., de-identification). The de-identification program uses preset single-line log regular expression matching rules to parse the data that passes the assessment. These regular expression matching rules are used to identify specific formats or content in the logs, such as sensitive data like personal identification information and account information. The program enters a loop assessment process, repeatedly assessing the identified sensitive information using a k-value (which may represent a loop or iteration condition, such as the type or quantity of sensitive information). This ensures that all sensitive information is correctly identified and processed. During the loop assessment process, sensitive information is de-identified based on the assessment results, such as replacing real data with placeholders, hash values, or obfuscating it, to protect privacy and security. Simultaneously or after the de-identification process, the system calculates the de-identification efficiency coefficient and the de-identification accuracy coefficient to evaluate the speed and accuracy of the de-identification process. The accuracy of data masking can be evaluated by comparing the differences between the data before and after masking, and by assessing whether necessary non-sensitive information has been retained. Based on the evaluation results, the system may update the data during the masking process in real time to optimize subsequent processing efficiency and accuracy. Once all loop checks are completed and all sensitive information has been masked, the system obtains the final masking result. The system will output the masked log information, or, for some information that does not require masking, directly output the original record. This protects sensitive information from leakage while maintaining the integrity and readability of the logs.

[0110] The technical effects of the above solution are as follows: Through de-identification, sensitive information in logs is effectively protected from unauthorized acquisition and use, reducing the risk of data leakage. While protecting privacy, the validity and readability of log records are maintained, which is helpful for subsequent log analysis and troubleshooting. By calculating the de-identification efficiency coefficient, the system can continuously optimize the processing flow and improve the speed of de-identification. By calculating the de-identification accuracy coefficient, the system can ensure the accuracy of de-identification and avoid accidental or missed deletion of sensitive information. Furthermore, the system supports real-time updates of data during the de-identification process, enabling rapid adjustments to processing strategies based on the latest evaluation results, improving the system's flexibility and adaptability.

[0111] In one embodiment of the present invention, the rewriting module includes:

[0112] Obtain the log information output by the application during runtime, rewrite the method data of the Java framework logging module through a de-identification program, and obtain the rewritten method data;

[0113] By rewriting the method, the desensitization rules for log records are added to obtain the added data information.

[0114] The added data information is preprocessed to obtain preprocessed added data information.

[0115] The working principle of the above technical solution is as follows: During operation, the system or framework continuously generates and outputs log information. This log information contains various types of data, including application running status, user behavior, and system exceptions. To achieve de-identification of the log information, the methods responsible for logging in the Java framework need to be rewritten. This typically involves modifying or extending the API calls of logging frameworks (such as Log4j, SLF4J, etc.). New logging methods are created using inheritance or decorator patterns. These methods de-identify the log information before internally calling the original logging methods. De-identification rules are added to the rewritten method data. These rules, based on regular expressions, keyword matching, or other custom logic, are used to identify and mark sensitive data that needs to be de-identified. De-identification rules can be flexibly configured to adapt to different data protection needs and application scenarios. Log records containing de-identification rule information are preprocessed. Preprocessing steps may include: identifying and extracting sensitive data from the log information; and applying the corresponding de-identification rules to replace the sensitive data with a de-identified form (such as placeholders, hash values, etc.). Ensure that non-sensitive data remains unchanged to maintain log integrity and readability. Preprocessed log information will no longer contain the original sensitive data, but will retain sufficient information for subsequent analysis and troubleshooting.

[0116] The technical effects of the above solution are as follows: By anonymizing log information, the leakage of sensitive data (such as user personal information and system passwords) is effectively prevented, enhancing data protection security. Although sensitive data is anonymized, the preprocessing steps ensure the integrity and readability of non-sensitive data, making the anonymized logs still highly usable and facilitating subsequent analysis and troubleshooting. Anonymization rules can be flexibly configured and extended according to actual needs to adapt to different data protection requirements and application scenarios. This makes the entire anonymization process more flexible and controllable. Anonymization of log information is achieved by rewriting and extending the log framework without modifying business code. This reduces interference with business logic and lowers maintenance costs. As part of the system security strategy, anonymization helps improve the overall system security. By reducing the risk of sensitive data exposure, the possibility of the system suffering malicious attacks and data leaks is reduced.

[0117] In one embodiment of the present invention, the preliminary judgment module includes:

[0118] A preliminary assessment is made of the rewritten method data; the preliminary assessment includes assessment of log length and regular expression rules;

[0119] Retrieve the log length judgment result and the regular expression rule judgment result;

[0120] Based on the log length determination result, obtain method data that conforms to the preset range of log length;

[0121] Then, based on the regular expression rule judgment result, method data that conforms to the preset range of the regular expression rule is obtained;

[0122] Then, the data obtained after the initial judgment is used to make the judgment.

[0123] The working principle of the above technical solution is as follows: A preliminary judgment is made on the rewritten log recording method data. This step is to filter out data that meets both the log length requirements and regular expression constraints to ensure the efficiency and accuracy of subsequent processing. The length of each log record is judged. This typically involves comparing whether the number of bytes or characters in the log record is within a preset range. The preset range can be set according to actual needs, such as excluding excessively long or short log records, as they may not contain valuable information or do not conform to specific log specifications. The log length judgment result is obtained, i.e., which log records meet the preset length range. Regular expression rule judgment is performed on the log records that pass the length judgment. This step is to identify whether the log records contain data with specific patterns, which may be sensitive information or content requiring special processing. Predefined regular expressions are used to match the log records, obtaining the regular expression rule judgment result, i.e., which log records meet the preset range of regular expression rules. The log length judgment result and the regular expression rule judgment result are combined, retaining only those method data that simultaneously meet both preset ranges. This data is considered the preliminary judgment pass data and will be used for subsequent de-identification processing or other analysis operations.

[0124] The technical effects of the above solution are as follows: By using both log length and regular expression rules for judgment, the log records that need to be processed can be filtered more accurately, avoiding the processing of irrelevant or invalid data and improving the accuracy and efficiency of data processing. The preliminary judgment process reduces the amount of data that needs to be processed in subsequent steps, thereby alleviating the system's processing burden and helping to improve overall system performance. The preliminary judgment ensures that only log records that meet specific requirements are retained for subsequent processing; these log records usually contain more valuable information, helping to improve the usability of log data. The preset ranges for log length and regular expression rules can be flexibly configured and expanded according to actual needs to adapt to different application scenarios and data protection requirements. This flexibility makes the entire preliminary judgment process more adaptable to changes and easier to maintain and update. Through preliminary judgment, some log records that may contain sensitive information but do not meet specific specifications can be excluded in advance, further improving data security. This helps reduce the risk of sensitive information leakage and protects user privacy and system security.

[0125] In one embodiment of the present invention, the desensitization parsing module includes:

[0126] Obtain preset single-line log regular expression matching rule information, and use a de-identification program to split the judgment data according to the preset single-line log regular expression matching information to obtain multiple k or v fields;

[0127] Iterate through each field k. When field k is within the preset range of fields to be desensitized, desensitize the v value corresponding to k to obtain the desensitization result.

[0128] If k is not within the preset range of fields to be desensitized, the loop continues to process the data and obtain the final processed data.

[0129] The working principle of the above technical solution is as follows: The system or program needs to read preset single-line log regular expression matching rules from the configuration file. These rules define how to identify key information (i.e., key or value fields) in the log, which is achieved through regular expressions. Using a de-identification program, based on the obtained regular expression matching rule information, the data that passes the initial judgment (i.e., log records that meet the preset range of log length and regular expression rules) is split. The splitting process extracts information from the log records, forming a set of multiple key (k) or value (v) fields. The program iterates through all the split key fields. The purpose of this iteration is to check whether each key field belongs to the preset range of fields that need to be de-identified. When a key field is found to be within the preset range of fields that need to be de-identified, the program de-identifies the corresponding value of that key field. The specific de-identification method may include replacement, encryption, truncation, etc., depending on the de-identification strategy and business requirements. After de-identification, the original value is replaced with the de-identified result, while key fields that are not within the range of fields that need to be de-identified and their corresponding values ​​remain unchanged. After the above steps, all fields requiring data masking have been masked, while fields that do not require masking remain unchanged. At this point, the program will output or store this final processed data.

[0130] The technical effects of the above solution are as follows: By de-identifying sensitive fields, the risk of sensitive information leakage can be effectively reduced, protecting user privacy and system security. This process is automated, reducing the need for manual intervention and improving processing efficiency and accuracy. Simultaneously, by using regular expression matching rules to split and identify log data, the fields requiring de-identification can be quickly and accurately located. The preset range of fields to be de-identified and the de-identification strategy can be flexibly configured and expanded according to actual needs. When business requirements change, only the regular expression matching rules and de-identification strategies in the configuration file need to be updated, without requiring large-scale modifications to the program. During the de-identification process, only the fields requiring de-identification are modified; other non-sensitive fields remain unchanged. This helps maintain data integrity and consistency, ensuring that the de-identified data still has usability. Since the entire de-identification process is automated through configuration files and the program, maintenance and management become relatively simple. Administrators only need to focus on updating the configuration files and de-identification strategies, without needing to delve into the specific implementation details of the program.

[0131] In one embodiment of the present invention, the performance calculation module includes:

[0132] Acquire data on the desensitization process and calculate the desensitization efficiency coefficient based on the data.

[0133] The formula for calculating the desensitization efficiency coefficient is as follows:

[0134]

[0135] Among them, T xl R is the desensitization efficiency coefficient. cg R represents the number of successfully anonymized logs. zs M represents the total number of de-identified logs. tm M represents the duration of the desensitization process. zm Q represents the total processing time. t Z is the preset time weight value. js For system resource savings, Z max Q represents the highest resource saving rate in history. z Preset resource weight values;

[0136] The desensitization efficiency coefficient is compared with a preset efficiency coefficient threshold to obtain the efficiency comparison result;

[0137] When the desensitization efficiency coefficient is greater than the preset efficiency coefficient threshold, the desensitization accuracy coefficient is calculated.

[0138] The formula for calculating the desensitization accuracy coefficient is as follows:

[0139]

[0140] Among them, D tm Z represents the desensitization accuracy coefficient. xx To properly de-identify sensitive information data, O zx The total sensitive information data is represented by Wbv, which represents the percentage reduction in false alarm rate. max Q is the percentage reduction in the maximum false alarm rate. w To preset the false alarm weight, L bv Z represents the false negative rate. jc Q represents the overall detection rate. lb Preset the underreporting weight;

[0141] The desensitization accuracy coefficient is compared with a preset accuracy threshold to obtain the accuracy comparison result;

[0142] Based on the accuracy comparison results, determine whether to update the desensitization process.

[0143] The working principle of the above technical solution is as follows: the system collects data on the de-identification process, including the number of successful de-identification logs (R). cg ), Total number of desensitized logs (R) zs Desensitization treatment duration (M) tm ) and total processing time (M zm The desensitization efficiency coefficient (T) is calculated based on the provided formula. xlAt the same time, a preset time weight value (Q) is considered. t ) and system resource savings rate (Z js ) and the historical maximum resource saving rate (Z max The ratio of ) and multiplied by the preset resource weight value (Q) z To comprehensively evaluate the efficiency of the desensitization process, the calculated desensitization efficiency coefficient (T) will be used. xl ) is compared with a preset efficiency coefficient threshold. If T xl If the value is greater than the preset threshold, it indicates that the desensitization process has reached the expected or better level in terms of efficiency. At this point, the desensitization accuracy coefficient (D) is further calculated. tm The desensitization accuracy coefficient is calculated by taking into account the correctly desensitized sensitive information data (Z). xx Total Sensitive Information Data (O) zx ), the percentage reduction in false alarm rate (W) bv ), Maximum false alarm rate reduction ratio (W) max ), false negative rate (L) bv ), total detection rate (Z) jc ) and the corresponding preset weights (Q) w and Q lb The calculated desensitization accuracy coefficient (D) tm The value is compared with a preset accuracy threshold. Based on the accuracy comparison result, the system determines whether the desensitization process needs to be updated. If D tm If the result is higher than the preset threshold, it indicates that the desensitization process is performing well in terms of accuracy and may not require immediate updates; otherwise, it may be necessary to optimize the desensitization strategy, algorithm, or parameters.

[0144] The technical effects of the above solution are as follows: By comprehensively considering multiple dimensions such as de-identification success rate, processing speed, system resource utilization, false positive rate, and false negative rate, a comprehensive evaluation of the de-identification process is achieved. Based on the evaluation results, the system can intelligently determine whether the de-identification process needs to be updated, providing strong support for optimization decisions. Through regular evaluation and adjustment, the de-identification strategy and algorithm are continuously optimized, thereby improving de-identification efficiency and accuracy and reducing the risk of sensitive information leakage. Considering system resource saving rate during the evaluation process helps optimize resource allocation and improve resource utilization efficiency. Evaluation indicators and thresholds can be flexibly configured and expanded according to actual needs to adapt to different business scenarios and security requirements. A continuous improvement cycle is established, ensuring that the de-identification process remains efficient and accurate through continuous evaluation, optimization, and updates.

[0145] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A non-intrusive method for desensitizing Java application logs, characterized in that, The method includes: S1. Rewrite the Java framework logging module methods using a desensitization program, add desensitization logic to the rewritten methods to obtain additional data information; S2. Obtain log record information through a desensitization procedure, make a preliminary judgment on the log record information, obtain preliminary judgment information, and obtain judgment pass data based on the preliminary judgment information; Wherein, S2 includes: A preliminary assessment is made of the log record information; the preliminary assessment includes assessment of the log length and the regular expression rules. Retrieve the log length judgment result and the regular expression rule judgment result; Based on the log length determination result, obtain method data that conforms to the preset range of log length; Then, based on the regular expression rule judgment result, method data that conforms to the preset range of the regular expression rule is obtained; This leads to the acquisition of data for the initial assessment. S3. The judgment data is parsed by the desensitization program according to the preset single-line log regular expression matching, and then the k value is judged in a loop to obtain the loop judgment information. The desensitization process is performed according to the loop judgment result to obtain the final processing result. S4. Calculate the desensitization efficiency coefficient, judge the desensitization efficiency, then calculate the desensitization accuracy coefficient, judge the desensitization accuracy, and update the desensitization process data in real time based on the judgment result. The formula for calculating the desensitization efficiency coefficient is as follows: Among them, T xl R is the desensitization efficiency coefficient. cg R represents the number of successfully anonymized logs. zs M represents the total number of de-identified logs. tm M represents the duration of the desensitization process. zm Q represents the total processing time. t Z is the preset time weight value. js For system resource savings, Z max Q represents the highest resource saving rate in history. z Preset resource weight values; The formula for calculating the desensitization accuracy coefficient is as follows: Among them, D tm Z represents the desensitization accuracy coefficient. xx To properly de-identify sensitive information data, O zx For total sensitive information data, W bv To reduce the false alarm rate by a certain percentage, W max Q is the percentage reduction in the maximum false alarm rate. w To preset the false alarm weight, L bv Z represents the false negative rate. jc Q represents the overall detection rate. lb Preset underreporting weights; S5. After all loop checks have completed the desensitization process, obtain the desensitization completion result and output the log information after desensitization or without desensitization.

2. The non-intrusive Java application log desensitization method according to claim 1, characterized in that, S1 includes: Obtain the log information output by the application during runtime, rewrite the method data of the Java framework logging module through a de-identification program, and obtain the rewritten method data; By rewriting the method, the desensitization rules for log records are added to obtain the added data information. The added data information is preprocessed to obtain preprocessed added data information.

3. The non-intrusive Java application log desensitization method according to claim 1, characterized in that, S3 includes: Obtain preset single-line log regular expression matching rule information, and use a de-identification program to split the judgment data according to the preset single-line log regular expression matching information to obtain multiple k or v fields; Iterate through each field k. When field k is within the preset range of fields to be desensitized, desensitize the v value corresponding to k to obtain the desensitization result. If k is not within the preset range of fields to be desensitized, the loop continues to process the data and obtain the final processed data.

4. The non-intrusive Java application log desensitization method according to claim 1, characterized in that, S4 includes: Acquire data on the desensitization process and calculate the desensitization efficiency coefficient based on the data. The desensitization efficiency coefficient is compared with a preset efficiency coefficient threshold to obtain the efficiency comparison result; When the desensitization efficiency coefficient is greater than the preset efficiency coefficient threshold, the desensitization accuracy coefficient is calculated. The desensitization accuracy coefficient is compared with a preset accuracy threshold to obtain the accuracy comparison result; Based on the accuracy comparison results, determine whether to update the desensitization process.

5. A desensitization system for implementing the non-intrusive Java application log desensitization method of claim 1, characterized in that, The desensitization system includes: The rewrite module is used to rewrite the methods of the Java framework logging module through a de-identification program. The rewritten methods add de-identification logic for the log content to obtain additional data information. The preliminary judgment module is used to obtain log record information through a de-identification program, perform a preliminary judgment on the log record information, obtain preliminary judgment information, and obtain judgment pass data based on the preliminary judgment information. The preliminary judgment module includes: A preliminary assessment is made of the log record information; the preliminary assessment includes assessment of the log length and the regular expression rules. Retrieve the log length judgment result and the regular expression rule judgment result; Based on the log length determination result, obtain method data that conforms to the preset range of log length; Then, based on the regular expression rule judgment result, method data that conforms to the preset range of the regular expression rule is obtained; This leads to the acquisition of data for the initial assessment. The desensitization and parsing module is used to parse the judgment data according to the preset single-line log regular expression matching through the desensitization program, and then perform loop judgment on the k value to obtain loop judgment information, perform desensitization processing according to the loop judgment result, and obtain the final processing result. The performance calculation module is used to calculate the desensitization efficiency coefficient, judge the desensitization efficiency, calculate the desensitization accuracy coefficient, judge the desensitization accuracy, and update the desensitization process data in real time based on the judgment result. The formula for calculating the desensitization efficiency coefficient is as follows: Among them, T xl R is the desensitization efficiency coefficient. cg R represents the number of successfully anonymized logs. zs M represents the total number of de-identified logs. tm M represents the duration of the desensitization process. zm Q represents the total processing time. t Z is the preset time weight value. js For system resource savings, Z max Q represents the highest resource saving rate in history. z Preset resource weight values; The formula for calculating the desensitization accuracy coefficient is as follows: Among them, D tm Z represents the desensitization accuracy coefficient. xx To properly de-identify sensitive information data, O zx For total sensitive information data, W bv To reduce the false alarm rate by a certain percentage, W max Q is the percentage reduction in the maximum false alarm rate. w To preset the false alarm weight, L bv Z represents the false negative rate. jc Q represents the overall detection rate. lb Preset underreporting weights; The information output module is used to obtain the desensitization result after all loop judgments have been completed, and output the log record information after desensitization or no desensitization.

6. The desensitization system for a non-intrusive Java application log desensitization method according to claim 5, characterized in that, The rewriting module includes: Obtain the log information output by the application during runtime, rewrite the method data of the Java framework logging module through a de-identification program, and obtain the rewritten method data; By rewriting the method, the desensitization rules for log records are added to obtain the added data information. The added data information is preprocessed to obtain preprocessed added data information.

7. The desensitization system for a non-intrusive Java application log desensitization method according to claim 5, characterized in that, The desensitization and parsing module includes: Obtain preset single-line log regular expression matching rule information, and use a de-identification program to split the judgment data according to the preset single-line log regular expression matching information to obtain multiple k or v fields; Iterate through each field k. When field k is within the preset range of fields to be desensitized, desensitize the v value corresponding to k to obtain the desensitization result. If k is not within the preset range of fields to be desensitized, the loop continues to process the data and obtain the final processed data.

8. The desensitization system for a non-intrusive Java application log desensitization method according to claim 5, characterized in that, The performance calculation module includes: Acquire data on the desensitization process and calculate the desensitization efficiency coefficient based on the data. The desensitization efficiency coefficient is compared with a preset efficiency coefficient threshold to obtain the efficiency comparison result; When the desensitization efficiency coefficient is greater than the preset efficiency coefficient threshold, the desensitization accuracy coefficient is calculated. The desensitization accuracy coefficient is compared with a preset accuracy threshold to obtain the accuracy comparison result; Based on the accuracy comparison results, determine whether to update the desensitization process.

Images