Method and apparatus for processing log, device and product

Abstract

Provided are a method and apparatus for processing a log, a device and a product. The method for processing a log comprises: determining that a log entry in a log comprises sensitive data (112); converting the sensitive data (112) into fuzzified data (122) to generate a fuzzified log entry (120), the sensitive data (112) having a first precision (114), the fuzzified data having a second precision (124), and the second precision (124) being lower than the first precision (114); and uploading the fuzzified log entry (120) to a server. In this way, the sensitive data (112) can be protected against leakage; in addition, the fuzzified data (122) can retain sufficient information for performance analysis and problem diagnosis, thereby helping engineers improve applications, so as to enhance the performance of the applications and improve user experience.

Description

Methods, apparatus, devices, and products for processing logs Technical Field

[0001] This disclosure relates to the field of data security, and more specifically to methods, apparatus, devices and products for processing logs. Background Technology

[0002] With the rapid development of the mobile internet, mobile applications have become an integral part of people's daily lives. Taking video applications as an example, they have not only completely changed the way people entertain themselves, but have also become an important platform for diverse activities such as information dissemination, social interaction, and creative expression. More and more users are sharing their life experiences and showcasing their talents through video applications, forming a vast content creation ecosystem.

[0003] However, with the surge in user numbers, privacy concerns have gradually emerged. While users enjoy convenient services, the security of their personal information cannot be ignored. Mobile applications typically generate large amounts of log data, including operation records, geolocation, and device information. If this data is not properly protected, it may face the risk of data leakage or misuse, jeopardizing user privacy and security. Summary of the Invention

[0004] In a first aspect of the embodiments of this disclosure, a method for processing logs is provided. The method includes determining that log entries in the logs include sensitive data. The method further includes generating obfuscated log entries by converting the sensitive data into obfuscated data, the sensitive data having a first precision and the obfuscated data having a second precision, wherein the second precision is lower than the first precision. Furthermore, the method includes uploading the obfuscated log entries to a server.

[0005] In a second aspect of the embodiments of this disclosure, an apparatus for processing logs is provided. The apparatus includes a sensitive data determination module configured to determine that log entries in the log include sensitive data. The apparatus also includes an obfuscated log generation module configured to generate obfuscated log entries by converting the sensitive data into obfuscated data, wherein the sensitive data has a first precision, the obfuscated data has a second precision, and the second precision is lower than the first precision. Furthermore, the apparatus includes an obfuscated log upload module configured to upload the obfuscated log entries to a server.

[0006] In a third aspect of embodiments of this disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement a method for processing logs. The method includes determining that log entries in the logs include sensitive data. The method further includes generating obfuscated log entries by converting the sensitive data into obfuscated data, the sensitive data having a first precision, the obfuscated data having a second precision, and the second precision being lower than the first precision. Furthermore, the method includes uploading the obfuscated log entries to a server.

[0007] In a fourth aspect of embodiments of this disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to implement a method for processing logs. The method includes determining that log entries in the logs include sensitive data. The method further includes generating obfuscated log entries by converting the sensitive data into obfuscated data, the sensitive data having a first precision, the obfuscated data having a second precision, and the second precision being lower than the first precision. Furthermore, the method includes uploading the obfuscated log entries to a server.

[0008] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or principal features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 illustrates a schematic diagram of an example environment in which various embodiments of the present disclosure may be implemented;

[0011] Figure 2 shows a flowchart of a method for processing logs according to some embodiments of the present disclosure;

[0012] Figure 3 illustrates a schematic diagram of an example process for processing logs according to some embodiments of the present disclosure;

[0013] Figure 4 shows a schematic diagram of an example classifier for classifying log entries according to some embodiments of the present disclosure;

[0014] Figure 5 shows a schematic diagram of an example obfuscation module for obfuscating log entries according to some embodiments of the present disclosure;

[0015] Figure 6 shows a block diagram of an apparatus for processing logs according to some embodiments of the present disclosure; and

[0016] Figure 7 shows a block diagram of a device capable of implementing several embodiments of the present disclosure. Detailed Implementation

[0017] It is understood that all user-related data involved in this technical solution should be obtained and used only after authorization from the user. This means that if it is necessary to use a user's personal information in this technical solution, the user's explicit consent and authorization are required before obtaining this data; otherwise, no related data collection and use will be carried out. It should also be understood that when implementing this technical solution, relevant laws and regulations should be strictly followed in the process of data collection, use, and storage, and necessary technical measures should be taken to protect user data security and ensure the secure use of data.

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0020] As mentioned above, with the rapid development of the mobile internet, mobile applications have become an indispensable part of people's daily lives. Application logs are automatically generated record files during application operation, typically containing application status, operation records, error messages, performance metrics, and system events. These logs are crucial tools for development engineers, operations engineers, and data analysts to understand application status, troubleshoot problems, optimize performance, and make critical decisions. Depending on different needs, logs record various events from application startup to runtime, such as user actions, background tasks, exceptions, and network requests.

[0021] To protect privacy, some technologies anonymize collected data on the server side. However, since sensitive data has already been transmitted to the server, there is still a risk of interception or leakage. Other technologies completely prohibit data collection; while this method maximizes user privacy, it also prevents necessary performance analysis and troubleshooting. Still other technologies allow users to decide whether to upload data, but due to potential lack of knowledge, users may make inappropriate choices, thus impacting user experience.

[0022] Therefore, embodiments of this disclosure provide a scheme for processing logs. In this scheme, before the logs are uploaded to the server, the client can obfuscate sensitive data. The client can determine whether a log entry in the log contains sensitive data. If it does, the client can obfuscate this sensitive data to generate obfuscated data with lower data precision, thereby obtaining obfuscated log entries containing the obfuscated data. The client can then upload the obfuscated log entries to the server without uploading the original log entries containing the sensitive data.

[0023] In this way, sensitive data is obfuscated before leaving the client, thus protecting it from leakage. Furthermore, the obfuscated data retains enough information for performance analysis and problem diagnosis, helping engineers improve the application to enhance performance and user experience.

[0024] Figure 1 illustrates a schematic diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented. As shown in Figure 1, environment 100 includes a client 102 and a server 104. The client 102 can be any device with computing or processing capabilities. For example, the client 102 can be a mobile phone, tablet computer, smart wearable device, laptop computer, desktop computer, etc. As shown in Figure 1, in environment 100, application 106 can run on client 102. Application 106 can be any application that generates logs during operation. For example, application 106 can be a lifestyle service application, social application, music application, video application, etc. Logs are record files automatically generated by an application during operation, typically including application operation records, status information, performance indicators, error messages, and system events. Logs can help development engineers, maintenance engineers, and data analysis engineers understand the application's working status, troubleshoot problems, and make corresponding decisions. Application 106 can generate log 108 during operation. Log 108 can include any number of log entries 110. For example, one line of data in log 108 can represent one log entry 110.

[0025] In environment 100, client 102 can access log 108 generated by application 106. Client 102 can then analyze log entries 110 to determine if they contain sensitive data. Sensitive data refers to information that, if accessed by unauthorized personnel, could cause harm, threat, or adverse consequences to the user. Sensitive data may include, for example, user privacy data, property-related data, and user identity-related data. In environment 100, client 102 can determine that log entry 110 contains sensitive data 112, and therefore can perform obfuscation on log entry 110. For log entries that do not contain sensitive data, client 102 may not perform obfuscation.

[0026] In environment 100, after client 102 determines that log entry 110 contains sensitive data 112, client 102 can obfuscate the sensitive data 112 to convert it into obfuscated data 122. Obfuscating data refers to replacing data with higher precision with data with lower precision. In environment 100, sensitive data 112 has a precision of 114, and the obfuscated data 122 generated based on sensitive data 112 has a precision of 124; therefore, precision 124 is lower than precision 114. In one example, sensitive data 112 could be time-related data, such as "2023-05-01 10:30:15," accurate to the second. After obfuscating sensitive data 112, obfuscated data 122 could be, for example, "2023-05-01 AM," accurate only to AM or PM, thus reducing the precision of obfuscated data 122. In another example, sensitive data 112 could be geographic location-related data, such as "longitude: x, latitude: y," accurate to the latitude and longitude. After obfuscating sensitive data 112, obfuscated data 122 could be, for example, "city A," accurate only to the city level, thus reducing the precision of obfuscated data 122. After converting sensitive data 112 into obfuscated data 122, non-sensitive data in log entry 110 can remain unchanged, thereby enabling the generation of obfuscated log entry 120.

[0027] In environment 100, after generating obfuscated log entry 120, client 102 can upload the log containing obfuscated log entry 120 to server 104 without uploading the log entry 110 containing sensitive data 112, so that server 104 can analyze the obfuscated log.

[0028] In this way, sensitive data 112 is obfuscated before leaving client 102, thus protecting sensitive data 112 from leakage. Furthermore, the obfuscated data 122 retains sufficient information for performance analysis and problem diagnosis, thereby helping engineers improve application 106 to enhance its performance and improve user experience.

[0029] Figure 2 illustrates a flowchart of a method 200 for processing logs according to some embodiments of the present disclosure. Method 200 can be performed, for example, by a client 102 in the environment 100 shown in Figure 1. As shown in Figure 2, at block 202, the client can determine that log entries in the log contain sensitive data. For example, in the environment 100 shown in Figure 1, client 102 can obtain log 108 generated by application 106. Client 102 can then determine whether log entries 110 contain sensitive data. In environment 100, client 102 can determine that log entries 110 contain sensitive data 112, and therefore can perform obfuscation on log entries 110. For log entries that do not contain sensitive data, client 102 may not perform obfuscation on them. In some embodiments, client 102 can utilize regular expression matching to detect whether sensitive data conforming to a specific format exists in log entries 110. In some embodiments, client 102 can check whether log entries 110 contain specific keywords (e.g., indicators or variable names associated with sensitive data). In some embodiments, client 102 may combine business logic and context to determine whether log entry 110 contains sensitive data. For example, logs associated with login operations may contain sensitive data.

[0030] In box 204, the client can generate obfuscated log entries by converting sensitive data into obfuscated data, where the sensitive data has a first precision, the obfuscated data has a second precision, and the second precision is lower than the first precision. For example, in environment 100 as shown in Figure 1, after client 102 determines that log entry 110 contains sensitive data 112, client 102 can obfuscate the sensitive data 112 to convert it into obfuscated data 122. In environment 100, sensitive data 112 has a precision of 114, and the obfuscated data 122 generated based on sensitive data 112 has a precision of 124, so precision 124 is lower than precision 114. In one example, sensitive data 112 can be time-related data, such as "2023-05-01 10:30:15", which is accurate to the second. After obfuscating sensitive data 112, obfuscated data 122 can be, for example, "2023-05-01 AM", which is only accurate to AM, PM, or PM, thus reducing the precision of obfuscated data 122. In another example, sensitive data 112 could be geographic location-related data, such as "longitude: x, latitude: y," accurate to the latitude and longitude. After obfuscating sensitive data 112, obfuscated data 122 could be, for example, "city A," accurate only to the city level, thus reducing the precision of obfuscated data 122. After converting sensitive data 112 into obfuscated data 122, non-sensitive data in log entry 110 can remain unchanged, thereby enabling the generation of obfuscated log entry 120.

[0031] In box 206, the client can upload obfuscated log entries to the server. For example, in environment 100 as shown in Figure 1, after generating obfuscated log entry 120, client 102 can upload the log containing obfuscated log entry 120 to server 104 without uploading log entry 110 containing sensitive data 112, so that server 104 can analyze the obfuscated log.

[0032] In this way, sensitive data is obfuscated before leaving the client, thus protecting it from leakage. Furthermore, the obfuscated data retains enough information for performance analysis and problem diagnosis, helping engineers improve the application to enhance performance and user experience.

[0033] In some embodiments, when converting sensitive data into obfuscated data, the client can determine the category of a log entry based on the log entry. The client can then determine an obfuscation strategy for that category based on the category of the log entry, and convert the sensitive data into obfuscated data by applying the obfuscation strategy to the log entry.

[0034] In some embodiments, the obfuscated log entry generated above is a first obfuscated log entry, and after generating the first obfuscated log entry, the client can determine its security. In response to a security level below a predetermined security threshold, the client can generate a second obfuscated log entry by converting the sensitive data into second obfuscated data with a third precision, which is lower than the first and second precisions. The client can then upload the second obfuscated log entry to the server without uploading the first obfuscated log entry.

[0035] In some embodiments, the client can obtain feedback data associated with obfuscated log entries. The client can then update the obfuscation policy based on the feedback data and apply the updated obfuscation policy to the client.

[0036] Figure 3 illustrates a schematic diagram of an example process 300 for processing logs according to some embodiments of the present disclosure. As shown in Figure 3, process 300 includes an application 302, a log collection module 304, a classifier 306, a fuzzing module 308, a security assessment module 310, a policy adjustment module 312, and a server 314. In process 300, while application 302 is running, log collection module 304 can capture logs generated by application 302 in real time, such as performance data, error messages, etc. These raw logs can be temporarily stored in the client's local cache for later use.

[0037] In process 300, the log collection module 304 can send raw logs to the classifier 306. The classifier 306 can classify the collected log entries to determine their categories. Log entry categories can include, for example, "time" (i.e., log entries related to time), "geographic location" (i.e., log entries related to geographic location), "object identifier" (i.e., log entries related to object identifier), "device" (i.e., log entries related to device), or "value" (i.e., log entries containing values). For example, a time-related log entry could be "logged in at 2023-05-01 10:30:15," and the classifier 306 can receive this log entry and output the classification result "time." Similarly, a geographic location-related log entry could be "located at latitude x, longitude y," and the classifier 306 can receive this log entry and output the classification result "geographic location." And a device information-related log entry could be "device X connected to network," and the classifier 306 can receive this log entry and output the classification result "device information."

[0038] In process 300, after determining the category of a log entry, classifier 306 can send the category to obfuscation module 308. Obfuscation module 308 can apply corresponding obfuscation strategies for different categories of log entries.

[0039] In some embodiments, the obfuscation module 308 can determine that the category of a log entry is a time-related entry, and then determine the obfuscation strategy to expand the time range of the sensitive data. For example, for a log entry of the category "time" and containing a timestamp, the obfuscation module 308 can convert the timestamp in the log entry into a time period (e.g., "morning", "afternoon", etc.). For example, the original log entry of the "time" category "logged in at 2023-05-01 10:30:15" can be obfuscated to "logged in at 2023-05-01 morning".

[0040] In some embodiments, the obfuscation module 308 can determine that the category of a log entry is a geographically related log entry, and then determine the obfuscation strategy to expand the geographical area of ​​the sensitive data. For example, for a log entry of the category "geographical location" that contains specific geographical coordinates, the obfuscation module 308 can convert the geographical coordinates in the log entry into a larger geographical area. For example, the original log entry of the "geographical location" category "located at latitude x, longitude y" can be obfuscated to "located at city A".

[0041] In some embodiments, the obfuscation module 308 can determine that the category of a log entry is a log entry that needs to be encrypted, and then determine the obfuscation strategy to strengthen the encryption method for sensitive data. For example, for a log entry of the category "object identifier", the obfuscation module 308 can change the encryption method of the object identifier from simple hash to salted hash (i.e., a more secure encryption method). For example, the original log entry of the "object identifier" category "user 12345 performed action X" can be obfuscated to "user 7b6a0…performed action X".

[0042] In some embodiments, the degree of obfuscation of sensitive data by the obfuscation module 308 can be dynamically adjusted based on the sensitivity of the data. For two log entries containing timestamps, their sensitivity can differ due to their different contexts. Therefore, the obfuscation module 308 can apply different obfuscation strategies to log entries of the same category but different sensitivity levels. For example, the obfuscation module 308 can convert timestamps in log entries with lower sensitivity into time periods (e.g., "morning," "afternoon," etc.), and timestamps in log entries with higher sensitivity into months (e.g., "January," "February," etc.). This improves the flexibility of obfuscation processing, thereby enhancing the security of sensitive data.

[0043] In process 300, the obfuscation module 308 can send the obfuscated log entry to the security evaluation module 310. The security evaluation module 310 can determine the security of the obfuscated log entry. If the evaluated security does not meet a predetermined security threshold, the obfuscation module 308 can generate another obfuscated log entry by converting the sensitive data in the original log entry into another obfuscated data. In this process, the obfuscation module 308 can use a stricter obfuscation strategy to convert the sensitive data into less precise obfuscated data. Then, the security evaluation module 310 can evaluate the security of the regenerated obfuscated log entry. If the security meets the predetermined security threshold, the client can upload the obfuscated log entry that meets the security requirements to the server 314. In this way, it is ensured that the obfuscated log entry meets the security requirements, thereby improving data security.

[0044] In some embodiments, when assessing the security of log entries, a sensitivity score, attack scenario score, and data combination score can be determined based on the context of the log entry. Then, a security score is determined based on these scores. Data sensitivity refers to the potential risk to user privacy posed by the information contained in the logs. Different types of log data have different sensitivities. For example, the security assessment module 310 can determine the corresponding sensitivity score based on whether the data in the log entry is highly sensitive, moderately sensitive, or low sensitive. Attack scenario refers to the strategies that attackers might employ. The security assessment module 310 can determine the attack scenario score based on scenarios where the data is likely to be attacked. Data combination refers to the fact that while a single log entry may have a low risk, combining it with other data sources or log entries could lead to a higher risk. The security assessment module 310 can determine the data combination score based on the correlation between the data in the log entry and other log entries. In this way, data sensitivity, attack scenarios, and data combination can be comprehensively considered to determine the security of log entries, thereby improving the accuracy of the assessed security.

[0045] In process 300, the security assessment module 310 can also send the security assessment results to the policy adjustment module 312. The policy adjustment module 312 can update the obfuscation policy based on the security assessment results. For example, the policy adjustment module 312 can determine whether the security assessment results meet the security requirements for log entries. If the security assessment results do not meet the security requirements, the policy adjustment module 312 can update the obfuscation policy to increase the degree of obfuscation of sensitive data. The updated obfuscation policy converts sensitive data into data with lower precision than the previous obfuscated data. For example, the existing policy for log entries categorized as "time" is accurate to the hour; the updated obfuscation policy could only display "AM" or "PM". As another example, the existing policy for log entries categorized as "geographic location" is accurate to the city; the updated obfuscation policy could expand the obfuscation level to the province. In this way, the obfuscation policy can be dynamically adjusted, thereby improving the security of obfuscated data.

[0046] In process 300, if the security assessment result does not meet the security requirements, the policy adjustment module 312 can also trigger the optimization classifier 306. By optimizing the classifier 306, the accuracy of the classification results generated by the classifier 306 can be improved, enabling the obfuscation module 308 to apply an obfuscation strategy more suitable for the log entry, thereby improving the security of the obfuscated log.

[0047] In some embodiments, if the security assessment result does not meet the security requirements, the security threshold used in the security assessment module 310 can be adjusted so that the security of the obfuscated log entries determined by the security assessment module 310 cannot meet the adjusted security threshold, thereby triggering an update to the obfuscation strategy or an optimization of the classifier 306, as well as re-obfuscating the original log entries.

[0048] In some embodiments, when uploading obfuscated logs to server 314, the data can be sent to server 314 via a secure transport protocol (e.g., HTTPS). The logs can be encrypted during this transmission to improve data security. In some embodiments, after receiving the obfuscated logs, server 314 can perform further analysis and processing, such as performance monitoring and error diagnosis, while adhering to the principle of data minimization and retaining only necessary information.

[0049] In some embodiments, when determining the category of a log entry, the classifier can generate preprocessed log entries by preprocessing the log entries, and then the classifier can extract log features from the preprocessed log entries. The classifier can determine the category of the log entry based on the log features.

[0050] Figure 4 illustrates a schematic diagram of an example classifier 400 for classifying log entries according to some embodiments of the present disclosure. As shown in Figure 4, the classifier 400 includes a preprocessing module 402, a feature extraction module 404, and a prediction module 406. The preprocessing module 402 can receive log entries 408 and clean and standardize them to generate preprocessed log entries 410. For example, the preprocessing module 402 can remove redundant information or irrelevant data that does not need to participate in log analysis from the log entries 408. Then, the preprocessing module 402 can standardize the data in the log entries 408, for example, by unifying data such as timestamps and location coordinates into the same standard format. In this way, the accuracy of the classification results for log entries 408 can be improved.

[0051] In classifier 400, preprocessed log entries 410 can be input to feature extraction module 404. Feature extraction module 404 can extract log features 412 from the preprocessed log entries 410. In this process, feature extraction module 404 can extract key information that can effectively represent the characteristics of the log entries from the preprocessed log entries 410 in text form, thereby converting the preprocessed log entries 410 into a data form that can be input into the machine learning model. For example, feature extraction module 404 can perform operations such as word segmentation, stop word removal, stemming, lemmatization, and punctuation removal on the preprocessed log entries 410. Then, feature extraction module 404 can parse log fields (e.g., including field names and values) from the processed text. Then, feature extraction module 404 can vectorize the extracted log fields to convert them into log features 412 that can be input into the prediction model.

[0052] Then, log feature 412 can be input into prediction module 406, which can predict the category 414 of log entry 408 based on log feature 412. Prediction module 406 can be, for example, a trained machine learning model such as a decision tree, support vector machine, or deep neural network. Category 414 can be, for example, "time," "geographic location," or "object identifier." Then, category 414 can be associated with log entry 408. In this way, the accuracy of the classification result can be improved, thereby improving the effect of subsequent fuzzing processing.

[0053] Figure 5 illustrates a schematic diagram of an example obfuscation module 500 for obfuscating log entries according to some embodiments of the present disclosure. As shown in Figure 5, the obfuscation module 500 can receive a log entry 502, which includes sensitive data 504 and a category 506. The category 506 can be a classification result for the log entry 502 generated by a classifier (e.g., classifier 400). Furthermore, the obfuscation module 500 can also obtain a mapping 508 from categories to obfuscation strategies. The mapping 508 includes multiple log categories (e.g., categories 510-1, 510-2, ..., 510-N, collectively referred to as category 510) and multiple obfuscation strategies corresponding to the multiple log categories (e.g., obfuscation strategies 512-1, 512-2, ..., 512-N, collectively referred to as obfuscation strategy 512). As shown in Figure 5, fuzzification strategy 512-1 corresponds to category 510-1, fuzzification strategy 512-2 corresponds to category 510-2, and fuzzification strategy 512-N corresponds to category 510-N.

[0054] For example, category 510-1 could be "time", and the obfuscation strategy 512-1 could be to convert a timestamp accurate to the second into a time period (e.g., "morning" or "afternoon"). As another example, category 510-2 could be "geographic location", and the obfuscation strategy 512-2 could be to convert geographic coordinates into cities.

[0055] Then, the obfuscation module 500 determines the obfuscation strategy 514 for category 506 based on the category 506 and mapping 508 of log entry 502. For example, if category 506 is the same as category 510-1, then obfuscation strategy 512-1 can be determined as obfuscation strategy 514 for category 506. After determining the obfuscation strategy 514, the obfuscation module 500 can apply the obfuscation strategy 514 to log entry 502 to convert sensitive data 504 into obfuscated data, thereby generating obfuscated log entry 516.

[0056] In some embodiments, the obfuscation module 500 can verify the obfuscated log entry 516 to determine if it still retains analytical value. In some embodiments, the obfuscation module 500 can perform functional verification on the obfuscated log entry 516 to determine if it can still effectively support a specific analytical scenario, such as performance monitoring or error troubleshooting. For example, the obfuscation module 500 can verify its functionality by determining that the obfuscated log entry 516 includes specific fields relevant to a particular analytical scenario. In some embodiments, the obfuscation module 500 can perform integrity verification on the obfuscated log entry 516. For example, the obfuscation module 500 can verify the integrity of the obfuscated log entry 516 by determining that the statistical characteristics of the log containing the obfuscated log entry 516 are consistent with the statistical characteristics of the original log. In this way, while enhancing data security, it is ensured that the log can still effectively support subsequent tasks such as performance analysis and fault diagnosis.

[0057] Figure 6 shows a block diagram of an apparatus 600 for processing logs according to some embodiments of the present disclosure. As shown in Figure 6, the apparatus 600 includes a sensitive data determination module 602, configured to determine that log entries in the log include sensitive data. The apparatus 600 also includes an obfuscated log generation module 604, configured to generate obfuscated log entries by converting the sensitive data into obfuscated data, wherein the sensitive data has a first precision, the obfuscated data has a second precision, and the second precision is lower than the first precision. Furthermore, the apparatus 600 includes an obfuscated log upload module 606, configured to upload the obfuscated log entries to a server.

[0058] In some embodiments, the obfuscated log generation module includes: a first category determination module configured to determine the category of a log entry based on the log entry; a category use module configured to determine an obfuscation policy for the category based on the category of the log entry; and a policy application module configured to convert sensitive data into obfuscated data by applying the obfuscation policy to the log entry.

[0059] In some embodiments, the category determination module includes: a preprocessing module configured to generate preprocessed log entries by preprocessing the log entries; a feature extraction module configured to extract log features from the preprocessed log entries; and

[0060] A feature utilization module is configured to determine the category of log entries based on log features. In some embodiments, the category utilization module includes:

[0061] The second category determination module is configured to determine that the category of a log entry is a time-related log entry; and the time range expansion module is configured to determine the obfuscation strategy as expanding the time range of sensitive data.

[0062] In some embodiments, the category usage module includes: a third category determination module configured to determine that the category of a log entry is a geographically related log entry; and a geographical area expansion module configured to determine the obfuscation strategy as expanding the geographical area of ​​sensitive data.

[0063] In some embodiments, the category using module includes: a fourth category determination module configured to determine that the category of a log entry is a log entry that needs to be encrypted; and an encryption method adjustment module configured to determine the obfuscation strategy as an enhanced encryption method for sensitive data.

[0064] In some embodiments, the obfuscated data is first obfuscated data, the obfuscated log entry is a first obfuscated log entry, and the obfuscated log upload module includes: a security determination module configured to determine the security of the first obfuscated log entry; a security comparison module configured to generate a second obfuscated log entry by converting sensitive data into second obfuscated data in response to a security threshold not being met, the second obfuscated data having a third precision, and the third precision being lower than the first precision and the second precision; and a log upload module configured to upload the second obfuscated log entry to a server without uploading the first obfuscated log entry.

[0065] In some embodiments, the apparatus 600 further includes: an evaluation result acquisition module configured to acquire a security evaluation result associated with an obfuscated log entry; a policy update module configured to update an obfuscation policy based on the security evaluation result; and a policy application module configured to apply the updated obfuscation policy to the client.

[0066] In some embodiments, updating the obfuscation strategy based on feedback data includes at least one of the following: updating the obfuscation strategy based on security assessment results, wherein the updated obfuscation strategy converts sensitive data into data with lower precision than the obfuscated data; adjusting the security threshold based on the security assessment results; or optimizing the classifier used to determine the category of log entries based on the security assessment results.

[0067] It is understood that by utilizing the apparatus 600 of this disclosure, at least one of the many advantages achievable by the methods or processes described above can be realized. For example, sensitive data is obfuscated before leaving the client, thereby protecting sensitive data from disclosure. Furthermore, obfuscated data retains sufficient information for performance analysis and problem diagnosis, thereby helping engineers improve the application to enhance application performance and improve user experience.

[0068] Figure 7 shows a block diagram of a device 700 capable of implementing various embodiments of the present disclosure. Device 700 may, for example, be client 102 as shown in Figure 1. As shown in Figure 7, device 700 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 702 or loaded from storage unit 708 into random access memory (RAM) 703. Various programs and data required for the operation of device 700 may also be stored in RAM 703. The CPU / GPU 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704. Although not shown in Figure 7, device 700 may also include a coprocessor.

[0069] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0070] The various methods or processes described above can be executed by CPU / GPU 701. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by CPU / GPU 701, one or more steps or actions in the methods or processes described above can be performed.

[0071] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0072] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0073] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0074] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​and conventional procedural programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0075] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0076] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0077] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0078] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing logs, comprising: The log entries in the log contain sensitive data; Obfuscated log entries are generated by converting the sensitive data into obfuscated data, wherein the sensitive data has a first precision, the obfuscated data has a second precision, and the second precision is lower than the first precision. as well as Upload the obfuscated log entries to the server.

2. The method according to claim 1, wherein generating the obfuscated log entry by converting the sensitive data into the obfuscated data comprises: The category of the log entry is determined based on the log entry; The obfuscation strategy for the category is determined based on the category of the log entry; as well as The sensitive data is transformed into obfuscated data by applying the obfuscation strategy to the log entries.

3. The method of claim 2, wherein determining the category of the log entry based on the log entry comprises: Preprocessed log entries are generated by preprocessing the log entries. Extract log features from the preprocessed log entries; as well as The category of the log entry is determined based on the log characteristics.

4. The method of claim 2, wherein determining the obfuscation strategy for the category based on the category of the log entry comprises: The category of the log entry is determined to be a time-related log entry; as well as The obfuscation strategy is determined to be to expand the time range of the sensitive data.

5. The method of claim 2, wherein determining the obfuscation strategy for the category based on the category of the log entry comprises: The category of the log entry is determined to be a geographically related log entry; as well as The obfuscation strategy is determined to expand the geographical area of ​​the sensitive data.

6. The method of claim 2, wherein determining the obfuscation strategy for the category based on the category of the log entry comprises: Determine that the category of the log entry is a log entry that needs to be encrypted; as well as The obfuscation strategy is determined as a way to strengthen the encryption of the sensitive data.

7. The method according to claim 1, wherein the obfuscated data is first obfuscated data, the obfuscated log entry is a first obfuscated log entry, and uploading the obfuscated log entry to the server comprises: Determine the security of the first obfuscated log entry; In response to the security not meeting a predetermined security threshold, a second obfuscated log entry is generated by converting the sensitive data into second obfuscated data, the second obfuscated data having a third precision, and the third precision being lower than the first precision and the second precision; as well as The second obfuscated log entry is uploaded to the server, but the first obfuscated log entry is not uploaded.

8. The method according to claim 1, further comprising: Obtain the security assessment results associated with the obfuscated log entries; The obfuscation strategy is updated based on the security assessment results. as well as Apply the updated obfuscation strategy to the client.

9. The method of claim 8, wherein updating the obfuscation strategy based on the feedback data comprises at least one of the following: The obfuscation strategy is updated based on the security assessment results, wherein the updated obfuscation strategy converts the sensitive data into data with lower precision than the obfuscated data. The security threshold is adjusted based on the security assessment results. or The classifier used to determine the category of the log entries is optimized based on the security assessment results.

10. An apparatus for processing logs, comprising: The sensitive data determination module is configured to determine whether log entries in the log contain sensitive data. The obfuscated log generation module is configured to generate obfuscated log entries by converting the sensitive data into obfuscated data, wherein the sensitive data has a first precision, the obfuscated data has a second precision, and the second precision is lower than the first precision. as well as The obfuscated log upload module is configured to upload the obfuscated log entries to the server.

11. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the method according to any one of claims 1 to 9.

Images