Data processing methods, apparatus, equipment and storage media

Abstract

This invention discloses a data processing method, apparatus, device, and storage medium, comprising: acquiring a log file to be processed; calling a trained privacy leakage risk assessment model to assess the risk of the log file to be processed, obtaining the risk level of the log file to be processed; determining a target de-identification strategy for the log file to be processed based on the risk level; and performing de-identification processing on the log file to be processed according to the target de-identification strategy, thereby obtaining a de-identified target log file. Using embodiments of this invention, different target de-identification strategies can be employed to de-identify log files with different privacy leakage risk levels, effectively solving the technical problem of log files being prone to privacy leakage.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] Existing terminals automatically generate numerous log files during operation, some of which may contain sensitive information such as ID card numbers, names, phone numbers, ages, and purchase records. If this sensitive information is leaked, it will pose a significant threat to user privacy.

[0003] Therefore, there is an urgent need for a method that can effectively prevent the leakage of users' private information in log files. Summary of the Invention

[0004] The purpose of this invention is to provide a data processing method, apparatus, device, and storage medium to solve the technical problem that log files generated by terminals are prone to privacy leaks.

[0005] In a first aspect, embodiments of the present invention provide a data processing method, including:

[0006] Obtain the log file to be processed;

[0007] The trained privacy leakage risk assessment model is invoked to perform a risk assessment on the log file to be processed, and the risk level of the log file to be processed is obtained.

[0008] Determine the target de-identification strategy for the log files to be processed based on the risk level;

[0009] According to the target de-identification strategy, the log file to be processed is de-identified to obtain the target log file after de-identification.

[0010] Furthermore, the risk level includes a first risk level and a second risk level, where the first risk level is greater than the second risk level. The target de-identification strategy includes a first de-identification strategy and a second de-identification strategy, where the de-identification processing intensity of the first de-identification strategy is greater than that of the second de-identification strategy. Determining the target de-identification strategy for the log file to be processed based on the risk level includes:

[0011] If the risk level is the first risk level, then the first de-identification strategy is determined as the target de-identification strategy for the log file to be processed;

[0012] If the risk level is the second risk level, then obtain the target lookup table corresponding to the log file to be processed, and use the second desensitization strategy and the target lookup table as the target desensitization strategy for the log file to be processed.

[0013] Furthermore, before the step of invoking the trained privacy leakage risk assessment model to perform a risk assessment on the log file to be processed, the data processing method further includes:

[0014] Obtain multiple training log files that have not undergone the aforementioned de-identification process, wherein the field types contained in the training log files include at least one of the following types: Chinese type, English type, and character type;

[0015] The risk level of a training log file containing one of the field types is determined as the first risk level, and the risk level of a training log file containing two or more different field types is determined as the second risk level.

[0016] Multiple log files to be trained and the risk level corresponding to each log file to be trained are used as training sample data to train the privacy leakage risk assessment model to be trained.

[0017] The model parameters are optimized based on the objective loss function until the privacy leakage risk assessment model converges, thus obtaining the trained privacy leakage risk assessment model.

[0018] Furthermore, if the risk level is the first risk level, the step of de-identifying the log file to be processed according to the target de-identification strategy includes:

[0019] According to the first desensitization strategy, the sensitive data in the log file to be processed undergoes an irreversible first desensitization process.

[0020] Furthermore, if the risk level is the second risk level, the step of de-identifying the log file to be processed according to the target de-identification strategy includes:

[0021] Determine the weight coefficient of the sensitive data corresponding to each field type in the log file to be processed in the log file to be processed.

[0022] Based on the second desensitization strategy, the target lookup table corresponding to each field type in the log file to be processed, and the weight coefficient of the sensitive data corresponding to each field type, the corresponding sensitive data is subjected to a preset number of recoverable second desensitization processes; wherein, the weight coefficient is proportional to the preset number of times, and the preset number of times is at least once.

[0023] Furthermore, after the step of de-identifying the log file to be processed according to the target de-identification strategy, the data processing further includes:

[0024] Obtain the public key of the target object, and encrypt the target lookup table using the public key to generate an encrypted lookup table;

[0025] The encrypted lookup table and the de-identified target log file are sent to the target object.

[0026] Furthermore, obtaining the log file to be processed includes:

[0027] Retrieve the encrypted log file sent by the log generator;

[0028] The encrypted log file is decrypted to obtain the log file to be processed.

[0029] Secondly, embodiments of the present invention provide a data processing apparatus, comprising:

[0030] The first acquisition module is used to acquire log files to be processed.

[0031] The risk assessment module is used to call a trained privacy leakage risk assessment model to assess the risk of the log file to be processed and obtain the risk level of the log file to be processed.

[0032] The determination module is used to determine the target de-identification strategy for the log file to be processed based on the risk level;

[0033] The desensitization module is used to desensitize the log file to be processed according to the target desensitization strategy to obtain the desensitized target log file.

[0034] Thirdly, embodiments of the present invention provide an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the data processing method described in any of the preceding claims.

[0035] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described data processing methods.

[0036] This invention provides a data processing method, apparatus, device, and storage medium. By calling a trained privacy leakage risk assessment model, the privacy leakage risk level of the log file to be processed can be determined. Thus, a corresponding target desensitization strategy can be determined according to different privacy leakage risk levels. Furthermore, different target desensitization strategies can be used to desensitize log files with different privacy leakage risk levels, effectively solving the technical problem that log files are prone to privacy leakage. Attached Figure Description

[0037] Figure 1 This is a schematic flowchart of a data processing method provided in an embodiment of the present invention;

[0038] Figure 2 This is another flowchart illustrating the data processing method provided in this embodiment of the invention;

[0039] Figure 3 This is a flowchart illustrating a method for training a privacy leakage risk assessment model provided in an embodiment of the present invention.

[0040] Figure 4 This is a schematic diagram of a data processing device provided in an embodiment of the present invention;

[0041] Figure 5 This is another structural schematic diagram of the data processing device provided in the embodiments of the present invention;

[0042] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0043] Figure 7 This is another structural schematic diagram of the electronic device provided in the embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0046] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0047] In current technologies, terminals automatically generate numerous log files during operation, some of which may contain sensitive information such as ID card numbers, names, phone numbers, ages, and purchase records. If this sensitive information is leaked, it poses a significant threat to user privacy.

[0048] To address the technical problems existing in related technologies, embodiments of the present invention provide a data processing method. Please refer to [link / reference needed]. Figure 1 , Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of the present invention, which includes steps 101 to 104.

[0049] Step 101: Obtain the log file to be processed.

[0050] In this embodiment, the log files to be processed are those automatically generated by the system during operation. Different log files contain the same or different data, which may include users' private information, such as ID card numbers and purchase records. Therefore, this embodiment mainly addresses the technical problem of privacy leakage caused by automatically generated log files of existing terminal systems by processing the log files automatically generated by the terminal system.

[0051] Step 102: Call the trained privacy leakage risk assessment model to perform a risk assessment on the log file to be processed, and obtain the risk level of the log file to be processed.

[0052] In this embodiment, the trained privacy leakage risk assessment model is mainly used to identify sensitive data in the input log files to be processed, so as to determine the privacy leakage risk level of the corresponding log files to be processed based on the identified sensitive data.

[0053] A well-trained privacy leakage risk assessment model can determine the privacy leakage risk level of different log files to be processed, thereby enabling the specification of different de-identification processes for different privacy leakage risk levels to ensure the security of sensitive data in files to be processed at different privacy leakage risk levels.

[0054] Specifically, the higher the privacy leakage risk level, the easier it is for sensitive data in the corresponding log files to be leaked; conversely, the lower the privacy leakage risk level, the more difficult it is for sensitive data in the corresponding log files to be leaked.

[0055] Step 103: Determine the target de-identification strategy for the log file to be processed based on the risk level.

[0056] After determining the privacy leakage risk level corresponding to different log files to be processed, this embodiment selects the desensitization strategy corresponding to the privacy leakage level from the preset desensitization strategy pool as the target desensitization strategy for the corresponding log file to be processed, based on the different privacy leakage risk levels.

[0057] The preset desensitization strategy pool includes at least two desensitization strategies, such as a desensitization strategy corresponding to a high privacy leakage risk level that reaches the preset level, and a desensitization strategy corresponding to a low privacy leakage risk level that does not reach the preset level.

[0058] It should be noted that the desensitization strategies in the preset desensitization strategy pool provided in this embodiment are not limited to the two desensitization strategies provided in the above embodiment. They may also include three or more desensitization strategies. The specific number of desensitization strategies needs to be set according to the actual application requirements, and no specific limitation is made here.

[0059] Step 104: According to the target de-identification strategy, the log file to be processed is de-identified to obtain the de-identified target log file.

[0060] After determining the target de-identification strategy for the log file to be processed, the de-identification process can be performed on the log file using the determined target de-identification strategy. Specifically, the de-identification process provided in this embodiment mainly involves determining the target de-identification strategy and transforming, obfuscating, or disguising the sensitive data in the corresponding log file to be processed, thereby achieving reliable protection of the privacy and sensitive data of the log file to be processed and effectively preventing the phenomenon of privacy leakage from log files generated by the terminal system.

[0061] In summary, this invention provides a data processing method, including acquiring a log file to be processed, calling a trained privacy leakage risk assessment model to assess the risk of the log file to be processed, obtaining the risk level of the log file to be processed, determining a target de-identification strategy for the log file to be processed based on the risk level, and performing de-identification processing on the log file to be processed according to the target de-identification strategy to obtain a de-identified target log file. By employing embodiments of this invention, different target de-identification strategies can be used to de-identify log files with different privacy leakage risk levels, effectively solving the technical problem that log files are prone to privacy leakage.

[0062] Please see Figure 2 , Figure 2 This is another flowchart illustrating the data processing method provided in this embodiment of the invention, which includes steps 201 to 210.

[0063] Step 201: Obtain the encrypted log file sent by the log generator.

[0064] In terminal systems, log files are generated by the terminal system's log generator. However, existing log generators in terminal systems typically do not encrypt log files, which can lead to potential privacy leaks in the generated log files.

[0065] To reduce the possibility of privacy leaks during log file transmission, this embodiment encrypts the log files generated by the log generator in the terminal system to obtain encrypted log files. This effectively reduces the possibility of privacy leaks during the transmission of the encrypted log files generated by the log generator.

[0066] Step 202: Decrypt the encrypted log file to obtain the log file to be processed.

[0067] In this embodiment, the log file generated by the log generator is encrypted using the system's public key, resulting in an encrypted log file. Therefore, the encrypted log file can be decrypted using the system's private key, effectively reducing the possibility of privacy leakage during transmission. This yields a log file with low privacy leakage risk.

[0068] Step 203: Call the trained privacy leakage risk assessment model to perform a risk assessment on the log file to be processed, and obtain the risk level of the log file to be processed.

[0069] As an optional embodiment, the risk level includes a first risk level and a second risk level, wherein the first risk level is greater than the second risk level. The target de-identification strategy provided in this embodiment includes a first de-identification strategy and a second de-identification strategy, wherein the de-identification processing intensity of the first de-identification strategy is greater than the de-identification processing intensity of the second de-identification strategy.

[0070] Specifically, the strength of data masking refers to the difficulty of leaking sensitive data in a log file after the data has been masked. For example, a higher masking strength makes it more difficult for sensitive data in the masked log file to be leaked; conversely, a lower masking strength makes it relatively easier for sensitive data in the masked log file to be leaked.

[0071] Step 204: If the risk level is the first risk level, then the first de-identification strategy is determined as the target de-identification strategy for the log file to be processed.

[0072] After determining that the first de-identification strategy is the target de-identification strategy for the log file to be processed, proceed to step 206.

[0073] Step 205: If the risk level is the second risk level, then obtain the target lookup table corresponding to the log file to be processed, and use the second desensitization strategy and the target lookup table as the target desensitization strategy for the log file to be processed.

[0074] In this embodiment, the field types contained in the log file to be processed include at least one of the following: Chinese, English, and character types. Specifically, when the log file to be processed contains only one of the aforementioned field types, due to the single data type, simple data masking may lead to the leakage of sensitive data in the log file, making it easy to crack. However, when the log file to be processed contains two or more different field types, due to the numerous and complex data types, even with simple data masking, it is still difficult to crack the log file after the leakage of sensitive data. Therefore, this embodiment defines the risk level of a log file to be processed containing only one of the aforementioned field types as the first risk level, and the risk level of a log file to be processed containing two or more different field types as the second risk level.

[0075] In this embodiment, the preset de-identification strategy pool includes a first de-identification strategy and a second de-identification strategy. The first de-identification strategy is primarily for log files with a high privacy leakage risk level, i.e., log files containing only one type of data from the stated field. The second de-identification strategy is primarily for log files with a low privacy leakage risk level, i.e., log files containing two or more different types of data from the stated field.

[0076] After determining the first de-identification strategy as the target de-identification strategy for the log file to be processed, proceed with steps 207 to 208.

[0077] Step 206: According to the first desensitization strategy, perform an irreversible first desensitization process on the sensitive data in the log file to be processed to obtain the target log file after the first desensitization process.

[0078] In this embodiment, since the first de-identification strategy targets log files with a high risk of privacy leakage, this embodiment employs an irreversible first de-identification process to de-identify the log files. This prevents log files containing only a single data type from being easily deduced or reconstructed from non-sensitive data in the event of a data leak, thus preventing the leakage of sensitive data containing users' personal privacy. Specifically, the irreversible first de-identification process mainly involves irreversibly processing sensitive data in log files containing only a single character type of data. For example, it may replace specific data in the sensitive data with a unique constant value or "*", or replace specific data in the sensitive data with a random value output by a random function.

[0079] It should be noted that the first unrecoverable desensitization process mainly involves modifying a portion of the sensitive data in the log file to be processed, making the modified data unrecoverable and underivative.

[0080] Because the sensitive data in a log file containing only a single data type is also of a single data type, its data structure is not complex. Any user can usually easily recall the actual data corresponding to the replaced data. However, when a user other than the user obtains the sensitive data after the initial anonymization process, since they are not the user, they cannot deduce the anonymized sensitive data from the unanonymized data, nor can they find any way to recover the anonymized data. This effectively improves the security of log files containing only a single data type.

[0081] Step 207: Determine the weight coefficient of the sensitive data corresponding to each field type in the log file to be processed in the log file to be processed.

[0082] In this embodiment, various types of lookup tables are provided, with each field type corresponding to one lookup table. Specifically, the lookup table is used in the de-identification process to perform corresponding processing on sensitive data in the log file to be processed, based on the data in the lookup table. This corresponding processing includes directly replacing a portion of the sensitive data with data from the lookup table, and using a preset specific function to calculate the data from the lookup table and a portion of the sensitive data, thereby replacing the calculated result with a portion of the sensitive data. It should be noted that the corresponding processing methods are not limited to the two methods mentioned above. Any method that replaces a portion of the sensitive data with data obtained from the data in the lookup table and a portion of the sensitive data is within the protection scope of this embodiment of the invention, and will not be listed here.

[0083] In some embodiments, the weight coefficient of sensitive data in the log file to be processed is determined as follows: calculate the total data length of all sensitive data in the log file to be processed, then calculate the data length of sensitive data corresponding to each field type, and then determine the weight coefficient of sensitive data corresponding to each field type in the log file to be processed based on the ratio of the data length of sensitive data corresponding to each field type to the total data length.

[0084] Step 208: Based on the second desensitization strategy, the target lookup table corresponding to each field type in the log file to be processed, and the weight coefficient of the sensitive data corresponding to each field type, perform a preset number of recoverable second desensitization processes on the corresponding sensitive data to obtain the target log file after the second desensitization process.

[0085] The weighting coefficient is proportional to the preset number of times, and the preset number of times is at least once.

[0086] In some embodiments, the recoverable second desensitization process mainly involves reversibly processing sensitive data in a log file containing two or more different field types. For example, it may replace the sensitive data with data from a target lookup table corresponding to the field types of the sensitive data, or use a pre-defined function to calculate a portion of the sensitive data from the target lookup table and then replace that portion with the calculated result. Thus, the original sensitive data can be recovered by subsequently using the corresponding target lookup table after the second desensitization process.

[0087] As an optional embodiment, when the log file to be processed includes two different field types of data (a first type of data and a second type of data), and the first weight coefficient of the first type of data is greater than the second weight coefficient of the second type of data, then according to the second desensitization strategy and the target lookup table corresponding to the first type of data, the sensitive data of the first type of data undergoes a first-time recoverable second desensitization process, and according to the second desensitization strategy and the target lookup table corresponding to the second type of data, the sensitive data of the second type of data undergoes a second-time recoverable second desensitization process, wherein the first time the number is greater than the second time the number is greater. Specifically, when the first weighting coefficient of the first type of data is 0.8 and the second weighting coefficient of the second type of data is 0.2, the sensitive data of the first type of data can be subjected to four recoverable second desensitization processes. For example, the sensitive data of the same part of the first type of data can be desensitized sequentially using four different desensitization processes according to the corresponding target reference table, or the sensitive data of different parts of the first type of data can be desensitized separately using four different desensitization processes according to the corresponding target reference table. The sensitive data of the second type of data can be subjected to one recoverable second desensitization process. For example, the sensitive data of the second type of data can be desensitized using any desensitization process according to the corresponding target reference table.

[0088] It should be noted that the recoverable second desensitization process mainly modifies a portion of the sensitive data in the log file to be processed based on a target lookup table corresponding to the character type, so that the modified portion of the data can be restored according to the corresponding target lookup table to obtain the original sensitive data.

[0089] Because the sensitive data in the log files to be processed contains two or more different data types, its data structure is complex and easily forgotten even by the user. Furthermore, when a user other than the user obtains this sensitive data after the second de-identification process, the complex data structure makes it difficult to easily deduce the de-identified sensitive data. Therefore, performing a recoverable second de-identification process on the log files to be processed not only reduces the risk of leakage of users' personal sensitive information but also helps users retrieve forgotten personal sensitive information, improving the user experience.

[0090] In this embodiment, the desensitization processing (including the first desensitization processing and the second desensitization processing) provided in this embodiment only processes a portion of the sensitive data in the log file to be processed, and the desensitization processing does not change the original characteristics of the sensitive data in the log file to be processed, the original association relationship with other data, or the rules. For example, when the sensitive data is a bank card number, since the first four digits of the bank card number represent the bank name, the desensitization processing of the sensitive data will not process the first four digits of the bank card number. In addition, in order to facilitate user confirmation, the desensitization processing will not process the last four digits of the bank card number, so as to maintain its original data. When the sensitive data is an email address, since the information following the "@" symbol is the displayed address information, the desensitization process will not process the "@" symbol and the data following it. Furthermore, to facilitate user confirmation, the desensitization process will not process the first preset number of data entries in the email address, thus preserving the original data. This preset number is determined based on the length of the sensitive data. For example, if the email address is longer than 5 entries, the preset number can be set to 1 or 2; if the email address is longer than 8 entries, the preset number can be set to 3 or 4. The specific value of the preset number is not limited here, as long as the number of data entries undergoing desensitization is not less than the preset number.

[0091] Step 209: Obtain the public key of the target object, and encrypt the target lookup table using the public key to generate an encrypted lookup table.

[0092] In this embodiment, when the de-identified target log file needs to be transmitted to the target object, this embodiment obtains the target object's public key and encrypts the target log file to be transmitted using the target object's public key to improve the security of the target log file during transmission and prevent data leakage of the target log file and target lookup table during transmission. This allows the de-identified sensitive data to be directly restored based on the leaked target log file and target lookup table.

[0093] Step 210: Send the encrypted lookup table and the de-identified target log file to the target object.

[0094] After the target object receives the encrypted lookup table and the target log file, it can decrypt the encrypted lookup table using its private key to obtain the target lookup table corresponding to the sensitive data in the de-identified target log file, and can restore the de-identified sensitive data based on the target lookup table.

[0095] As an optional embodiment, please refer to Figure 3 , Figure 3This is a flowchart illustrating a training method for a privacy leakage risk assessment model provided in an embodiment of the present invention, such as... Figure 3 As shown, the training method of the privacy leakage risk assessment model provided in this embodiment includes steps 301 to 304;

[0096] Step 301: Obtain multiple training log files that have not undergone the aforementioned desensitization process.

[0097] In this embodiment, the log file to be trained is similar to the log file to be processed provided in the above embodiment, and the field types it contains also include at least one of Chinese type, English type, and character type.

[0098] Step 302: The risk level of the log file to be trained containing one of the field types is determined as the first risk level, and the risk level of the log file to be trained containing two or more different field types is determined as the second risk level.

[0099] Step 303: Use the multiple log files to be trained and the risk level corresponding to each log file to be trained as training sample data to train the privacy leakage risk assessment model to be trained.

[0100] Step 304: Optimize the model parameters based on the objective loss function until the privacy leakage risk assessment model converges, thus obtaining the trained privacy leakage risk assessment model.

[0101] In this embodiment, the target loss function can be an L1 function or other loss functions that can improve the accuracy of the model.

[0102] In one implementation, a preset number of training iterations can be set, such as 10,000. Once the privacy leakage risk assessment model has been trained 10,000 times, it can be determined that the model has converged, thus obtaining a well-trained privacy leakage risk assessment model.

[0103] In another implementation, a preset recognition accuracy, such as 90%, can be set. Thus, when the privacy leakage risk assessment model is validated, if its recognition accuracy reaches 90% or higher, the model is considered converged, and a well-trained privacy leakage risk assessment model can be obtained.

[0104] In summary, this invention provides a data processing method, including acquiring a log file to be processed, calling a trained privacy leakage risk assessment model to assess the risk of the log file to be processed, obtaining the risk level of the log file to be processed, determining a target de-identification strategy for the log file to be processed based on the risk level, and performing de-identification processing on the log file to be processed according to the target de-identification strategy to obtain a de-identified target log file. By employing this invention, different target de-identification strategies can be used to de-identify log files with different privacy leakage risk levels, effectively solving the technical problem that log files are prone to privacy leakage.

[0105] Based on the method described in the above embodiments, this embodiment will be further described from the perspective of a data processing device. The data processing device can be implemented as an independent entity or integrated into an electronic device, such as a terminal, which may include a mobile phone, a tablet computer, etc.

[0106] Please see Figure 4 , Figure 4 This is a schematic diagram of a data processing device provided in an embodiment of the present invention, such as... Figure 4 As shown, the data processing apparatus 400 provided in this embodiment of the invention includes:

[0107] The first acquisition module 401 is used to acquire the log file to be processed.

[0108] In this embodiment, the first acquisition module 401 is specifically used to: acquire the encrypted log file sent by the log generator; and decrypt the encrypted log file to obtain the log file to be processed.

[0109] The risk assessment module 402 is used to call the trained privacy leakage risk assessment model to perform a risk assessment on the log file to be processed, and obtain the risk level of the log file to be processed.

[0110] The determination module 403 is used to determine the target de-identification strategy for the log file to be processed based on the risk level.

[0111] In this embodiment, the risk level includes a first risk level and a second risk level, where the first risk level is greater than the second risk level. The target de-identification strategy includes a first de-identification strategy and a second de-identification strategy, where the de-identification processing intensity of the first de-identification strategy is greater than that of the second de-identification strategy. The determining module 403 is specifically used to: if the risk level is the first risk level, determine the first de-identification strategy as the target de-identification strategy for the log file to be processed; if the risk level is the second risk level, obtain a target lookup table corresponding to the log file to be processed, and use the second de-identification strategy and the target lookup table as the target de-identification strategy for the log file to be processed.

[0112] The desensitization module 404 is used to perform desensitization processing on the log file to be processed according to the target desensitization strategy, so as to obtain the target log file after desensitization processing.

[0113] In one implementation, if the risk level is the first risk level, the desensitization module 404 is specifically used to: perform an irreversible first desensitization process on the sensitive data in the log file to be processed according to the first desensitization strategy.

[0114] In another implementation, if the risk level is the second risk level, the desensitization module 404 is specifically used to: determine the weight coefficient of the sensitive data corresponding to each field type in the log file to be processed in the log file to be processed; and perform a preset number of recoverable second desensitization processes on the corresponding sensitive data according to the second desensitization strategy, the target lookup table corresponding to each field type in the log file to be processed, and the weight coefficient of the sensitive data corresponding to each field type; wherein the weight coefficient is proportional to the preset number of times, and the preset number of times is at least once.

[0115] As a preferred embodiment, please refer to Figure 5 , Figure 5 This is another structural schematic diagram of the data processing device provided in the embodiments of the present invention, such as... Figure 5 As shown, the data processing device 400 further includes:

[0116] The model training module 405 is used to acquire multiple log files to be trained that have not undergone the aforementioned de-identification processing. The log files to be trained contain at least one type of field, including Chinese, English, and character types. The risk level of the log files to be trained containing one of the aforementioned field types is determined as a first risk level, and the risk level of the log files to be trained containing two or more different of the aforementioned field types is determined as a second risk level. The multiple log files to be trained and the risk levels corresponding to each log file to be trained are used as training sample data to train the privacy leakage risk assessment model to be trained. The model parameters are optimized based on the objective loss function until the privacy leakage risk assessment model converges, thereby obtaining the trained privacy leakage risk assessment model.

[0117] The second acquisition module 406 is used to acquire the encryption public key of the target object and encrypt the target lookup table using the encryption public key to generate an encryption lookup table.

[0118] The sending module 407 is used to send the encrypted lookup table and the de-identified target log file to the target object.

[0119] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0120] Additionally, please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device can be a mobile terminal such as a smartphone, tablet computer, or other similar device. Figure 6 As shown, the electronic device 600 includes a processor 601 and a memory 602. The processor 601 and the memory 602 are electrically connected.

[0121] The processor 601 is the control center of the electronic device 600. It connects various parts of the electronic device through various interfaces and lines. By running or loading the application program stored in the memory 602 and calling the data stored in the memory 602, it performs various functions of the electronic device 600 and processes data, thereby monitoring the electronic device 600 as a whole.

[0122] In this embodiment, the processor 601 in the electronic device 600 loads the instructions corresponding to the processes of one or more applications into the memory 602 according to the following steps, and the processor 601 runs the applications stored in the memory 602 to realize various functions.

[0123] The electronic device 600 can implement the steps of any embodiment of the data processing method provided in the embodiments of the present invention. Therefore, it can achieve the beneficial effects that any data processing method provided in the embodiments of the present invention can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0124] Please see Figure 7 , Figure 7 This is another structural schematic diagram of the electronic device provided in the embodiments of the present invention, such as... Figure 7 As shown, Figure 7 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the data processing method provided in the above embodiments. The electronic device 700 can be a mobile terminal such as a smartphone or a laptop computer.

[0125] RF circuit 710 is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals and vice versa, thereby enabling communication with communication networks or other devices. RF circuit 710 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 710 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). The aforementioned wireless networks may use various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messages, and any other suitable communication protocols, including those that have not yet been developed.

[0126] The memory 720 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing method in the above embodiment. The processor 780 executes various functional applications and data processing by running the software programs and modules stored in the memory 720.

[0127] Memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, memory 720 may further include memory remotely located relative to processor 780, which can be connected to electronic device 700 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0128] The input unit 730 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 730 may include a touch-sensitive surface 731 and other input devices 732. The touch-sensitive surface 731, also known as a touch display screen or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 731), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 731 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 780, and can receive and execute commands sent by the processor 780. In addition, the touch-sensitive surface 731 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 731, the input unit 730 may also include other input devices 732. Specifically, other input devices 732 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0129] Display unit 740 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of electronic device 700. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 740 may include display panel 741, optionally configured as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms. Further, touch-sensitive surface 731 may cover display panel 741. When touch-sensitive surface 731 detects a touch operation on or near it, it transmits the information to processor 780 to determine the type of touch event. Subsequently, processor 780 provides corresponding visual output on display panel 741 according to the type of touch event. Although in the figures, touch-sensitive surface 731 and display panel 741 are implemented as two separate components to achieve input and output functions, in some embodiments, touch-sensitive surface 731 and display panel 741 can be integrated to achieve input and output functions.

[0130] The electronic device 700 may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 741 according to the ambient light level, and the proximity sensor can generate an interruption when the flip is closed or shut down. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the electronic device 700, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0131] Audio circuitry 760, speaker 761, and microphone 762 provide an audio interface between the user and electronic device 700. Audio circuitry 760 converts received audio data into electrical signals and transmits them to speaker 761, where speaker 761 converts them into sound signals for output. Conversely, microphone 762 converts collected sound signals into electrical signals, which are then received by audio circuitry 760, converted back into audio data, and processed by processor 780. The audio data is then transmitted via RF circuitry 710 to, for example, another terminal, or output to memory 720 for further processing. Audio circuitry 760 may also include an earphone jack to facilitate communication between peripheral headphones and electronic device 700.

[0132] Electronic device 700, through transmission module 770 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 770 is shown in the figure, it is understood that it is not an essential component of electronic device 700 and can be omitted as needed without changing the essence of the invention.

[0133] The processor 780 is the control center of the electronic device 700. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 700 by running or executing software programs and / or modules stored in the memory 720, and by calling data stored in the memory 720, thereby providing overall monitoring of the electronic device. Optionally, the processor 780 may include one or more processing cores; in some embodiments, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 780.

[0134] The electronic device 700 also includes a power supply 790 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to the processor 780 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. The power supply 790 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0135] Although not shown, the electronic device 700 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing operations.

[0136] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0137] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any embodiment of the data processing method provided by the present invention.

[0138] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0139] Since the instructions stored in the storage medium can execute the steps in any embodiment of the data processing method provided in the embodiments of the present invention, the beneficial effects that any data processing method provided in the embodiments of the present invention can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0140] The foregoing has provided a detailed description of a data processing method, apparatus, device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application. Moreover, those skilled in the art can make several improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A data processing method, characterized in that, include: Obtain the log file to be processed; The trained privacy leakage risk assessment model is invoked to perform a risk assessment on the log file to be processed, and the risk level of the log file to be processed is obtained. The target de-identification strategy for the log file to be processed is determined based on the risk level; wherein, the risk level includes a first risk level and a second risk level, the first risk level is greater than the second risk level, the target de-identification strategy includes a first de-identification strategy and a second de-identification strategy, and the de-identification processing intensity of the first de-identification strategy is greater than the de-identification processing intensity of the second de-identification strategy. Before the step of calling the trained privacy leakage risk assessment model to assess the risk of the log files to be processed, the data processing method further includes: obtaining multiple log files to be trained that have not undergone the de-identification process, wherein the field types contained in the log files to be trained include at least one type of Chinese, English, and character; determining the risk level of the log files to be trained containing one of the field types as a first risk level, and determining the risk level of the log files to be trained containing two or more different field types as a second risk level; using the multiple log files to be trained and the risk levels corresponding to each log file to be trained as training sample data to train the privacy leakage risk assessment model to be trained; optimizing the model parameters based on the objective loss function until the privacy leakage risk assessment model converges, thereby obtaining the trained privacy leakage risk assessment model; According to the target de-identification strategy, the log file to be processed is de-identified to obtain the target log file after de-identification. If the risk level is the first risk level, the step of performing desensitization processing on the log file to be processed according to the target desensitization strategy includes: performing an unrecoverable first desensitization processing on the sensitive data in the log file to be processed according to the first desensitization strategy; If the risk level is the second risk level, the step of performing desensitization processing on the log file to be processed according to the target desensitization strategy includes: determining the weight coefficient of the sensitive data corresponding to each field type in the log file to be processed in the log file to be processed; and performing a preset number of recoverable second desensitization processing on the corresponding sensitive data according to the second desensitization strategy, the target lookup table corresponding to each field type in the log file to be processed, and the weight coefficient of the sensitive data corresponding to each field type; wherein, the weight coefficient is proportional to the preset number of times, and the preset number of times is at least once.

2. The data processing method as described in claim 1, characterized in that, The step of determining the target de-identification strategy for the log files to be processed based on the risk level includes: If the risk level is the first risk level, then the first de-identification strategy is determined as the target de-identification strategy for the log file to be processed; If the risk level is the second risk level, then obtain the target lookup table corresponding to the log file to be processed, and use the second desensitization strategy and the target lookup table as the target desensitization strategy for the log file to be processed.

3. The data processing method as described in claim 1, characterized in that, After the step of de-identifying the log file to be processed according to the target de-identification strategy, the data processing further includes: Obtain the public key of the target object, and encrypt the target lookup table using the public key to generate an encrypted lookup table; The encrypted lookup table and the de-identified target log file are sent to the target object.

4. The data processing method as described in claim 1, characterized in that, The process of obtaining the log file to be processed includes: Retrieve the encrypted log file sent by the log generator; The encrypted log file is decrypted to obtain the log file to be processed.

5. A data processing apparatus, characterized in that, include: The first acquisition module is used to acquire log files to be processed. The risk assessment module is used to call a trained privacy leakage risk assessment model to assess the risk of the log file to be processed and obtain the risk level of the log file to be processed. The determination module is used to determine the target de-identification strategy for the log file to be processed based on the risk level; wherein, the risk level includes a first risk level and a second risk level, the first risk level is greater than the second risk level, the target de-identification strategy includes a first de-identification strategy and a second de-identification strategy, and the de-identification processing intensity of the first de-identification strategy is greater than the de-identification processing intensity of the second de-identification strategy. The risk assessment module is further configured to: acquire multiple training log files that have not undergone the aforementioned de-identification processing, wherein the field types contained in the training log files include at least one of Chinese, English, and character types; determine the risk level of the training log files containing one of the aforementioned field types as a first risk level, and determine the risk level of the training log files containing two or more different of the aforementioned field types as a second risk level; use the multiple training log files and the risk levels corresponding to each training log file as training sample data to train the privacy leakage risk assessment model to be trained; optimize the model parameters based on the objective loss function until the privacy leakage risk assessment model converges, thereby obtaining the trained privacy leakage risk assessment model; The desensitization module is used to desensitize the log file to be processed according to the target desensitization strategy to obtain the target log file after desensitization. If the risk level is the first risk level, the desensitization module is used to: perform an irreversible first desensitization process on the sensitive data in the log file to be processed according to the first desensitization strategy; If the risk level is the second risk level, the desensitization module is used to: determine the weight coefficient of the sensitive data corresponding to each field type in the log file to be processed in the log file to be processed; and perform a preset number of recoverable second desensitization processes on the corresponding sensitive data according to the second desensitization strategy, the target lookup table corresponding to each field type in the log file to be processed, and the weight coefficient of the sensitive data corresponding to each field type; wherein the weight coefficient is proportional to the preset number of times, and the preset number of times is at least once.

6. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 4.

Images