Log desensitization processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a log desensitization processing method and device, electronic equipment and a storage medium, belongs to the technical field of artificial intelligence, and is suitable for financial science and technology scenes. The method comprises the following steps: acquiring desensitization configuration information from a desensitization configuration database; in response to the log output request, obtaining a desensitized data type, and obtaining an original business log according to the desensitized data type; performing content type identification on the original business log to obtain a log content type; performing desensitization processing on the original business log based on the log content type and the desensitization configuration information, and outputting a target business log; wherein the target business log comprises a business desensitization field; and updating the desensitization configuration information stored in the desensitization configuration database based on the service desensitization field. According to the embodiment of the invention, the accuracy of log desensitization can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology and is applicable to financial technology scenarios, and in particular to a log desensitization processing method and device, electronic equipment, and storage medium. Background Art

[0002] Log desensitization is a technology that identifies and desensitizes sensitive information in logs, improving data security and compliance. Log desensitization can be applied in multiple scenarios. For example, in financial scenarios, desensitization is required when storing and processing bank transaction logs and insurance product application and claims logs.

[0003] Currently, log masking primarily involves masking data during log generation. This requires embedding masking logic into the log generation code, creating fixed masking rules. However, when the log scenario or content structure changes, these fixed masking rules may no longer apply, affecting the accuracy of log masking.

[0004] Therefore, how to improve the accuracy of log desensitization has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of the embodiments of the present application is to provide a log desensitization processing method and device, electronic device and storage medium, aiming to improve the accuracy of log desensitization.

[0006] To achieve the above objectives, a first aspect of an embodiment of the present application provides a log desensitization processing method, the method comprising:

[0007] Obtain desensitized configuration information from the desensitized configuration database;

[0008] In response to a log output request, obtaining a desensitized data type, and obtaining an original business log according to the desensitized data type;

[0009] Performing content type identification on the original business log to obtain the log content type;

[0010] Desensitizing the original business log based on the log content type and the desensitizing configuration information, and outputting a target business log; wherein the target business log includes a business desensitization field;

[0011] The desensitizing configuration information stored in the desensitizing configuration database is updated based on the business desensitizing field.

[0012] In some embodiments, the desensitizing configuration information includes an original desensitizing field; and updating the desensitizing configuration information stored in the desensitizing configuration database based on the service desensitizing field includes:

[0013] Deduplication processing is performed on the business desensitized field based on the original desensitized field to obtain a deduplication desensitized field;

[0014] Integrate the deduplicated desensitized field with the original desensitized field to obtain a target desensitized field;

[0015] The original desensitized field in the desensitized configuration information is updated based on the target desensitized field.

[0016] In some embodiments, before obtaining the desensitizing configuration information from the desensitizing configuration database, the method further includes:

[0017] Responding to the desensitization configuration instruction, obtaining log samples and preset desensitization parameters;

[0018] Perform field recognition on the log sample using a pre-trained desensitized field recognition model to obtain sample desensitized fields;

[0019] Integrate the sample desensitization field with the preset desensitization parameter to obtain the original desensitization parameter;

[0020] Desensitization rules are configured according to the original desensitization parameters to obtain the desensitization configuration information.

[0021] In some embodiments, the log content type includes a character type log, and the original desensitized field includes a character desensitized field; and desensitizing the original business log based on the log content type and the desensitizing configuration information, and outputting the target business log, includes:

[0022] If the log content type indicates that the original business log is a character type log, performing preliminary desensitization on the original business log based on the character desensitization field to obtain first desensitization information;

[0023] Performing model desensitization on the original business log using the desensitized field recognition model to obtain second desensitized information;

[0024] The sensitive information of the original business log is replaced based on the first desensitized information and the second desensitized information to obtain the target business log, and the target business log is output.

[0025] In some embodiments, the log content type further includes an object type log, and the original desensitization field further includes a desensitization filter field; and desensitizing the original business log based on the log content type and the desensitization configuration information and outputting the target business log further includes:

[0026] If the log content type indicates that the original business log is an object type log, attribute filtering is performed on the original business log based on the desensitizing filter field to obtain an initial log;

[0027] Performing character conversion on the initial log to obtain a character log;

[0028] The character log is content-desensitized based on the desensitization filter field to obtain the target business log, and the target business log is output.

[0029] In some embodiments, the performing attribute filtering on the original business log based on the desensitized filtering field to obtain the initial log includes:

[0030] Performing object attribute recognition on the original business log to obtain original object attributes;

[0031] Matching the desensitized filter field with the original object attribute to obtain matching object attributes;

[0032] The matching object attributes in the original business log are filtered to obtain the initial log.

[0033] In some embodiments, after updating the desensitization configuration information stored in the desensitization configuration database based on the service desensitization field, the method further includes:

[0034] For each of the desensitized data types, obtaining the business desensitized fields within a preset time period to obtain a historical desensitized record;

[0035] If the historical desensitization record indicates that no desensitization processing has been performed on each of the original business logs within the preset period, the desensitized data type is confirmed as an exempted data type;

[0036] In response to the log output request, if the desensitized data type is the exempted data type, the original business log is confirmed as an exempted business log, and the exempted business log is output.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a log desensitization processing device, the device comprising:

[0038] A desensitizing configuration information acquisition module is used to obtain desensitizing configuration information from a desensitizing configuration database;

[0039] A business log acquisition module, configured to obtain a desensitized data type in response to a log output request, and obtain an original business log according to the desensitized data type;

[0040] A log content identification module, configured to identify the content type of the original business log and obtain the log content type;

[0041] A log desensitization module, configured to perform desensitization processing on the original business log based on the log content type and the desensitization configuration information, and output a target business log; wherein the target business log includes a business desensitization field;

[0042] A configuration information updating module is used to update the desensitizing configuration information stored in the desensitizing configuration database based on the business desensitizing field.

[0043] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0044] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect.

[0045] The log desensitization processing method and device, electronic device and storage medium proposed in this application obtain desensitization configuration information from a desensitization configuration database to provide a unified standard for subsequent desensitization processing, ensuring the consistency and standardization of desensitization rules in different business scenarios. In response to a log output request, the desensitized data type is obtained, and the corresponding original business log can be obtained according to the desensitized data type, so as to achieve unified interception and desensitization when the log is output, without the need to perform log desensitization in the log generation stage, which can improve the efficiency of log desensitization. Furthermore, the content type of the original business log is identified, which helps to determine the corresponding desensitization method according to different log content types, and the original business log is desensitized based on the desensitization configuration information, which can effectively filter sensitive information, prevent data leakage, and ensure the security of business data. Finally, the desensitization configuration information is updated based on the business desensitization field contained in the target business log, which realizes the dynamic optimization of the desensitization configuration and helps to improve the accuracy of log desensitization. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the log desensitization processing method provided by the embodiment of the present application;

[0047] Figure 2 This is a flow chart of a log desensitization processing method provided by another embodiment of the present application;

[0048] Figure 3 yes Figure 1 A flowchart of step S104 in FIG.

[0049] Figure 4 yes Figure 1 Another flowchart of step S104 in FIG.

[0050] Figure 5 yes Figure 4 Flowchart of step S401 in FIG.

[0051] Figure 6 yes Figure 1 Flowchart of step S105 in FIG.

[0052] Figure 7 This is a flow chart of a log desensitization processing method provided by another embodiment of the present application;

[0053] Figure 8 This is a schematic diagram of the structure of the log desensitization processing device provided in an embodiment of the present application;

[0054] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0056] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0058] First, let’s analyze some of the terms used in this application:

[0059] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0060] Logs are detailed events or operation flows that are automatically generated by computer systems, applications, or network devices during operation and recorded in chronological order. They are used to track state changes, user behavior, error diagnosis, and security audits. The essence of a log is a structured data stream, usually stored in text or binary form, containing key fields such as timestamp, event level (such as INFO, ERROR), source module, and specific description. Based on the source, they can be divided into system logs (such as Linux syslog), application logs (such as web server access logs), and security logs (such as firewall interception records). Efficient log management requires a combination of collection, storage, analysis, and desensitization technologies to balance operational value and privacy protection needs.

[0061] String: A string is a finite sequence of zero or more characters and is a data type used to represent text in programming languages. Strings are generally immutable, meaning their contents cannot be modified directly after creation. Operations such as concatenation and replacement generate new strings. Strings have functions including storing text information, formatting output, and parsing data. They are a core data type for scenarios such as log processing, user input, and network communication.

[0062] Object: An object is an entity that encapsulates data (attributes) and behaviors (methods) in object-oriented programming or data models. In the programming context, an object is the instantiation of a class, allocating memory and storing specific states (such as variable values) and functions (such as functions) through the template defined by the class. In a broader sense, an object can also refer to any identifiable, operable, independent data unit (such as key-value pairs in JSON and records in a database). Its core characteristics include unique identification (such as memory addresses), encapsulation (hiding internal details), interactivity (through message passing or method calls), and polymorphism (different implementations of the same behavior by different objects).

[0063] SLF4J (Simple Logging Facade for Java) is a widely used logging facade framework (logging abstraction layer) in the Java ecosystem. It aims to provide a unified programming interface for different logging systems (such as Logback, Log4j2, JDK Logging, etc.). SLF4J itself does not handle log output. Instead, it delegates log calls to the actual logging library through a bridge (such as slf4j-log4j12) or native bindings (such as slf4j-simple). Compared with directly using a specific logging framework, SLF4J has advantages including decoupling (avoiding code pollution), performance optimization (delaying parameter splicing and reducing invalid logging overhead), and compatibility (supporting the coexistence of multiple logging systems). SLF4J is often used in conjunction with Logback (the default implementation) or Log4j2.

[0064] Log desensitization is a technology that identifies and desensitizes sensitive information in logs, improving data security and compliance. Log desensitization can be applied in multiple scenarios. For example, in financial scenarios, desensitization is required when storing and processing bank transaction logs and insurance product application and claims logs.

[0065] Currently, log masking primarily involves performing masking during the log generation phase. This requires embedding masking logic into the log generation code, creating fixed masking rules. Masking is then performed on the log output content that needs to be processed during the log generation phase. However, if the log scenario or content structure changes, these fixed masking rules may not apply, affecting the accuracy of log masking.

[0066] Based on this, the embodiments of the present application provide a log desensitization processing method and device, an electronic device, and a storage medium, aiming to improve the accuracy of log desensitization.

[0067] The log desensitization processing method and device, electronic device, and storage medium provided in the embodiments of the present application are specifically illustrated by the following embodiments. First, the log desensitization processing method in the embodiments of the present application is described.

[0068] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0069] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0070] The log desensitization processing method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The log desensitization processing method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the log desensitization processing method, etc., but is not limited to the above forms.

[0071] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0072] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0073] Figure 1 This is an optional flow chart of the log desensitization processing method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0074] Step S101, obtaining desensitization configuration information from a desensitization configuration database;

[0075] Step S102: In response to the log output request, obtain the desensitized data type, and obtain the original business log according to the desensitized data type;

[0076] Step S103, identifying the content type of the original business log to obtain the log content type;

[0077] Step S104: desensitizing the original business log based on the log content type and the desensitizing configuration information, and outputting a target business log; wherein the target business log includes a business desensitization field;

[0078] Step S105: updating the desensitizing configuration information stored in the desensitizing configuration database based on the service desensitizing field.

[0079] In the embodiment of the present application, steps S101 to S105 are provided, by obtaining desensitizing configuration information from a desensitizing configuration database, for subsequent desensitization processing, providing a unified standard to ensure the consistency and standardization of desensitization rules under different business scenarios. In response to a log output request, a desensitized data type is obtained, and the corresponding original business log can be obtained according to the desensitized data type, so as to achieve unified interception and desensitization during log output, without the need to perform log desensitization in the log generation stage, which helps to improve the efficiency of log desensitization. Furthermore, the content type of the original business log is identified, which helps to determine the corresponding desensitization method according to different log content types, and the original business log is desensitized based on the desensitization configuration information, which can effectively filter sensitive information, prevent data leakage, and ensure the security of business data. Finally, the desensitization configuration information is updated based on the business desensitization field contained in the target business log, which realizes dynamic optimization of the desensitization configuration and helps to improve the accuracy of log desensitization.

[0080] It is understandable that as the business develops, the desensitization requirements may change. By dynamically adjusting the desensitization configuration information recorded in the desensitization configuration database according to the desensitization results during the log desensitization process, the desensitization configuration information can be updated in a timely manner to adapt to new desensitization requirements, maintain the effectiveness and adaptability of log desensitization processing, and improve the accuracy, efficiency and security of log desensitization work.

[0081] See also Figure 2 In some embodiments, before step S101, the log desensitization processing method may further include but is not limited to steps S201 to S204:

[0082] Step S201, in response to the desensitization configuration instruction, obtaining a log sample and preset desensitization parameters;

[0083] Step S202: Perform field recognition on the log sample using a pre-trained desensitized field recognition model to obtain sample desensitized fields.

[0084] Step S203: integrating the sample desensitization field with the preset desensitization parameters to obtain the original desensitization parameters;

[0085] Step S204: configure desensitization rules according to the original desensitization parameters to obtain desensitization configuration information.

[0086] In the steps S201 to S204 shown in the embodiment of the present application, by responding to the desensitization configuration instruction, obtaining the log sample and the preset desensitization parameters, and using the pre-trained desensitization field recognition model to perform field recognition on the log sample, it is possible to accurately find the fields that need to be desensitized in the log sample, thereby improving the accuracy and pertinence of desensitization. The sample desensitization field is integrated with the preset desensitization parameters to form the original desensitization parameters. Finally, the desensitization rules are configured according to the original desensitization parameters to obtain the desensitization configuration information, so that the desensitization rules can better adapt to different business scenarios and desensitization requirements.

[0087] In step S201 of some embodiments, the desensitization configuration instruction is triggered by a technician, and the desensitization rules, fields, and other information pre-configured by the technician constitute preset desensitization parameters, which include original desensitization fields, regular expressions, and the like.

[0088] Log samples are business logs that are manually screened by technical personnel and do not contain real customers or users. For example, they can be logs generated after executing test cases.

[0089] In some embodiments, log samples can change dynamically according to different business scenarios to adapt to the desensitization requirements of different business scenarios; alternatively, log samples can be composed of logs from multiple business scenarios, which helps to improve the applicability and comprehensiveness of desensitization configuration information.

[0090] Specifically, business scenarios are distinguished by desensitized data types. Desensitized data types are used to indicate which business scenario the business log comes from. For example, in the fintech scenario, business logs come from business scenarios such as bank transaction data, life insurance product data, auto insurance product data, and property insurance product data.

[0091] In step S202 of some embodiments, the desensitized field recognition model is an offline deployed model, which can avoid leakage of sensitive data; and the desensitized field recognition model is pre-trained through sample logs.

[0092] In some embodiments, the desensitized field recognition model can be a large model (such as GPT), a named entity recognition model (such as a NER model), a deep learning model (such as BERT), etc. The specific selection needs to be based on the actual application scenario, but is not limited to this.

[0093] The desensitized field identification model is used to identify sensitive fields in each log sample to obtain multiple original fields. These multiple original fields are then deduplicated and integrated to obtain the sample desensitized fields.

[0094] In one embodiment, the desensitized field recognition model is a large model. When using the desensitized field recognition model to identify sensitive fields, it is necessary to set a prompt word template, and then combine the log sample with the prompt word template to obtain the log content to be parsed.

[0095] Furthermore, the log content to be parsed is input into the desensitized field recognition model, and the sample desensitized field corresponding to the log sample is output.

[0096] For example, the prompt word template can be:

[0097] The input content is: {log sample}. Please detect and mark the sensitive information in the above text.

[0098] The output content can be in the format of: {"Detection result":[{"Type":"Sensitive information type","Original text":"Original sensitive information text","Start position":Start index,"End position":End index},...]}.

[0099] In step S203 of some embodiments, the sample desensitized fields are integrated with the original desensitized fields in the preset desensitization parameters and duplicates are removed to obtain the original desensitized parameters, which include regular expressions, integrated original desensitized fields, and the like.

[0100] In step S204 of some embodiments, desensitization configuration information is generated by configuring desensitization rules for the original desensitization parameters. Specifically, desensitization rule configuration is to convert the original desensitization parameters into configuration information that can be recognized and used by a computer.

[0101] Subsequently, the desensitizing configuration information is stored in the desensitizing configuration database.

[0102] In step S101 of some embodiments, in response to a scheduled desensitization configuration task, desensitization configuration information is obtained from a desensitization configuration database; wherein the scheduled desensitization configuration task is used to instruct the desensitization configuration information to be obtained from the desensitization configuration database at a preset interval. When the desensitization configuration information is updated, the latest desensitization configuration information can be obtained by responding to the scheduled desensitization configuration task to guide log desensitization, thereby improving the accuracy and efficiency of log desensitization.

[0103] Among them, the preset interval time can be set to 30 seconds, 60 seconds, etc., and the specific setting needs to be combined with the actual application scenario, but is not limited to this.

[0104] It should be noted that obtaining the desensitizing configuration information from the desensitizing configuration database is performed periodically, while desensitizing the business logs is performed in real time.

[0105] In step S102 of some embodiments, the log output request is generated by a specific business. For example, in a banking transaction scenario, when a transaction is completed, a transaction record is generated, which corresponds to a transaction log. In order to store the transaction log, a log output request is generated.

[0106] When the transaction log does not need to be desensitized, the transaction log is directly stored in the designated storage space in response to the log output request.

[0107] In the case where the transaction log needs to be desensitized, in response to the log output request, the transaction log is intercepted and desensitized, and then the desensitized transaction log is stored in the designated storage space.

[0108] It should be noted that there can be multiple log output requests, and each log output request can correspond to a different business scenario and instruct the output of multiple business logs. Therefore, it is necessary to obtain the desensitized data type corresponding to the business log indicated by the log output request. The desensitized data type is used to indicate which business scenario the business log originates from, so as to accurately obtain one or more original business logs of the desensitized data type.

[0109] In addition, when desensitizing the original business log, if there are fields in the original business log that need to be desensitized, the original business log is recorded, and the corresponding desensitized data type and the time of the desensitization processing are recorded. By counting whether the original business log under the desensitized data type is desensitized, it is possible to determine whether the log desensitization process needs to be performed later based on the desensitization records corresponding to the desensitized data type after a period of desensitization processing (such as 48 hours later); if the original business log under the desensitized data type is not desensitized during this period of time, the original business log under the desensitized data type is directly output later, which helps to improve the efficiency of log desensitization processing.

[0110] In step S103 of some embodiments, the content type of the original business log is identified to obtain the log content type, wherein the target log content type includes character type log and object type log; for different target log content types, the specific means of desensitization processing will also be different, which will be specifically explained in the following embodiment.

[0111] In some embodiments, the original masked field includes a character masked field and a masked filter field.

[0112] See also Figure 3 In some embodiments, step S104 may include but is not limited to steps S301 to S303:

[0113] Step S301: If the log content type indicates that the original business log is a character type log, the original business log is preliminarily desensitized based on the character desensitization field to obtain first desensitized information;

[0114] Step S302: Desensitizing the original business log using a desensitizing field recognition model to obtain second desensitized information.

[0115] Step S303: Replace sensitive information in the original business log based on the first desensitized information and the second desensitized information to obtain a target business log, and output the target business log.

[0116] In the steps S301 to S303 shown in the embodiment of the present application, when the log content type characterizes that the original business log is a character type log, the original business log is preliminarily desensitized through the character desensitization field to obtain the first desensitized information, which can quickly process the preset sensitive information; further, the original business log is model-desensitized through the desensitized field recognition model to obtain the second desensitized information, which can utilize the recognition ability of the desensitized field recognition model for non-preset sensitive information to dig out sensitive content that may be missed by the preliminary desensitization, and help enhance the comprehensiveness and accuracy of the desensitization of the character type log. Finally, the sensitive information of the original business log is replaced based on the first desensitized information and the second desensitized information to desensitize the sensitive information, obtain the target business log, and output the target business log, which effectively improves the accuracy of the log desensitization, reduces the risk of sensitive data leakage, and ensures the security of the business data.

[0117] In step S301 of some embodiments, the original business log is first traversed, and the sensitive field portion corresponding to the character desensitized field is identified through string matching rules or regular expressions and other technologies; then, the identified sensitive fields are desensitized using specific desensitization rules, such as the middle characters of the name can be replaced by "*", and some numbers of the ID card number and mobile phone number can be retained and the rest replaced by "*"; finally, the first desensitized information is obtained by combining the sensitive field portion and the desensitized field content.

[0118] In step S302 of some embodiments, a desensitized field recognition model is used to perform model desensitization on the original business log, and the generalization ability of the desensitized field recognition model is utilized to identify sensitive fields that have not been desensitized in the initial desensitization process, and perform desensitization processing; then, the sensitive field part and the desensitized field content are combined to obtain second desensitized information.

[0119] In step S303 of some embodiments, first, the first desensitized information and the second desensitized information are deduplicated and integrated to obtain third desensitized information. Specifically, if there are identical desensitized fields in the first desensitized information and the second desensitized information, the identical desensitized fields in the first desensitized information are retained, and the identical desensitized fields in the second desensitized information are removed.

[0120] Furthermore, sensitive information in the original business log is replaced based on the third desensitizing information to obtain a target business log, and the target business log is output to be stored in a designated storage space.

[0121] It is understandable that because the desensitized fields contained in the first desensitized information and the desensitization rules adopted for the desensitization processing are pre-configured by technical personnel, they can better meet the needs of the business scenario, thereby ensuring the matching degree between the business log and the business scenario.

[0122] In other embodiments, if the original business log is preliminarily desensitized based on the character desensitized field, and the first desensitized information obtained does not contain any desensitized information, it is necessary to use a desensitized field recognition model to perform model desensitization on the original business log, which can more accurately desensitize the log and ensure the security and compliance of the business log.

[0123] See also Figure 4 In some embodiments, step S104 may also include but is not limited to steps S401 to S403:

[0124] Step S401: If the log content type indicates that the original business log is an object type log, attribute filtering is performed on the original business log based on the desensitized filter field to obtain an initial log;

[0125] Step S402, performing character conversion on the initial log to obtain a character log;

[0126] Step S403 : Desensitize the character log based on the desensitization filter field to obtain a target business log, and output the target business log.

[0127] In the steps S101 to S106 shown in the embodiment of the present application, when the log content type characterizes the original business log as an object type log, the original business log is subjected to attribute filtering based on the desensitizing filter field to obtain an initial log, which can accurately eliminate the attributes involving sensitive information, reduce the amount of data for subsequent processing, and improve processing efficiency. Further, the initial log is subjected to character conversion to obtain a character log, so that the log data is unified into a character format, which is convenient for subsequent desensitization processing. Finally, the character log is subjected to content desensitization based on the desensitizing filter field to obtain a target business log, and the target business log is output, thereby improving the accuracy and efficiency of desensitizing the object type log.

[0128] See also Figure 5 In some embodiments, step S401 may also include but is not limited to steps S501 to S503:

[0129] Step S501: performing object attribute recognition on the original business log to obtain original object attributes;

[0130] Step S502: Match the desensitized filter field with the original object attributes to obtain matching object attributes;

[0131] Step S503: Filter the matching object attributes in the original business log to obtain an initial log.

[0132] Steps S501 to S503 shown in the embodiment of the present application can accurately locate the attributes of various objects in the original business log by identifying the object attributes of the original business log. Then, based on the desensitized filter field, the original object attributes are matched to obtain the matching object attributes, which can effectively filter out the object attributes that need to be filtered. Finally, the matching object attributes in the original business log are filtered to obtain the initial log, which can quickly filter out sensitive object attributes and improve the efficiency of log desensitization processing.

[0133] In step S501 of some embodiments, original object attributes are obtained by identifying all attributes in the original service log.

[0134] In step S502 of some embodiments, the original object attributes are matched based on the desensitized filter field to find attributes that are identical between the desensitized filter field and the original object attributes, thereby obtaining matching object attributes. Specifically, the matching method can be hash table search, binary search, traversal search, etc., and the specific selection should be based on the actual application scenario, but is not limited to these.

[0135] In step S503 of some embodiments, the matching object attributes in the original business log are filtered, so that the attributes that need to be desensitized can be quickly screened out to obtain the initial log.

[0136] In step S403 of some embodiments, the initial log is first traversed, and the sensitive field portion corresponding to the desensitizing filter field is identified through string matching rules or regular expressions and other techniques; then, the identified sensitive field is desensitized using specific desensitizing rules, and the sensitive field portion is replaced with the desensitized field content to obtain the target business log, and the target business log is output to be stored in the designated storage space.

[0137] In some embodiments, the target log content type also includes a table type log.

[0138] The log desensitization processing method may also include but is not limited to the following steps:

[0139] If the log content type indicates that the original business log is a table type log, the original business log is desensitized using the desensitized field recognition model to obtain table desensitized information;

[0140] The sensitive information in the original business log is replaced based on the table desensitization information to obtain the target business log, and the target business log is output.

[0141] Specifically, by leveraging the desensitizing field recognition model's ability to identify non-preset sensitive information, sensitive information in table-type logs can be quickly identified, helping to enhance the comprehensiveness and accuracy of desensitizing table-type logs.

[0142] In one embodiment, the Slf4jLogger interface is used for log output, wherein the method attribute startTime is defined to record the start time of log printing, the attribute needHandleLogger is defined to record the original business log that needs to be processed, and the method handle(String str) is defined to enhance logger.info(String format), and handle(String str,Object...arguments) is defined to enhance logger.info(format,Object...arguments).

[0143] First, the handle(String str) method is used for processing: the isJsonObject(str) method is called to identify the content type of the original business log and obtain the log content type.

[0144] If the log content type indicates that the original business log is a character-type log, perform preliminary desensitization on the original business log based on the character desensitization field to obtain first desensitized information. Perform model desensitization on the original business log using the desensitization field recognition model to obtain second desensitized information. Replace sensitive information in the original business log based on the first and second desensitized information to obtain the target business log, and output the target business log; at the same time, record the startTime.

[0145] If the log content type indicates that the original business log is an object-type log, call the handle(String str,Object…arguments) method to process it. The CustomerPropertyFilter method is called to filter the original business log properties based on the desensitizing filter field to obtain the initial log. The JSON.toJsonString method is called to convert the initial log to character strings to obtain a character log. Finally, the character log is desensitized based on the desensitizing filter field to obtain the target business log and output it. The start time is also recorded.

[0146] The start time, desensitized data type, and name of the original business log after desensitization are recorded in needHandleLogger.

[0147] In some embodiments, the target business log includes a business desensitization field, which is a field that has been desensitized.

[0148] See also Figure 6 In some embodiments, step S105 includes but is not limited to steps S601 to S603:

[0149] Step S601: Deduplication processing is performed on the business desensitized field based on the original desensitized field to obtain a deduplication desensitized field;

[0150] Step S602: integrating the deduplicated desensitized field with the original desensitized field to obtain a target desensitized field.

[0151] Step S603: Update the original desensitized field in the desensitization configuration information based on the target desensitized field.

[0152] In steps S601 to S603 shown in the embodiment of the present application, by deduplicating the business desensitized field based on the original desensitized field, a deduplicated desensitized field is obtained, which can effectively remove the repeated fields existing in the business desensitized field. Then, the deduplicated desensitized field is integrated with the original desensitized field to obtain the target desensitized field. Finally, the original desensitized field in the desensitizing configuration information is updated based on the target desensitizing field, so as to realize dynamic adjustment of the desensitizing configuration information recorded in the desensitizing configuration database according to the desensitization result in the log desensitization process, timely update the desensitizing configuration information to adapt to the new desensitization requirements, maintain the effectiveness and adaptability of the desensitization process, and improve the accuracy, efficiency and security of the log desensitization work as a whole.

[0153] First, the business desensitized field is deduplicated based on the original desensitized field. If there are desensitized fields in the business desensitized field that are identical to the original desensitized field, the identical desensitized fields in the business desensitized field are removed to obtain the deduplicated desensitized field.

[0154] Furthermore, the deduplicated desensitized fields are integrated with the original desensitized fields to obtain the target desensitized fields, which can ensure that there are no duplicate desensitized fields in the target desensitized fields.

[0155] Finally, the original desensitized field in the desensitization configuration information is replaced with the target desensitized field, thereby achieving field update.

[0156] It is understandable that the updated desensitizing configuration information in the desensitizing configuration database will be obtained and used to guide log desensitization during the next scheduled desensitization configuration task, thereby dynamically adjusting the desensitizing configuration information according to changes in business scenarios, which helps to improve the accuracy and efficiency of log desensitization processing.

[0157] See also Figure 7 In some embodiments, after step S105, the log desensitization processing method may further include but is not limited to steps S701 to S703:

[0158] Step S701: For each desensitized data type, obtain the business desensitized fields within a preset period to obtain a historical desensitized record;

[0159] Step S702: If the historical desensitization record indicates that no desensitization process has been performed on each original business log within the preset period, the desensitized data type is determined as an exempted data type;

[0160] Step S703 , in response to the log output request, if the desensitized data type is an exempted data type, the original business log is confirmed as an exempted business log, and the exempted business log is output.

[0161] In the steps S701 to S703 shown in the embodiment of the present application, for each desensitized data type, the business desensitization field within the preset time period is obtained to form a historical desensitization record, so as to clearly understand the processing of each desensitized data type within the preset time period. Then, based on the historical desensitization record, it is judged whether the original business log within the preset time period is desensitized. If the original business log is not desensitized, the corresponding desensitized data type is confirmed as an exempted data type, thereby accurately identifying the desensitized data type that does not require desensitization, avoiding the desensitization operation on the business log that does not require desensitization, and saving system resources. Finally, in response to the log output request, if the desensitized data type is an exempted data type, the original business log is confirmed as an exempted business log, and the exempted business log is output, ensuring that the exempted business log is directly output under the premise of business log security and compliance, thereby improving the flexibility and rationality of log desensitization processing, as well as the efficiency of log desensitization processing.

[0162] In step S701 of some embodiments, the preset time period needs to be set based on actual scenarios, such as 48 hours, 72 hours, etc. For each desensitized data type, the business desensitization field of each target business log within the preset time period before the current moment is obtained to form a historical desensitization record corresponding to each desensitized data type.

[0163] Specifically, if the historical desensitization record is empty, it indicates that no desensitization processing has been performed on each original business log within the preset period, that is, no desensitization processing is required, and the desensitized data type is confirmed as an exempted data type.

[0164] It should be noted that the exempted data type means that the business log under the exempted data type does not contain sensitive information and does not need to be desensitized. Therefore, it can be output directly to improve the efficiency of log desensitization processing.

[0165] After the current moment, in response to a log output request, the desensitized data type is obtained. If the desensitized data type is an exempted data type, the original business log is confirmed as an exempted business log and the exempted business log is output.

[0166] The log desensitization processing method provided in the embodiment of the present application can be quickly configured in various business scenarios in the financial technology scenario.

[0167] First, the log desensitization configuration information can be flexibly configured, and the desensitization configuration information recorded in the desensitization configuration database can be dynamically adjusted according to the desensitization results during the log desensitization process to maintain the effectiveness and adaptability of the log desensitization processing.

[0168] Secondly, there is no need to modify the log generation code, and there is no need to worry about log desensitization during business development. You only need to call the log desensitization processing method when outputting business logs, which helps improve the efficiency of log desensitization.

[0169] Furthermore, for desensitized data types that do not require desensitization processing, they can be directly output after running for a period of time, which helps to improve the flexibility and rationality of log desensitization processing, as well as the efficiency of log desensitization processing.

[0170] See also Figure 8 The embodiment of the present application further provides a log desensitization processing device, which can implement the above-mentioned log desensitization processing method, and the log desensitization processing device includes:

[0171] The desensitization configuration information acquisition module 801 is used to obtain the desensitization configuration information from the desensitization configuration database;

[0172] The business log acquisition module 802 is used to obtain the desensitized data type in response to the log output request, and obtain the original business log according to the desensitized data type;

[0173] The log content identification module 803 is used to identify the content type of the original business log and obtain the log content type;

[0174] The log desensitization module 804 is used to desensitize the original business log based on the log content type and desensitization configuration information, and output a target business log; wherein the target business log includes a business desensitization field;

[0175] The configuration information updating module 805 is used to update the desensitizing configuration information stored in the desensitizing configuration database based on the business desensitizing field.

[0176] The specific implementation of the log desensitization processing device is basically the same as the specific embodiment of the above-mentioned log desensitization processing method, and will not be repeated here.

[0177] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the log desensitization processing method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, an in-vehicle computer, or the like.

[0178] See also Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0179] The processor 901 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0180] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the log desensitization processing method of the embodiment of this application;

[0181] Input / output interface 903, used to implement information input and output;

[0182] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0183] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0184] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0185] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned log desensitization processing method.

[0186] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0187] The log desensitization processing method and device, electronic device and storage medium provided in the embodiments of the present application provide a unified standard for subsequent desensitization processing by obtaining desensitization configuration information from a desensitization configuration database, thereby ensuring the consistency and standardization of desensitization rules in different business scenarios. In response to a log output request, a desensitized data type is obtained, and the corresponding original business log can be obtained according to the desensitized data type, so as to achieve unified interception and desensitization when the log is output, without the need to perform log desensitization in the log generation stage, which can improve the efficiency of log desensitization. Furthermore, the content type of the original business log is identified, which helps to determine the corresponding desensitization method according to different log content types, and the original business log is desensitized based on the desensitization configuration information, which can effectively filter sensitive information, prevent data leakage, and ensure the security of business data. Finally, the desensitization configuration information is updated based on the business desensitization field contained in the target business log, thereby achieving dynamic optimization of the desensitization configuration and helping to improve the accuracy of log desensitization.

[0188] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0189] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0190] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0191] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0192] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0193] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0195] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0197] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0198] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A log desensitization processing method, characterized in that: The method comprises: Obtain desensitized configuration information from the desensitized configuration database; In response to a log output request, obtaining a desensitized data type, and obtaining an original business log according to the desensitized data type; Performing content type identification on the original business log to obtain the log content type; Desensitizing the original business log based on the log content type and the desensitizing configuration information, and outputting a target business log; wherein the target business log includes a business desensitization field; The desensitizing configuration information stored in the desensitizing configuration database is updated based on the business desensitizing field.

2. The method according to claim 1, characterized in that The desensitizing configuration information includes an original desensitizing field; and updating the desensitizing configuration information stored in the desensitizing configuration database based on the service desensitizing field includes: Deduplication processing is performed on the business desensitized field based on the original desensitized field to obtain a deduplication desensitized field; Integrate the deduplicated desensitized field with the original desensitized field to obtain a target desensitized field; The original desensitized field in the desensitized configuration information is updated based on the target desensitized field.

3. The method according to claim 2, characterized in that Before acquiring the desensitizing configuration information from the desensitizing configuration database, the method further includes: Responding to the desensitization configuration instruction, obtaining log samples and preset desensitization parameters; Perform field recognition on the log sample using a pre-trained desensitized field recognition model to obtain sample desensitized fields; Integrate the sample desensitization field with the preset desensitization parameter to obtain the original desensitization parameter; Desensitization rules are configured according to the original desensitization parameters to obtain the desensitization configuration information.

4. The method according to claim 3, characterized in that The log content type includes a character type log, and the original desensitized field includes a character desensitized field; and desensitizing the original business log based on the log content type and the desensitizing configuration information, and outputting a target business log, including: If the log content type indicates that the original business log is a character type log, performing preliminary desensitization on the original business log based on the character desensitization field to obtain first desensitization information; Performing model desensitization on the original business log using the desensitized field recognition model to obtain second desensitized information; The sensitive information of the original business log is replaced based on the first desensitized information and the second desensitized information to obtain the target business log, and the target business log is output.

5. The method according to claim 4, characterized in that The log content type further includes an object type log, and the original desensitization field further includes a desensitization filter field; performing desensitization processing on the original business log based on the log content type and the desensitization configuration information and outputting the target business log further includes: If the log content type indicates that the original business log is an object type log, attribute filtering is performed on the original business log based on the desensitizing filter field to obtain an initial log; Performing character conversion on the initial log to obtain a character log; The character log is content-desensitized based on the desensitization filter field to obtain the target business log, and the target business log is output.

6. The method according to claim 5, characterized in that The attribute filtering of the original business log based on the desensitized filtering field to obtain the initial log includes: Performing object attribute recognition on the original business log to obtain original object attributes; Matching the desensitized filter field with the original object attribute to obtain matching object attributes; The matching object attributes in the original business log are filtered to obtain the initial log.

7. The method according to any one of claims 1 to 6, characterized in that After updating the desensitizing configuration information stored in the desensitizing configuration database based on the service desensitizing field, the method further includes: For each of the desensitized data types, obtaining the business desensitized fields within a preset time period to obtain a historical desensitized record; If the historical desensitization record indicates that no desensitization processing has been performed on each of the original business logs within the preset period, the desensitized data type is confirmed as an exempted data type; In response to the log output request, if the desensitized data type is the exempted data type, the original business log is confirmed as an exempted business log, and the exempted business log is output.

8. A log desensitization processing device, characterized in that: The device comprises: A desensitizing configuration information acquisition module is used to obtain desensitizing configuration information from a desensitizing configuration database; A business log acquisition module, configured to obtain a desensitized data type in response to a log output request, and obtain an original business log according to the desensitized data type; A log content identification module, configured to identify the content type of the original business log and obtain the log content type; A log desensitization module, configured to perform desensitization processing on the original business log based on the log content type and the desensitization configuration information, and output a target business log; wherein the target business log includes a business desensitization field; A configuration information updating module is used to update the desensitizing configuration information stored in the desensitizing configuration database based on the business desensitizing field.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images