Log data desensitization method, system, electronic device and storage medium

Abstract

The present invention discloses a log data desensitization method, system, electronic device and storage medium, including executing log printing on an object; judging whether a desensitization annotation exists in a field printed by the log, and performing desensitization processing if exists; if not, judging whether to perform desensitization processing based on a configuration file; if the configuration file does not match a corresponding configuration, desensitization processing is not performed, and the original log is directly output; if the configuration file matches a corresponding configuration, desensitization processing is performed on sensitive information in the log shown according to the read configuration attribute. The present invention chooses to turn on or off the log desensitization according to the actual situation of the customer, thereby satisfying flexible system configuration.

Description

Technical Field

[0001] The present invention relates to the field of log desensitization technology, and in particular to a log data desensitization method, system, electronic device and storage medium. Background Art

[0002] In modern banking and financial services, it is inevitable to store and use some private information of customers. This private information may include: personal ID number, bank card number, mobile phone number, home address, etc. During the development, use, and operation and maintenance of the system, this information will inevitably be exposed to the development and operation personnel. If this information falls into the hands of people with ulterior motives, it may cause unpredictable losses and damages.

[0003] Existing solutions often use a very direct string replacement method to implement, that is, after receiving user information or message information, part of the information is replaced with the * character. For example, the phone number 13043212315 will be replaced with 13*******15 after desensitization processing.

[0004] The means of use provided by the existing technology are often executed through a function or a method, which is called by the developer in the business code to complete the replacement.

[0005] In the existing banking and financial services, business entity objects have the following characteristics:

[0006] There are many fields in the object that need to be desensitized;

[0007] The rules for each field to be desensitized are inconsistent;

[0008] The existing business code has already reached a certain scale. If it is to be modified, there are too many places to be modified.

[0009] The technical literacy of developers varies greatly, and even if specifications are established, errors are still easy to occur.

[0010] If the desensitization rules change, a lot of code will need to be modified.

[0011] Different environments require different desensitization strategies. For example, the development environment uses test data, which does not pose a risk of data leakage. To facilitate development and debugging, data desensitization is often not required. However, the production environment needs to conceal the user's real information.

[0012] In the case of microservices, a technical support service is needed to perform protective transformation from an overall perspective. In actual development, the existing problems may not be limited to the above business scenarios. To overcome the above problems, it is necessary to implement a general technical support component from the perspective of the overall architecture: the relevant desensitization configuration can be customized, and the technical team leader responsible for microservices can implement the desensitization configuration work for different fields and different objects through simple operations. At the same time, it reduces the difficulty of development, allowing developers to complete data desensitization without perception, thereby reducing the cost of learning and use. Summary of the invention

[0013] In view of the shortcomings existing in the above problems, the present invention provides a log data desensitization method, system, electronic device and storage medium.

[0014] To achieve the above object, the present invention provides a log data desensitization method, comprising:

[0015] Perform log printing on the object;

[0016] Determine whether there are desensitization annotations in the fields printed by the log, and perform desensitization processing if so;

[0017] If it does not exist, determine whether to perform desensitization based on the configuration file;

[0018] If the configuration file does not match the corresponding configuration, no desensitization is performed and the original log is directly output;

[0019] If the configuration file matches the corresponding configuration, the sensitive information in the displayed log is desensitized according to the read configuration attributes.

[0020] Preferably, desensitization rules are stored in the configuration file.

[0021] Preferably, the desensitization rule configuration method includes adding desensitization rules to the object or using a YAML file to configure the rules.

[0022] Preferably, adding a desensitization rule to an object includes:

[0023] The desensitization annotation @SensitiveChineseName is introduced in the code, which is used to identify the fields that need to be desensitized;

[0024] Define an entity class named UserEntity, which implements the Serializable interface, indicating that objects of the entity class are serialized;

[0025] In the entity class, the field userNamePattern uses the @SensitiveChineseName annotation;

[0026] Define desensitization rules through @SensitiveInfo annotation;

[0027] Desensitization processing is performed based on the desensitization rules.

[0028] Preferably, defining the desensitization rules includes:

[0029] strategy = SensitivechineseName.class: specifies the desensitization strategy class as SensitivechineseName;

[0030] begin=1:indicates that desensitization starts from the first character of the string;

[0031] end=1:Indicates that desensitization ends at the first character;

[0032] replaceChar="*": replaces the de-sensitized part with an asterisk *.

[0033] This application also provides a log data desensitization system, including:

[0034] Printing module, used to print logs for objects;

[0035] The judgment module is used to judge whether there is a desensitization annotation in the field printed by the log. If so, desensitization processing is performed; if not, whether to perform desensitization processing is determined based on the configuration file;

[0036] The matching module is used to directly output the original log without performing desensitization processing if the configuration file does not match the corresponding configuration; if the configuration file matches the corresponding configuration, desensitization processing is performed on the sensitive information in the displayed log according to the read configuration attributes.

[0037] The present invention also provides an electronic device, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above method.

[0038] The present invention also provides a storage medium storing a computer program executable by an electronic device, and when the program runs on the electronic device, the electronic device executes the above method.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention chooses to turn on or off the log desensitization according to the actual situation of the customer, thereby meeting flexible system configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the log data desensitization method of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Reference Figure 1 The present invention provides a log data desensitization method, comprising:

[0044] Perform log printing on the object;

[0045] Determine whether there are desensitization annotations in the fields printed by the log, and perform desensitization processing if so;

[0046] If it does not exist, determine whether to perform desensitization based on the configuration file;

[0047] If the configuration file does not match the corresponding configuration, no desensitization is performed and the original log is directly output;

[0048] If the configuration file matches the corresponding configuration, the sensitive information in the displayed log is desensitized according to the read configuration attributes.

[0049] In this embodiment, the log desensitization execution process is as follows:

[0050] Execute log printing on an object: This is the starting point of the process, which means executing a log printing operation on an object.

[0051] Determine whether there is a desensitization annotation on the field: Check whether the field printed in the log is marked as requiring desensitization.

[0052] If there is no desensitization annotation on the field, proceed to the next step.

[0053] If there is a desensitization annotation on the field, the desensitization process will continue.

[0054] Query the filter configuration in YAML: If there is no desensitization annotation, the system will query the filter configuration in the configuration file (such as YAML format) to determine whether desensitization is required.

[0055] Determine whether the desensitization configuration matches: Based on the queried configuration, determine whether the field needs to be desensitized.

[0056] If the desensitization configuration matches, the desensitization process continues.

[0057] If no corresponding configuration is matched, no desensitization is performed and the original log is directly output.

[0058] Read configuration properties of the configuration file: If the desensitization configuration matches, the system will read the specific properties about desensitization in the configuration file, such as desensitization rules, format, etc.

[0059] Desensitize according to configuration: Desensitize sensitive information in the log according to the read configuration attributes.

[0060] Print output: After the desensitization process is completed, the processed log will be printed out.

[0061] Specifically, this application uses a component approach to call the desensitization function. That is, the desensitization component is independent of each business component and is integrated into each business service in the form of an SDK. At the same time, the desensitization component integrates the commonly used logback log component in the system, so that each service can automatically obtain the technical capability of log desensitization without making project adjustments.

[0062] Furthermore, the component provides a built-in interface function, which can be injected into the microservice using the injection function of the Spring framework, and the capability can be used by explicit calls. In this way, developers can use log desensitization precisely where the service needs it. This method is suitable for business scenarios where it is clear that the data needs to be protected and desensitization operations are performed.

[0063] At the same time, in order to facilitate the use and operation of developers, the component also provides implicit calls. This call is combined with the Logback log component, and a log interceptor is added by defining an appender. For all logs to be printed, the desensitized data can be directly output, thus realizing non-perception calls and facilitating the use of developers.

[0064] The following is how to integrate the components, that is, just integrate the following parts into the pom file.

[0065] <dependency>

[0066] <groupid> com.dcits.bifang< / groupid>

[0067] <artifactid> bifang-log-springboot-starter< / artifactid>

[0068] <version> {version.lastVersion}< / version>

[0069] <! --Use the latest version of Bifang component library-->

[0070] < / dependency>

[0071] In this embodiment, through in-depth investigation and research on the business and collection of user usage habits, the component provides a variety of preset desensitization strategy scenarios, which basically cover more than 90% of business scenarios. Developers can use them directly without having to set them up by themselves.

[0072] At the same time, through good naming standards, developers can understand the meaning of the names, thereby reducing cognitive ambiguity that is easy to occur during the development process, and reducing development learning costs and usage costs.

[0073] The annotations provided by the component by default currently include the following strategies, among which the desensitization strategy is set according to the existing industry practices. Developers can use configuration method A(2) according to actual conditions, or use a custom method to set it separately:

[0074] @SensitiveAddress: Address strategy, by default, retains the first five characters and the last four characters;

[0075] @SensitiveBankCard: Bank card policy, all codes are coded by default;

[0076] @SensitiveChineseName: Chinese name strategy, retain the surname by default and censor the name;

[0077] @SensitiveEmail: Email policy, by default, the first two characters and the email address @ suffix are retained, and the rest are censored;

[0078] @SensitiveFixedPhone: The landline number desensitization strategy retains the last four digits by default and the rest are all coded;

[0079] @SensitiveIdCard: Identity card number strategy, by default, the last four digits are retained and the rest are coded;

[0080] @SensitiveMobile: Mobile phone number policy, by default, retains the first three and last four digits, and the rest are coded;

[0081] @SensitivePassword: Password policy: All passwords are encrypted by default.

[0082] In this embodiment, due to the complexity of the business, the currently preset desensitization strategy may not cover all business scenarios. The component provides a very free expansion method, such as expansion by implementing interfaces, by defining regular functions, or by customizing desensitization annotations. The following shows the two most common expansion methods.

[0083] Extension method A: Implementing the IStrategy interface

[0084] The component provides an interface definition named IStrategy. Users only need to implement the method defined in the interface: desensitization(String source, int begin, int end, String replaceChar). After the system is restarted, it will automatically match the corresponding desensitization scheme and processing content that meet the requirements according to the defined method.

[0085] Extension method B: Custom regular expressions

[0086] The component provides the annotation: @SensitiveInfo. Users only need to implement the required configuration items of the annotation: pattern (content to be matched, using regular expressions) and replaceChar (what characters to use to replace when the component matches data content that meets the requirements).

[0087] After the expansion is completed, the configured desensitization policy can be used normally according to the previous usage links. This solution can basically cover more than 99% of the business scenarios in our daily business operations, and can be appropriately extended to other business areas, such as interface data desensitization.

[0088] In this embodiment, by default, the component recommends using annotations to formulate desensitization strategies, because this is the most flexible solution. However, some projects are for organizational asset sedimentation, and it is not convenient to add and process global fields. There will be more code modification costs, or source code modification cannot be performed.

[0089] The component provides a solution for configuration through configuration files, that is, in the application.yml file that comes with the Spring framework, a universal solution can be implemented through configuration, making it applicable to more business scenarios and development scenarios.

[0090] Configuration method A: Add desensitization rules to objects

[0091] Introducing desensitization annotations: First, a desensitization annotation @SensitiveChineseName is introduced in the code. This annotation is used to identify fields that need to be desensitized.

[0092] Define entity class: Define an entity class named UserEntity, which implements the Serializable interface, indicating that objects of this class can be serialized.

[0093] Use the default desensitization rule: In the UserEntity class, there is a field userNamePattern, which uses the @SensitiveChineseName annotation. This means that the default Chinese name desensitization rule will be applied to this field when logging or other output is performed.

[0094] (Optional) Custom desensitization rules: If you need to use custom desensitization rules, you can use the @SensitiveInfo annotation to implement it. In this annotation, you can specify the class of the desensitization strategy (strategy attribute) and the specific parameters of desensitization, such as the start position (begin), end position (end), and replacement character (replaceChar).

[0095] strategy=SensitivechineseName.class: specifies the desensitization strategy class as SensitivechineseName.

[0096] begin=1:Indicates that desensitization starts from the first character of the string.

[0097] end=1:Indicates that the desensitization ends at the first character. There may be a typo here. Usually, end should be greater than begin, or this setting means that only the first character is desensitized.

[0098] replaceChar="*": replaces the de-sensitized part with an asterisk *.

[0099] Desensitization process: When the log system or other systems need to print or display the userNamePattern field of the UserEntity object, they will decide whether to perform desensitization and how to perform desensitization based on the annotations on the field.

[0100] If no custom desensitization rules are used, the default Chinese name desensitization rules will be used.

[0101] If there are custom desensitization rules, desensitization will be performed according to the strategy and parameters specified in the @SensitiveInfo annotation.

[0102] Output after desensitization: After the desensitization process is completed, the sensitive information will be replaced with asterisks or other specified characters, and then the processed results will be output.

[0103] Configuration method B: Use YAML file to configure rules

[0104] Add the categories attribute in the application.yml file:

[0105] sensitive:

[0106] log-info:

[0107] # Enable log desensitization

[0108] enable:true

[0109] categories:

[0110] #Fields named name and chineseName will only retain the first and last characters

[0111] -keywords:name,chineseName

[0112] pre-length:1

[0113] suf-length:1

[0114] -keywords:mail,email

[0115] pre-length:3

[0116] suf-length:3

[0117] #Fields named mobile and phone will only retain the first three characters and the last four characters

[0118] -keywords:mobile,phone

[0119] pre-length:3

[0120] suf-length:4

[0121] -keywords:address,addr

[0122] pre-length:3

[0123] suf-length:4

[0124] In this embodiment, the technologies used are all open source and popular technical components on the market, which are widely applicable to Java projects using the Spring framework, and there is basically no threshold for use. At present, the components rely on the Logback log component by default, but they can also be easily replaced with common log components such as Log4j2. This technology basically does not use additional resources during operation, performs very well under performance testing, and does not take up too many system resources.

[0125] This application also provides a log data desensitization system, including:

[0126] Printing module, used to print logs for objects;

[0127] The judgment module is used to judge whether there is a desensitization annotation in the field printed by the log. If so, desensitization processing is performed; if not, whether to perform desensitization processing is determined based on the configuration file;

[0128] The matching module is used to directly output the original log without performing desensitization processing if the configuration file does not match the corresponding configuration; if the configuration file matches the corresponding configuration, the sensitive information in the displayed log is desensitized according to the read configuration attributes.

[0129] The present invention also provides an electronic device, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above method.

[0130] The present invention also provides a storage medium storing a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the above method.

[0131] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A log data desensitization method, characterized in that: include: Perform log printing on the object; Determine whether there are desensitization annotations in the fields printed by the log, and perform desensitization processing if so; If it does not exist, determine whether to perform desensitization based on the configuration file; If the configuration file does not match the corresponding configuration, no desensitization is performed and the original log is directly output; If the configuration file matches the corresponding configuration, the sensitive information in the displayed log is desensitized according to the read configuration attributes.

2. The log data desensitization method according to claim 1, characterized in that: The configuration file shown has desensitization rules stored in it.

3. The log data desensitization method according to claim 2, characterized in that: The desensitization rule configuration method includes adding desensitization rules to the object or using a YAML file to configure the rules.

4. The log data desensitization method according to claim 3, characterized in that: Adding desensitization rules to an object includes: The desensitization annotation @SensitiveChineseName is introduced in the code, which is used to identify the fields that need to be desensitized; Define an entity class named UserEntity, which implements the Serializable interface, indicating that objects of the entity class are serialized; In the entity class, the field userNamePattern is annotated with @SensitiveChineseName; Define desensitization rules through @SensitiveInfo annotation; Desensitization processing is performed based on the desensitization rules.

5. The log data desensitization method according to claim 4, characterized in that: Defining desensitization rules includes: strategy = SensitivechineseName.class: specifies the desensitization strategy class as SensitivechineseName; begin=1:indicates that desensitization starts from the first character of the string; end=1:Indicates that desensitization ends at the first character; replaceChar="*": replaces the de-sensitized part with an asterisk *.

6. A log data desensitization system, characterized in that: include: Printing module, used to print logs for objects; The judgment module is used to judge whether there is a desensitization annotation in the field printed by the log, and if so, perform desensitization processing; If it does not exist, determine whether to perform desensitization based on the configuration file; The matching module is used to directly output the original log without performing desensitization processing if the configuration file does not match the corresponding configuration; if the configuration file matches the corresponding configuration, desensitization processing is performed on the sensitive information in the displayed log according to the read configuration attributes.

7. An electronic device, characterized in that: The method comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: It stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the method described in any one of claims 1 to 5.

Images