Data desensitization method, device, electronic device and readable storage medium

Abstract

The present application provides a data desensitization method, device, electronic device and readable storage medium, which relates to the field of data processing technology. The method includes: receiving query information of a user's query initial data, the query information includes the first permission information of the user; when the initial data is a first type of data, generating a structured query language SQL statement based on a target database storing the initial data, the first type of data is data that has not been separated from the service; parsing the SQL statement to obtain second permission information, the second permission information is used to characterize the permission required for data query on the target database; when the first permission information matches the second permission information, adding the initial data to a message queue; desensitizing the initial data in the message queue to obtain the target data. The present application can improve the effect of data desensitization.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data desensitization method, device, electronic device and readable storage medium. Background Art

[0002] Data security is an important part of the data storage and data transmission process. In order to achieve data security, data desensitization is usually adopted in related technologies. In related technologies, before desensitizing the data, it is necessary to determine the data that needs to be desensitized through manual screening, and then process the data in batches. However, in actual application, some data cannot be separated from the service, and the content of this part of the data changes dynamically with the service. It is difficult for related technologies to determine whether desensitization is needed for dynamically changing data, resulting in poor data desensitization effect.

[0003] It can be seen that the relevant technology has the problem of poor data desensitization effect. Summary of the invention

[0004] The embodiments of the present application provide a data desensitization method, device, electronic device and readable storage medium to solve the problem of poor data desensitization effect in the related art.

[0005] To solve the above problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a data desensitization method, comprising:

[0007] Receive query information of a user for querying initial data, wherein the query information includes first authority information of the user;

[0008] In the case where the initial data is first type data, generating a structured query language SQL statement based on a target database storing the initial data, the first type data being data that has not been separated from the service;

[0009] Parsing the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database;

[0010] When the first permission information matches the second permission information, adding the initial data to a message queue;

[0011] The initial data in the message queue is desensitized to obtain target data.

[0012] In a second aspect, an embodiment of the present application further provides a data desensitization device, including:

[0013] A receiving module, configured to receive query information of a user for querying initial data, wherein the query information includes first authority information of the user;

[0014] A generating module, configured to generate a structured query language SQL statement based on a target database storing the initial data when the initial data is first type data, the first type data being data that has not been separated from the service;

[0015] A parsing module, used to parse the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database;

[0016] an adding module, configured to add the initial data to a message queue when the first permission information matches the second permission information;

[0017] The first desensitizing module is used to perform desensitization processing on the initial data in the message queue to obtain target data.

[0018] In a third aspect, an embodiment of the present application further provides an electronic device, including a transceiver and a processor.

[0019] The transceiver is used to receive query information of a user for querying initial data, wherein the query information includes the first authority information of the user;

[0020] The processor is configured to generate a structured query language SQL statement based on a target database storing the initial data when the initial data is first type data, the first type data being data that has not been separated from the service;

[0021] The processor is further used to parse the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database;

[0022] The processor is further configured to add the initial data to a message queue when the first permission information matches the second permission information;

[0023] The processor is also used to perform desensitization processing on the initial data in the message queue to obtain target data.

[0024] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the data desensitization method described in the first aspect above are implemented.

[0025] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data desensitization method described in the first aspect above are implemented.

[0026] In a sixth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the data desensitization method as described in the first aspect above.

[0027] In an embodiment of the present application, a user's query information for querying initial data is received, and the query information includes the user's first permission information; when the initial data is a first type of data, a structured query language SQL statement is generated based on the target database storing the initial data, and the first type of data is data that has not been separated from the service; the SQL statement is parsed to obtain second permission information, and the second permission information is used to characterize the permission required for data query on the target database; when the first permission information matches the second permission information, the initial data is added to the message queue; the initial data in the message queue is desensitized to obtain the target data. In this way, by generating an SQL statement, parsing the SQL statement to determine the second permission information for querying the initial data, and adding the initial data to the message queue and desensitizing it when the first permission information and the second permission information match, the desensitization of the initial data is achieved when the initial data is not separated from the service and changes dynamically, thereby improving the effect of data desensitization. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0029] Figure 1 It is a flow chart of a data desensitization method provided in an embodiment of the present application;

[0030] Figure 2 It is a schematic diagram of a module for data desensitization provided in an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of a desensitization engine module provided in an embodiment of the present application;

[0032] Figure 4 is a schematic diagram of an automatic grading recommendation module provided in an embodiment of the present application;

[0033] Figure 5 It is a structural diagram of a data desensitization device provided in an embodiment of the present application;

[0034] Figure 6 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] See also Figure 1 , Figure 1 is a flow chart of a data desensitization method provided in an embodiment of the present application, such as Figure 1 As shown, the following steps are included:

[0037] Step 101: Receive query information of a user for querying initial data, wherein the query information includes first authority information of the user.

[0038] The above query information may be query information generated after the user selects initial data in the result data table of the service business. The above initial data is the data that the user needs to query, and the initial data may include log files, database data, message queue data, file upload data, HTTP network transmission data, etc. It should be noted that, if Figure 2 As shown, the initial data can be collected and stored by the collection access module for the original service data. When the user queries the initial data, the content of the initial data is the content most recently collected by the collection access module.

[0039] Furthermore, after the data is collected, it is necessary to pre-process the data to improve its accuracy and security. Figure 2 As shown in the figure, the collected data can be processed by the processing and warehousing module. The cleaning operators include forward and reverse serialization, regular extraction, assignment, replacement, KV extraction and traversal. The data type and label after cleaning can be set for different data. Finally, the cleaned data results can be synchronously distributed to different data storage components or databases.

[0040] The first permission information is used to characterize the data content that the user is allowed to query. For example, the data to be queried can be classified into levels 1 to 4, where level 1 data is public information related to the operation and maintenance services of the enterprise; level 2 data is a structured description of generally important operation and maintenance data, including data types, granularity and data characteristics, or general data operation data, daily business information, etc.; level 3 data is important business user data, business servers, system component information, terminals or network equipment, etc.; level 4 is user privacy data that is explicitly not allowed to be disclosed in relevant regulations, system server or database connection information, etc. It should be noted that data of different levels have different sensitivities, and the permissions required for query are also different. The content that can be queried by different permission information is also set differently.

[0041] It should be noted that when classifying data, it is necessary to follow the principles of executability, timeliness, objectivity, etc., and to determine the level of service data used by users. The main purpose is to judge the impact of data leakage on enterprises, industries, and society, and determine the corresponding classification level based on the degree of impact.

[0042] In some implementations, the user's authority information may also be hierarchically arranged, for example, including levels 1 to 4, each level of authority corresponds to each level of data one by one, and each level of authority can query data of the corresponding level and data lower than the corresponding level.

[0043] Step 102: When the initial data is first type data, generate a Structured Query Language (SQL) statement based on a target database storing the initial data, wherein the first type data is data that has not been separated from the service.

[0044] The above-mentioned first type of data is data that has not been separated from the service. It should be noted that the data that has not been separated from the service is in a state of dynamic change, that is, the content of the data will always change, and the change in the data content will cause the sensitivity level of the data to change. At this time, if the first type of data needs to be queried, it is necessary to determine the permission level for querying the first type of database through the database currently storing the first type of data, and then determine whether the user is allowed to query the first type of data.

[0045] The sensitivity level of the data stored in the above target database is the same level. When the initial data needs to be queried, the target database currently storing the initial data is determined, and an SQL statement is generated based on the target database to facilitate subsequent parsing of the SQL statement to obtain a second permission level for querying the SQL statement.

[0046] In some implementations, an SQL statement is generated based on a target database storing the initial data, and specifically, an SQL statement is generated based on a database type of the target database, wherein the database type may be MySql, Hdfs, Doris, ElasticSearch, and Ignite, and different database types are used to store data of different sensitivity levels.

[0047] Step 103: parse the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database.

[0048] It should be noted that the SQL statement is generated based on the target database, and the sensitivity level of the data stored in the target database is the same, that is, the permissions required to query any data in the target database are the same. The SQL statement is parsed to obtain the second permission information required for querying the data in the target database.

[0049] The SQL statement is parsed by performing format verification, result data table extraction, and data field decomposition and rewriting on the SQL statement through a real-time desensitization engine to obtain the second permission information.

[0050] Step 104: When the first permission information matches the second permission information, add the initial data to a message queue.

[0051] The first permission information matches the second permission information, and the level corresponding to the first permission information may be higher than or equal to the level corresponding to the second permission information. When the first permission information matches the second permission information, it is considered that the user has the permission to query the initial data, and the initial data is desensitized so that the user can query the desensitized data.

[0052] When the first permission information and the second permission information do not match, it is considered that the user does not have the permission to query the initial data. At this time, a prompt message can be sent to the terminal where the user queries the initial data to prompt the user that the query condition is not met.

[0053] In some implementations, when the first permission information and the second permission information do not match, the query information may be sent to the service control terminal, and further audited by the service control terminal to determine whether the user is allowed to query the initial data. The service control terminal is a terminal for management and control by an administrator, and when receiving the query information, it may send an instruction of approval or disapproval. When sending an instruction of approval, it is considered that the user has the authority to query the initial data, and the initial data is still added to the message queue; when sending an instruction of disapproval, it is considered that the user does not have the authority to query the initial data, and a prompt message may be sent to the terminal for the user to query the initial data to prompt the user that the query conditions are not met.

[0054] In some embodiments, Figure 2 As shown, the dynamic permission control module can be used to determine whether the first permission information and the second permission information match. The dynamic permission control module is used to dynamically verify whether the user has fine-grained permissions at the data result table level when the operation and maintenance platform user performs non-desensitizing query, use and export of sensitive data.

[0055] Furthermore, considering the scenario where employees in special departments need to query any data, this module provides a business data whitelist, which means that non-masked result data under any business can be queried without approval.

[0056] In some implementations, the message queue may be a message queue such as Kafka or Queue.

[0057] Step 105: desensitize the initial data in the message queue to obtain target data.

[0058] After obtaining the target data, the target data can be sent to the terminal where the user queries the initial data, and the desensitized initial data, i.e., the target data, can be displayed to the user through the terminal; alternatively, the target data can be stored in a data lake warehouse so that the user can query the target data corresponding to the SQL statement in the data lake warehouse.

[0059] The initial data in the message queue is desensitized by embedding a general desensitizing operator function in the Flink real-time computing engine, and the initial data is desensitized by the general desensitizing operator function to obtain the target data.

[0060] Furthermore, the Flink real-time computing engine can provide an open application programming interface (OpenAPI) for users to customize desensitization. It reads the initial data from the message queue through user-defined desensitization and preset desensitization operator functions, and desensitizes the initial data to obtain the target data.

[0061] In some embodiments, Figure 2 As shown in the figure, the desensitization engine module can be used to desensitize the initial data. The desensitization engine module is used to desensitize the data to avoid security risks of user operation and maintenance data, such as illegal leakage, unauthorized tampering, counterfeiting and illegal use. It should be noted that the essence of data desensitization is to change part of its numerical content under the premise of retaining the original characteristics, format, attributes or statistical characteristics of the data, so as to effectively prevent the risk of unauthorized personnel obtaining sensitive data through illegal means.

[0062] Among them, the desensitization engine module includes two modes: offline desensitization and real-time desensitization to achieve desensitization of different data. In the embodiment of the present application, the initial data is the first type of data, so the initial data is desensitized by real-time desensitization.

[0063] Specifically, Figure 3 As shown, the desensitizing engine module includes a real-time desensitizing engine, which generates SQL statements and parses the SQL statements to obtain second permission information. When the first permission information and the second permission information match, the real-time desensitizing engine desensitizes the initial data according to the desensitizing algorithm to obtain target data, and saves the target data.

[0064] It should be noted that the real-time desensitization engine is suitable for scenarios where the data does not leave the original service, and performs real-time desensitization on the results of querying, using, and exporting sensitive data. Here, we take the sensitive data content that needs to be displayed in a certain operation and maintenance business application as an example. Generally, developers or users need to have permission to see all the data, but in order to ensure that the returned data is available and secure, it is not desired that operation and maintenance personnel or third-party users can retrieve or download the real original data. Therefore, during use, as long as the real-time desensitization engine is set up, data can be desensitized in real time. The real-time desensitization engine is generally deployed in the physical parallel and logical series locations of production environment services.

[0065] In an embodiment of the present application, a user's query information for querying initial data is received, and the query information includes the user's first permission information; when the initial data is a first type of data, a structured query language SQL statement is generated based on the target database storing the initial data, and the first type of data is data that has not been separated from the service; the SQL statement is parsed to obtain second permission information, and the second permission information is used to characterize the permission required for data query on the target database; when the first permission information matches the second permission information, the initial data is added to the message queue; the initial data in the message queue is desensitized to obtain the target data. In this way, by generating an SQL statement, parsing the SQL statement to determine the second permission information for querying the initial data, and adding the initial data to the message queue and desensitizing it when the first permission information and the second permission information match, the desensitization of the initial data is achieved when the initial data is not separated from the service and changes dynamically, thereby improving the effect of data desensitization.

[0066] In some embodiments, Figure 2 As shown, the target data can be managed through the data exploration module. It should be noted that after all data is anonymized and saved, users can filter and query valid target data through standard SQL commands. The query results will be uniformly stored in the data lake warehouse, and the query and export services of the target data will be implemented through the data exploration module. Furthermore, in order to achieve a balance between maximizing storage space and usage efficiency as much as possible, the expiration time of query and export history records is generally set to 7 days.

[0067] In one embodiment, before receiving the query information of the user for querying the initial data, the method further includes:

[0068] Acquire the initial data and a label corresponding to the initial data, wherein the label is used to characterize relevant information of the initial data;

[0069] Classifying the initial data based on the label to obtain a level corresponding to the initial data, where the level is used to characterize the sensitivity of the initial data;

[0070] The initial data is stored in a target database corresponding to the level.

[0071] The label of the above-mentioned initial data is a label generated according to the content of the initial data, wherein the label may include multiple keywords, and the multiple keywords are keywords obtained by extracting data from the initial data. By including a label including multiple keywords, the content of the initial data can be quickly determined, and then the level of the initial data can be determined.

[0072] In some implementations, tags are divided into business tags, source tags, data description tags, and custom tags. Among them, business tags include two categories: data domain and region. Data domain tags mainly include technical operation data and business user data; source tags mainly include system devices and components such as business servers, network devices, terminal devices, and file storage systems; data description tags display information such as the structure, granularity, and type of data.

[0073] In the embodiment of the present application, the initial data and the label corresponding to the initial data are obtained, and the label is used to characterize the relevant situation of the initial data; the initial data is classified based on the label to obtain the level corresponding to the initial data, and the level is used to characterize the sensitivity of the initial data; the initial data is stored in the target database corresponding to the level. In this way, the level corresponding to the initial data is determined by the label corresponding to the initial data, and then the initial data is stored in the target database corresponding to the level.

[0074] It should be noted that since the initial data is first type data, the content of the initial data changes dynamically. When the content of the initial data changes, it is necessary to redetermine the level corresponding to the initial data through the method of the embodiment of the present application and then store the initial data in the database corresponding to the redetermined level.

[0075] In some embodiments, Figure 2 As shown in FIG. 1 , the initial data can be classified by the automatic classification module. Specifically, Figure 4 As shown, the automatic grading recommendation module can label the initial data with multiple labels that can characterize the characteristics on the data management page, construct a sample set of data levels and labels, and store the mapping relationship between the corresponding levels and preset samples, so that the initial data can be intelligently graded through the method of the embodiment of the present application.

[0076] In some embodiments, the initial data is graded based on the label to obtain the level corresponding to the initial data. The feature vector of the initial data is calculated based on the keywords in the label, and then the feature vector of the preset sample is compared with the feature vector of the initial data to determine the feature vector of the preset sample that is closest to the feature vector of the initial data. The level of the preset sample is set as the level of the initial data, thereby determining the level of the initial data.

[0077] In one embodiment, the tag includes at least one keyword, and grading the initial data based on the tag to obtain the level corresponding to the initial data includes:

[0078] Calculating the word frequency and length of each keyword in the at least one keyword;

[0079] Calculate the target weight corresponding to each keyword based on the word frequency and the length;

[0080] Calculating the similarity between the initial data and each of the plurality of preset samples based on the target weights corresponding to all the keywords;

[0081] The level corresponding to the initial data is determined based on the similarity.

[0082] In the embodiment of the present application, the frequency and length of each keyword in the at least one keyword are calculated; the target weight corresponding to each keyword is calculated based on the frequency and length; the similarity between the initial data and each preset sample in the plurality of preset samples is calculated based on the target weights corresponding to all keywords; and the level corresponding to the initial data is determined based on the similarity. In this way, the level corresponding to the initial data is determined by the frequency and length of the keywords in the tag.

[0083] In some implementations, the frequency of a keyword is the ratio of the keyword to all keywords in the tag, which is specifically expressed by the following formula:

[0084]

[0085] In the formula, F(V i ) is the keyword V i The word frequency, count(V i ) is the keyword V i The number of is the number of all keywords, and n is the types of all keywords.

[0086] In some implementations, in addition to the above formula, the frequency of a keyword may also be the number of times the keyword appears in the initial data, and then the number of times all keywords appear is normalized to obtain the frequency of each keyword.

[0087] In some implementations, the length of the keyword may be calculated by calculating a feature vector of the keyword and setting the length of the feature vector to the length of the keyword.

[0088] Furthermore, the length of the keyword can also be processed by distinguishing the odd and even lengths of the keyword to obtain the optimized keyword length. The calculation process can be expressed by the following formula:

[0089]

[0090] In the formula, L(V i ) is the optimized keyword length, len(V i ) is the keyword V i The length of the feature vector.

[0091] The target weight corresponding to each keyword is calculated based on the word frequency and the length. The weights may be pre-set for the word frequency and the length, and then the word frequency and the length are weighted and summed by the pre-set weights to obtain the target weight. For example, the calculation process is shown in the following formula:

[0092] W(V i )=k1*F(V i )+k2*L(V i )

[0093] In the formula, W(V i ) is the target weight, F(V i ) is the keyword V i The word frequency, L(V i ) is the optimized keyword length, k1 and k2 are the pre-set weights.

[0094] In some implementations, in addition to the above formula, the product of the keyword frequency and length can also be set as the target weight. For example, by formula W(V i )=F(V i )*L(V i ) to calculate the target weight.

[0095] In one embodiment, the calculating the target weight corresponding to each keyword based on the word frequency and the length includes:

[0096] Weighting the word frequency and the length to obtain an intermediate weight corresponding to each keyword;

[0097] The target weight corresponding to each keyword is calculated based on the intermediate weight, the target weight being the sum of the target product and the target difference, the target product being the product of the intermediate weight and the damping coefficient, and the target difference being the difference between 1 and the damping coefficient.

[0098] It should be noted that, in order to further improve the accuracy of calculating the target weight of each keyword, the embodiment of the present application also introduces a damping coefficient, and the target weight of each keyword is optimized through the damping coefficient.

[0099] In the embodiment of the present application, the word frequency and the length are weighted to obtain the intermediate weight corresponding to each keyword; based on the intermediate weight, the target weight corresponding to each keyword is calculated, and the target weight is the sum of the target product and the target difference, the target product is the product of the intermediate weight and the damping coefficient, and the target difference is the difference between 1 and the damping coefficient. In this way, the target weight is optimized by the damping coefficient, so that the level corresponding to the initial data determined later by the target weight is more accurate.

[0100] The process of optimizing the target weight by the damping coefficient is expressed by the following formula:

[0101] X Vec =d*W(V i )+(1-d)

[0102] In the formula, d is the damping coefficient, X Vec is the optimized target weight, W(V i ) is the target weight before optimization (i.e., the intermediate weight).

[0103] In some implementations, the target weight may be further optimized by the weight vector product of the keywords in the tag, that is, the product of the damping coefficient and the weight vector product is calculated, and then the sum of the product and the difference between 1 and the damping coefficient is set as the optimized target weight.

[0104] Specifically, it can be expressed by the following formula:

[0105]

[0106] The formula is Keywords in tags V i The weight vector product of .

[0107] In one embodiment, determining the level corresponding to the initial data based on the similarity includes:

[0108] Selecting N preset samples with the highest similarity from the multiple preset samples, where N is a positive integer greater than 1;

[0109] Obtaining levels corresponding to the N preset samples;

[0110] The maximum level corresponding to the N preset samples is set as the level corresponding to the initial data.

[0111] The N preset samples are samples with corresponding levels. The N preset samples can be labeled in the manner of the embodiment of the present application, and the weight corresponding to each keyword in the label is calculated, so as to calculate the initial data with each preset sample to obtain the similarity.

[0112] Among them, N preset samples can be expressed as: T = {(X1, Y1), (X2, Y2), ..., (X n ,Y n )}, where X n is the feature vector corresponding to the keyword, Y n is the corresponding level, and T is a sample set including N preset samples.

[0113] The above similarity can be the similarity calculated by the Euclidean distance similarity measurement method. As an example, it should be clear that other similar matching distance algorithms including but not limited to Pearson, cosine, etc. should be included in the protection scope of this application. According to any given feature vector Xs of the initial data, find the N preset samples closest to Xs from the sample set T, and the set covering these N preset samples is recorded as N vec (Xs).

[0114] After obtaining N preset samples, the levels corresponding to the N preset samples are counted, and the most levels are set as the levels corresponding to the initial data, so that the levels can be more accurate.

[0115] The above process can be expressed by the following formula:

[0116] Y s =arg(MAX(I Nvec(Xs) ))

[0117] Y s is the level corresponding to the initial data.

[0118] It should be noted that the selection of N value has a great influence on the final result. When N value is small, the model complexity is high, the training error will be reduced, and the generalization ability will be weakened; when N value is large, the model complexity is low, the training error will increase, and the generalization ability will be improved to a certain extent. Therefore, in the embodiment of the present invention, the cross-validation classification accuracy is performed by grid traversal of N value (1≤N≤10, the value range of N is determined by the experience of business experts), all the obtained accuracy rates are added and averaged, and finally the N value with the highest accuracy rate is the most appropriate.

[0119] In some embodiments, the level corresponding to the initial data is determined based on the similarity, and the level corresponding to the preset sample with the highest similarity can be directly set as the level of the initial data. This can more quickly determine the level of the initial data, improve data processing speed, and improve the overall response rate.

[0120] In one embodiment, after storing the initial data in the target database corresponding to the level, the method further includes:

[0121] In the case where the initial data is second type data, reading the initial data in the database, the second type data being data out of service;

[0122] Performing desensitization processing on the initial data to obtain the target data;

[0123] The target data is stored in a data lake warehouse.

[0124] The second type of data is data that is out of service. It should be noted that data that is out of service is relatively more stable, and the content of the data will not change dynamically. At this time, the second type of data can be desensitized in advance to reduce the resource occupation of temporary desensitization when users need to query, and improve the access efficiency when users query.

[0125] Specifically, the out-of-service data can be stored in a distributed data storage component, and the out-of-service data in the storage component can be batch-read and then batch-desensitized to achieve large-scale data desensitization, thereby reducing the resource occupation of temporary desensitization when users need to query and improving the access efficiency when users query. The desensitized data will be stored in the data lake warehouse for subsequent user queries or use.

[0126] In some embodiments, Figure 3 As shown, the desensitizing engine module includes an offline desensitizing engine, which is used for offline desensitizing of general static data files in production environments or external third-party services. It also supports synchronous processing of various large quantities of sensitive data, and finally distributes them to databases specified by various data analysis scenarios.

[0127] Among them, the offline desensitization engine is mainly composed of data extraction, data offline desensitization algorithm and data output. The data desensitization algorithm integrates many desensitization methods, such as offset, regularization, stage, replacement, masking and hashing. Users can select specific desensitization algorithms according to the security requirements of their business and the characteristics of the data structure, and desensitize the data through specific desensitization algorithms to obtain desensitized data.

[0128] See also Figure 5 , Figure 5 is a structural diagram of a data desensitization device provided in an embodiment of the present application, such as Figure 5 As shown, the data desensitization device 500 includes:

[0129] A receiving module 501 is used to receive query information of a user for querying initial data, wherein the query information includes the first authority information of the user;

[0130] A generating module 502, configured to generate a structured query language SQL statement based on a target database storing the initial data when the initial data is first type data, the first type data being data that has not been separated from the service;

[0131] A parsing module 503 is used to parse the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database;

[0132] An adding module 504, configured to add the initial data to a message queue when the first permission information matches the second permission information;

[0133] The first desensitizing module is used to perform desensitization processing on the initial data in the message queue to obtain target data.

[0134] In one embodiment, before the receiving module 501, the data desensitization device 500 further includes:

[0135] An acquisition module, used to acquire the initial data and a label corresponding to the initial data, wherein the label is used to characterize relevant information of the initial data;

[0136] A grading module, used to grade the initial data based on the label to obtain a grade corresponding to the initial data, wherein the grade is used to characterize the sensitivity of the initial data;

[0137] The first storage module is used to store the initial data in a target database corresponding to the level.

[0138] In one embodiment, the tag includes at least one keyword, and the grading module includes:

[0139] A first calculation unit, configured to calculate the frequency and length of each keyword in the at least one keyword;

[0140] A second calculation unit, used for calculating the target weight corresponding to each keyword based on the word frequency and the length;

[0141] A third calculation unit, configured to calculate the similarity between the initial data and each of the plurality of preset samples based on the target weights corresponding to all the keywords;

[0142] A determining unit is used to determine the level corresponding to the initial data based on the similarity.

[0143] In one embodiment, the determining unit includes:

[0144] A screening subunit, used to screen the N preset samples with the highest similarity from the multiple preset samples, where N is a positive integer greater than 1;

[0145] An acquisition subunit, used to acquire levels corresponding to the N preset samples;

[0146] The setting subunit is used to set the maximum level corresponding to the N preset samples as the level corresponding to the initial data.

[0147] In one embodiment, the second computing unit comprises:

[0148] A weighting subunit, used for weighting the word frequency and the length to obtain an intermediate weight corresponding to each keyword;

[0149] A calculation subunit is used to calculate the target weight corresponding to each keyword based on the intermediate weight, wherein the target weight is the sum of the target product and the target difference, the target product is the product of the intermediate weight and the damping coefficient, and the target difference is the difference between 1 and the damping coefficient.

[0150] In one embodiment, after the first storage module, the data desensitization device 500 further includes:

[0151] a reading module, configured to read the initial data in the database when the initial data is second type data, the second type data being data out of service;

[0152] A second desensitization module, used for performing desensitization processing on the initial data to obtain the target data;

[0153] The second storage module is used to store the target data in a data lake warehouse.

[0154] The data desensitization device provided in the embodiment of the present application is capable of implementing each process of each embodiment of the above-mentioned data desensitization method. The technical features correspond one to one and can achieve the same technical effect. To avoid repetition, they will not be described here.

[0155] It should be noted that the data desensitizing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in an electronic device.

[0156] For details, see Figure 6 The embodiment of the present application further provides an electronic device, including a bus 601, a transceiver 602, an antenna 603, a bus interface 604, a processor 605 and a memory 606.

[0157] The transceiver 602 is used to receive the query information of the user for querying the initial data, and the query information includes the first authority information of the user;

[0158] The processor 605 is configured to generate a structured query language SQL statement based on a target database storing the initial data when the initial data is first type data, the first type data being data that has not been separated from the service;

[0159] The processor 605 is further configured to parse the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database;

[0160] The processor 605 is further configured to add the initial data to a message queue when the first permission information matches the second permission information;

[0161] The processor 605 is further used to perform desensitization processing on the initial data in the message queue to obtain target data.

[0162] In one embodiment, the transceiver 602 is further used to obtain the initial data and a tag corresponding to the initial data, where the tag is used to characterize relevant information of the initial data;

[0163] The processor 605 is further configured to classify the initial data based on the label to obtain a level corresponding to the initial data, where the level is used to represent the sensitivity of the initial data;

[0164] The processor 605 is further configured to store the initial data in a target database corresponding to the level.

[0165] In one embodiment, the tag includes at least one keyword, and grading the initial data based on the tag to obtain the level corresponding to the initial data includes:

[0166] Calculating the word frequency and length of each keyword in the at least one keyword;

[0167] Calculate the target weight corresponding to each keyword based on the word frequency and the length;

[0168] Calculating the similarity between the initial data and each of the plurality of preset samples based on the target weights corresponding to all the keywords;

[0169] The level corresponding to the initial data is determined based on the similarity.

[0170] In one embodiment, determining the level corresponding to the initial data based on the similarity includes:

[0171] Selecting N preset samples with the highest similarity from the multiple preset samples, where N is a positive integer greater than 1;

[0172] Obtaining levels corresponding to the N preset samples;

[0173] The maximum level corresponding to the N preset samples is set as the level corresponding to the initial data.

[0174] In one embodiment, the calculating the target weight corresponding to each keyword based on the word frequency and the length includes:

[0175] Weighting the word frequency and the length to obtain an intermediate weight corresponding to each keyword;

[0176] The target weight corresponding to each keyword is calculated based on the intermediate weight, the target weight being the sum of the target product and the target difference, the target product being the product of the intermediate weight and the damping coefficient, and the target difference being the difference between 1 and the damping coefficient.

[0177] In one embodiment, the transceiver 602 is further configured to read the initial data in the database when the initial data is second type data, the second type data being out-of-service data;

[0178] The processor 605 is further configured to perform desensitization processing on the initial data to obtain the target data;

[0179] The processor 605 is further configured to store the target data in a data lake warehouse.

[0180] exist Figure 6 In the embodiment, a bus architecture (represented by bus 601) is shown, bus 601 may include any number of interconnected buses and bridges, bus 601 links various circuits including one or more processors represented by processor 605 and memory represented by memory 606. Bus 601 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 604 provides an interface between bus 601 and transceiver 602. Transceiver 602 may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by processor 605 is transmitted on a wireless medium via antenna 603, and further, antenna 603 also receives data and transmits the data to processor 605.

[0181] The processor 605 is responsible for managing the bus 601 and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 606 can be used to store data used by the processor 605 when performing operations.

[0182] Optionally, the processor 605 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD).

[0183] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned image rendering method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0184] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the embodiment of the above-mentioned image rendering method is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0185] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the embodiment of the above-mentioned image rendering method are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0186] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0187] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0188] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A data desensitization method, characterized in that: include: Receive query information of a user for querying initial data, wherein the query information includes first authority information of the user; In the case where the initial data is first type data, generating a structured query language SQL statement based on a target database storing the initial data, the first type data being data that has not been separated from the service; Parsing the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database; When the first permission information matches the second permission information, adding the initial data to a message queue; The initial data in the message queue is desensitized to obtain target data.

2. The method according to claim 1, characterized in that Before receiving the query information of the user for querying the initial data, the method further includes: Acquire the initial data and a label corresponding to the initial data, wherein the label is used to characterize relevant information of the initial data; Classifying the initial data based on the label to obtain a level corresponding to the initial data, where the level is used to characterize the sensitivity of the initial data; The initial data is stored in a target database corresponding to the level.

3. The method according to claim 2, characterized in that The tag includes at least one keyword, and the grading of the initial data based on the tag to obtain the level corresponding to the initial data includes: Calculating the word frequency and length of each keyword in the at least one keyword; Calculate the target weight corresponding to each keyword based on the word frequency and the length; Calculating the similarity between the initial data and each of the plurality of preset samples based on the target weights corresponding to all the keywords; The level corresponding to the initial data is determined based on the similarity.

4. The method according to claim 3, characterized in that The determining the level corresponding to the initial data based on the similarity includes: Selecting N preset samples with the highest similarity from the multiple preset samples, where N is a positive integer greater than 1; Obtaining levels corresponding to the N preset samples; The maximum level corresponding to the N preset samples is set as the level corresponding to the initial data.

5. The method according to claim 3, characterized in that The calculating the target weight corresponding to each keyword based on the word frequency and the length includes: Weighting the word frequency and the length to obtain an intermediate weight corresponding to each keyword; The target weight corresponding to each keyword is calculated based on the intermediate weight, the target weight being the sum of the target product and the target difference, the target product being the product of the intermediate weight and the damping coefficient, and the target difference being the difference between 1 and the damping coefficient.

6. The method according to claim 2, characterized in that After storing the initial data in the target database corresponding to the level, the method further includes: In the case where the initial data is second type data, reading the initial data in the database, the second type data being data out of service; Performing desensitization processing on the initial data to obtain the target data; The target data is stored in a data lake warehouse.

7. A data desensitization device, characterized in that: include: A receiving module, configured to receive query information of a user for querying initial data, wherein the query information includes first authority information of the user; A generating module, configured to generate a structured query language SQL statement based on a target database storing the initial data when the initial data is first type data, the first type data being data that has not been separated from the service; A parsing module, used to parse the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database; an adding module, configured to add the initial data to a message queue when the first permission information matches the second permission information; The first desensitizing module is used to perform desensitization processing on the initial data in the message queue to obtain target data.

8. An electronic device, characterized in that: Including transceiver and processor, The transceiver is used to receive query information of a user for querying initial data, wherein the query information includes the first authority information of the user; The processor is configured to generate a structured query language SQL statement based on a target database storing the initial data when the initial data is first type data, the first type data being data that has not been separated from the service; The processor is further used to parse the SQL statement to obtain second permission information, where the second permission information is used to represent the permission required for querying data in the target database; The processor is further configured to add the initial data to a message queue when the first permission information matches the second permission information; The processor is also used to perform desensitization processing on the initial data in the message queue to obtain target data.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps in the data desensitization method as claimed in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the data desensitization method according to any one of claims 1 to 6.

11. A computer program product, characterized in that It comprises computer instructions, which, when executed by a processor, implement the steps of the data desensitization method as described in any one of claims 1 to 6.

Images