A method and system for real-time desensitization of database data

Abstract

The present invention provides a method and system for real-time desensitization of database data. The present invention relates to the technical field of database information security. The present invention can desensitize data in real time according to the data in online production, and make the desensitized data in multiple tables consistent through algorithms, so as to solve the problems in the prior art that the data is inconsistent before and after desensitization, the data cannot be desensitized in real time, and the encryption keys used in the production environment and the Perf environment are inconsistent, resulting in the data synchronized to the Perf environment being unusable.

Description

Technical Field

[0001] The present invention relates to the technical field of database information security, and in particular to a method and system for real-time desensitization of database data. Background Art

[0002] In modern information management, data security and privacy protection have become one of the focuses of attention for enterprises and individuals. With the growth of data and the continuous increase in data flows, data desensitization technology has emerged. Data desensitization is a technology that protects data privacy by replacing, masking, or transforming original sensitive information. Especially in the scenario of troubleshooting in the pre-release environment simulating the production environment, how to ensure the privacy and consistency of data during the use of the pre-production environment has become an important technical challenge;

[0003] The existing technologies face the following major problems in the process of data desensitization:

[0004] 1. Data consistency issue: In data scenarios involving multiple tables or multiple systems, although the original data is consistent, the desensitization processing methods in different tables are not unified, resulting in inconsistency in the same data after desensitization, which brings problems to subsequent data analysis and use.

[0005] 2. Performance bottleneck: For scenarios where large amounts of data need to be processed in real time, existing desensitization solutions often desensitize the data once and for all, which cannot meet real-time requirements.

[0006] The encryption keys used by the production environment and the Perf environment are inconsistent: The encryption keys used by the production environment and the Perf environment are inconsistent, resulting in the data synchronized to the Perf environment being unusable Summary of the invention

[0007] In view of the above technical problems, the present invention proposes a method and system for real-time desensitization of database data, and the technical solutions adopted are as follows:

[0008] A real-time desensitization method for database data, the method comprising:

[0009] S1: The data in the online main database is divided into sensitive data and non-sensitive data. The non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. The sensitive data is collected and sent to the middleware.

[0010] S2: Generate a corresponding desensitization method label based on the type of each original field in the sensitive data;

[0011] S3: The real-time data masking program retrieves sensitive data from the middleware, determines the masking method for each field based on the masking method tag, obtains a judgment result, generates fake data of the corresponding type according to the judgment result, and sends the fake data to the masking library;

[0012] S4: Calculate the MD5 value of the original field, use the calculation result as the seed of the pseudo-random number, generate fixed numbers according to the seed of the pseudo-random number, and piece together the generated fixed numbers to form a complete masked data record.

[0013] Preferably, the S1 includes:

[0014] S11: Preset the sensitive data classification standard, embed the classification standard into the online main library, and the online main library divides the original data in the online main library into sensitive data and non-sensitive data according to the classification standard;

[0015] S12: Synchronize the non-sensitive data to the masking library in real time through master-slave synchronization, and transfer the sensitive data to the middleware located between the online main library and the masking library.

[0016] Preferably, the S2 includes:

[0017] S21: Analyze the type of each original field in the sensitive data, and the types include numbers, strings, dates, and keywords;

[0018] S22: Generate corresponding masking methods according to different original field types, and mark the corresponding masking method tags for each generated masking method.

[0019] Preferably, the S3 includes:

[0020] S31: The real-time data masking program retrieves sensitive data from the middleware, encrypts the sensitive data, and determines the masking method that each field should adopt according to the masking method tag;

[0021] S32: Generate fake data of the corresponding type according to the judgment result, send the generated fake data to the masking library to replace the original sensitive data, and the generation method of the fake data is obtained through the following formula:

[0022] A: Obtain the sensitive data P, and the string of P is N, and the fake data replacement system introduces a high-dimensional density matrix, 、 ..., Generate a confusion vector of length N , generate a natural number permutation vector containing 1 to N , introduce multiple non-linear functions 、 ..., ;

[0023] B: Based on the matrix - vector function introduced in step A, perform dummy - data replacement on the sensitive data. And the replacement process is as follows:

[0024] B1: Convert P into the corresponding ASCII - value sequence to obtain the vector ;

[0025] B2: Perform non - linear transformation on the vector and the above - mentioned multiple non - linear functions to obtain an intermediate change result , and, The calculation formula of

[0026] ;

[0027] ;

[0028] ;

[0029] B3: Convert the intermediate change result with the confusion vector to obtain a confusion change result , and, The calculation formula of

[0030] ;

[0031] B4: Perform multi - matrix transformation on the mixed change result to obtain an applied permutation vector , and, the calculation formula of the applied permutation vector is as follows:

[0032] ;

[0033] C: Use the applied permutation vector as the replaced data, and send the vector to the desensitization library;

[0034] Preferably, the S4 includes:

[0035] S41: Perform MD5 hashing on each original field, and the original field is converted into a hash value after the operation;

[0036] S42: Use the hash value as the seed of the pseudo - random number. Through the pseudo - random number generator, according to the preset digital length, divide the hash value into several fixed - length numbers;

[0037] S43: Piece together several fixed - length numbers to form a complete desensitized data record.

[0038] A real-time data masking system for a database, the system comprising:

[0039] Data transmission system: Divide the data in the online master database into sensitive data and non-sensitive data. The non-sensitive data is synchronously replicated from the online master database to the masked database in real time through master-slave replication. The sensitive data is collected and sent to the middleware;

[0040] Label generation system: Generate corresponding data masking method labels according to the type of each original field in the sensitive data;

[0041] Fake data replacement system: The real-time data masking program obtains the sensitive data from the middleware, determines the data masking method for each field according to the data masking method label, obtains the determination result, and generates fake data of the corresponding type according to the determination result, and sends the fake data to the masked database;

[0042] Recording system: Calculate the MD5 value of the original field, use the calculation result as the seed of the pseudo-random number, and generate fixed numbers according to the seed of the pseudo-random number. The generated fixed numbers are pieced together to form a complete masked data record.

[0043] Preferably, the data transmission system comprises:

[0044] Data classification system: Preset the sensitive data classification criteria, embed the classification criteria into the online master database, and the online master database divides the original data in the online master database into sensitive data and non-sensitive data according to the classification criteria;

[0045] Data synchronization system: Synchronously replicate the non-sensitive data to the masked database in real time through master-slave replication, and transmit the sensitive data to the middleware located between the online master database and the masked database.

[0046] Preferably, the label generation system comprises:

[0047] Data analysis system: Analyze the type of each original field in the sensitive data, and the types include numbers, strings, dates, and keywords;

[0048] Marking system: Generate corresponding data masking methods according to different original field types, and mark each generated data masking method with a corresponding data masking method label.

[0049] Preferably, the fake data replacement system comprises:

[0050] Data masking method judgment system: The real-time data masking program obtains the sensitive data from the middleware, encrypts the sensitive data, and determines the data masking method that should be adopted for each field according to the data masking method label;

[0051] Fake data generation system: According to the judgment result, generate fake data of the corresponding type, and send the generated fake data to the desensitization library to replace the original sensitive data.

[0052] Preferably, the recording system includes:

[0053] Data conversion system: Perform MD5 hash operation on each original field, and the original field is converted into a hash value after the operation;

[0054] Pseudo-random number generation system: Use the hash value as the seed of the pseudo-random number, and through the pseudo-random number generator, divide the hash value into several fixed-length numbers according to the preset digital length;

[0055] Combination system: Piece together several fixed-length numbers to form a complete desensitized data record.

[0056] Advantages of the present invention: It can desensitize data in real time according to the data in online production, and ensure the consistency of desensitized data in multiple tables to ensure the normal operation of the business in the desensitization library environment. Description of the Drawings

[0057] Figure 1 A database data real-time desensitization system according to the present invention;

[0058] Figure 2 A desensitization method judgment system according to the present invention. Detailed Embodiments

[0059] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0060] An embodiment of the present invention, a method for real-time desensitization of database data, the method includes:

[0061] S1: Divide the data in the online main database into sensitive data and non-sensitive data. The non-sensitive data is synchronously replicated from the online main database to the desensitization library in real time through master-slave synchronization, and the sensitive data is collected and sent to the middleware;

[0062] S2: Generate corresponding desensitization method tags according to the type of each original field in the sensitive data;

[0063] S3: The real-time desensitization program obtains the sensitive data from the middleware, and according to the desensitization method tag, judges the desensitization method of each field to obtain a judgment result, and generates fake data of the corresponding type according to the judgment result, and sends the fake data to the desensitization library;

[0064] S4: Calculate the MD5 value of the original field, use the calculation result as the seed of the pseudo-random number, and generate fixed numbers according to the seed of the pseudo-random number. Piece together the generated fixed numbers to form a complete desensitized data record.

[0065] The working principle and effects of the above technical solution are as follows: Sensitive data includes personal identity information, financial data, customer records, etc. Once these data are leaked, they may pose a threat to personal privacy or enterprise security. Non-sensitive data refers to data that will not cause significant threats to individuals or enterprises even if leaked. Non-sensitive data is directly and real-time synchronized from the online master database to the desensitized database through the MySQL master-slave synchronization technology. The real-time synchronization of non-sensitive data to the desensitized database through the MySQL master-slave synchronization technology reduces the complexity of data migration and processing. Sensitive data is collected from TiDB by TiCDC and sent to Kafka. The collection and transmission of sensitive data through TiCDC to Kafka realizes data isolation and encrypted transmission, reducing the risk of data leakage. As a message middleware, Kafka is responsible for real-time data stream transmission to ensure the real-time collection and transmission of sensitive data. For each identified sensitive data field, analyze its data type and usage. According to the type of the field and business requirements, generate a desensitization method label. The desensitization method label includes DML desensitization or DDL desensitization. The desensitized data will be written into the downstream desensitized database. To ensure the consistency of the desensitized data when generating fake data, the present invention calculates the MD5 value based on the original field. The MD5 value is used as the seed of the pseudo-random number to generate fixed numbers. Piece together the selected fake data to form a desensitized data record. By using the MD5 value as the seed of the pseudo-random number, it can be ensured that the generated fake data is consistent under the same original field. The present invention can desensitize data in real time according to the data in online production, and ensure the consistency of desensitized data in multiple tables to ensure the normal operation of the business in the desensitized database environment.

[0066] In an embodiment of the present invention, the S1 includes:

[0067] S11: Preset a sensitive data classification standard, embed the classification standard into the online master database, and the online master database divides the original data in the online master database into sensitive data and non-sensitive data according to the classification standard;

[0068] S12: Synchronize the non-sensitive data to the desensitized database in real time through master-slave synchronization, and transmit the sensitive data to the middleware located between the online master database and the desensitized database.

[0069] The working principle and effects of the above technical solution are as follows: Preset the classification criteria for sensitive data, embed the set classification criteria for sensitive data into the online main database. The main database divides the data into two categories: sensitive data and non-sensitive data by using the predefined criteria. For the data classified as non-sensitive, the system will synchronize this data to the desensitized database in real time through the MySQL master-slave synchronization technology. The desensitized database is a database dedicated to storing non-sensitive data. For the data classified as sensitive, the system will not directly synchronize it to the desensitized database, but transmit it to the middleware Kafka located between the online main database and the desensitized database. By presetting the classification criteria in the main database, the system can accurately identify and separate sensitive data and non-sensitive data, and take corresponding processing measures. The non-sensitive data is updated to the desensitized database in real time through master-slave synchronization, ensuring the availability and consistency of the data; while the sensitive data is isolated and further processed through the middleware, such as desensitization or encryption, increasing the security. This method not only improves the efficiency of data management, but also reduces the risk of leakage of sensitive information, and is suitable for data processing environments that require high security and privacy protection.

[0070] In one embodiment of the present invention, S2 includes:

[0071] S21: Analyze the type of each original field in the sensitive data, and the types include numbers, strings, dates, and keywords;

[0072] S22: Generate corresponding desensitization methods according to different original field types, and mark each generated desensitization method with a corresponding desensitization method label.

[0073] The working principle and effects of the above technical solution are as follows: Analyze the original types of each field in the sensitive data to identify the basic structure of the data. According to the analyzed original field types, generate corresponding desensitization methods for each data type. Add corresponding labels to each generated desensitization method. The desensitization method labels include DML desensitization or DDL desensitization. By analyzing the sensitive data field types and generating corresponding desensitization methods, this process not only improves the accuracy of data privacy protection, but also enhances the flexibility and applicability. Identifying the field types enables the system to apply specific desensitization strategies for each data type, effectively reducing the risk of information leakage while maintaining the availability and accuracy of the data. This method manages desensitization strategies in a labeled manner, which is convenient for maintenance and update, helps to better ensure the compliance and security of sensitive data processing, and provides a robust and adjustable data privacy solution for enterprises.

[0074] In one embodiment of the present invention, S3 includes:

[0075] S31: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field according to the desensitization method tag;

[0076] S32: According to the judgment result, generate fake data of the corresponding type, and send the generated fake data to the desensitization library to replace the original sensitive data. Moreover, the generation method of the fake data is obtained through the following formula:

[0077] A: Obtain sensitive data P, and the string of P is N, and the fake data replacement system introduces a high-dimensional density matrix, , ..., Generate a confusion vector of length N , generate a natural number permutation vector containing 1 to N , introduce multiple non-linear functions , ..., ;

[0078] B: According to the matrix vector function introduced in step A, perform fake data replacement on the sensitive data. Moreover, the replacement process is as follows:

[0079] B1: Convert P into the corresponding ASCII value sequence to obtain a vector ;

[0080] B2: Perform non-linear conversion on the vector and the above multiple non-linear functions to obtain an intermediate change result . Moreover, 's calculation formula is as follows:

[0081]

[0082]

[0083]

[0084] B3: Convert the intermediate change result with the confusion vector to obtain a confusion change result . Moreover, 's calculation formula is as follows:

[0085]

[0086] B4: Perform multi-matrix transformation on the mixed change result to obtain an applied permutation vector . Moreover, the applied permutation vector 's calculation formula is as follows:

[0087]

[0088] C: Replace the said application replacement vector as the data after replacement, and send the vector to the desensitization library.

[0089] The working principle and effect of the above technical solution are as follows: The replacement algorithm is used for replacement processing. The core of this algorithm lies in converting the original data into an encrypted data that is difficult to be reverse-analyzed through a series of mathematical transformation processes. In this process, mathematical tools such as high-dimensional density matrices, confusion vectors, and non-linear functions are introduced to ensure that the encrypted data is both secure and difficult to be cracked. The specific encryption process includes the following steps: Convert the highly sensitive data (set as P, with a string length of N) into a corresponding sequence of ASCII values to obtain a vector. Perform non-linear transformation on this vector with a series of non-linear functions to generate an intermediate transformation result. Combine the intermediate transformation result with the confusion vector to perform further confusion transformation. Perform multi-matrix transformation on the confused result to finally obtain an application replacement vector, which is the data after replacement.

[0090] Through the above replacement processing, even if the data is leaked, it is difficult for attackers to obtain the original sensitive information from it. The system uses a shielding algorithm for processing. The core of this algorithm lies in shielding or replacing the key parts of sensitive information, so that the original information cannot be directly recognized. For example, when processing ID card information, only part of the numbers or characters can be retained, and the other parts are replaced with asterisks * or other characters. When processing sensitive texts such as medical records or passwords, a similar shielding strategy can also be adopted. The system will create a dedicated highly sensitive database based on the highly sensitive data after desensitization processing. This database is set as the data output end for storing and managing the desensitized sensitive data. At the same time, the system will also perform detailed parameter configuration on this database, including primary keys, foreign keys, data formats and types, and performance optimization parameters, etc.

[0091] These parameter configurations can ensure the stable operation and efficient access of the database, and at the same time facilitate data managers to manage and maintain the desensitized data. By performing static desensitization processing on highly sensitive data, it can be ensured that these sensitive information is not leaked during the storage process, thus greatly improving the security of the data. Even if the database is attacked or leaked, attackers cannot directly obtain the original highly sensitive data, but can only obtain the encrypted or shielded data, which greatly reduces the risk brought by data leakage. During the desensitization process, the system tries its best to retain the original features and structures of the data, so that the desensitized data can still maintain its usability to a certain extent and meet the needs of data analysis, mining, etc. The system supports different desensitization processing for different types of sensitive data.

[0092] In one embodiment of the present invention, S4 includes:

[0093] S41: Perform MD5 hashing on each original field, and the original field is converted into a hash value after the operation;

[0094] S42: Use the hash value as the seed of the pseudo-random number, and through the pseudo-random number generator, according to the preset digital length, divide the hash value into several fixed-length numbers;

[0095] S43: Piece together several fixed-length numbers to form a complete desensitized data record.

[0096] The working principle and effect of the above technical solution are as follows: First, the data conversion system performs MD5 hashing on each original field. MD5 is an encryption hash function used to generate a fixed-length hash value of 128 bits. After the input value of the original field is converted by MD5, a fixed hash value will be generated regardless of its original length and content. Use the generated MD5 hash value as the seed of the pseudo-random number generator. This approach enables the same output to be consistently generated for the same input field. Using the pseudo-random number generator, according to the specified scheme and requirements, the generated hash value is further converted into one or more fixed-length numbers. These numbers are the so-called pseudo-random numbers, which logically replace the original sensitive data. And set the length of each number as needed to match the business requirements. Finally, the combination system is responsible for splicing these independent fixed-length numbers and recombining them into a complete desensitized data record. The pieced-together data is consistent with the original data in structure for continued use in downstream system applications. In this way, the formed desensitized data record has a sense of reality and consistency, but does not contain any original sensitive information. The recording system ensures that the generated desensitized data is both consistent and does not expose the original information by performing MD5 hashing on the original field and then using the pseudo-random number generation mechanism. The system uses the MD5 hash value as the seed to generate fixed-length pseudo-random numbers, making the desensitized data predictable and consistent each time it is generated, suitable for multiple tests and analyses. In addition, the combination system that integrates various numbers to form a complete record ensures that the desensitized data remains structurally complete and logically coherent. This method not only enhances data privacy protection but also provides highly realistic and fully functional alternative data for business operations.

[0097] In one embodiment of the present invention, a database data real-time desensitization system, the system includes:

[0098] Data transmission system: The data in the online master database is divided into sensitive data and non-sensitive data. The non-sensitive data is synchronously replicated from the online master database to the desensitized database in real time through master-slave synchronization. The sensitive data is collected and sent to the middleware;

[0099] Label generation system: According to the type of each original field in the sensitive data, generate corresponding desensitization method labels;

[0100] Fake data replacement system: The real-time desensitization program obtains the sensitive data from the middleware, and according to the desensitization method label, determines the desensitization method for each field to obtain a judgment result, and generates fake data of the corresponding type according to the judgment result, and sends the fake data to the desensitized database;

[0101] Recording system: Calculate the MD5 value of the original field, use the calculation result as the seed of the pseudo-random number, and generate fixed numbers according to the seed of the pseudo-random number, and piece together the generated fixed numbers to form a complete desensitized data record.

[0102] The working principle and effect of the above technical solution are as follows: Sensitive data includes personal identity information, financial data, customer records, etc. Once these data are leaked, they may pose a threat to personal privacy or enterprise security. Non-sensitive data refers to data that will not pose a major threat to individuals or enterprises even if leaked. The non-sensitive data is synchronously replicated from the online master database to the desensitized database in real time through MySQL master-slave synchronization technology. The non-sensitive data is synchronously replicated to the desensitized database through MySQL master-slave synchronization technology in real time, reducing the complexity of data migration and processing. The sensitive data is collected by TiCDC from TiDB and sent to Kafka. The sensitive data is collected by TiCDC and sent to Kafka, realizing data isolation and encrypted transmission, reducing the risk of data leakage. Kafka, as a message middleware, is responsible for real-time data stream transmission to ensure the real-time collection and transmission of sensitive data. For each identified sensitive data field, analyze its data type and usage. According to the type of the field and business requirements, generate desensitization method labels. The desensitization method labels include DML desensitization or DDL desensitization. The desensitized data will be written into the downstream desensitized database. In order to ensure the consistency of the desensitized data when generating fake data, the present invention calculates the MD5 value according to the original field. The MD5 value is used as the seed of the pseudo-random number to generate fixed numbers. The selected fake data is pieced together to form a desensitized data record. By using the MD5 value as the seed of the pseudo-random number, it can be ensured that the generated fake data is consistent under the same original field. The present invention can desensitize data in real time according to the data in online production, and ensure the consistency of desensitized data in multiple tables to ensure the normal operation of the business in the desensitized database environment.

[0103] In an embodiment of the present invention, the data transmission system includes:

[0104] Data classification system: preset sensitive data classification criteria, embed the classification criteria into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification criteria;

[0105] Data synchronization system: synchronize non-sensitive data to the desensitized database in real time through master-slave synchronization, and transmit sensitive data to the middleware between the online main database and the desensitized database.

[0106] The working principle and effect of the above technical solution are: preset sensitive data classification criteria, embed the set sensitive data classification criteria into the online main database, and the main database uses the predefined criteria to divide the data into two categories: sensitive data and non-sensitive data. For the data classified as non-sensitive, the system will synchronize these data to the desensitized database in real time through the MySQL master-slave synchronization technology. The desensitized database is a database specifically used to store non-sensitive data. For the data classified as sensitive, the system will not directly synchronize it to the desensitized database, but transmit it to the middleware Kafka between the online main database and the desensitized database. By presetting the classification criteria in the main database, the system can accurately identify and separate sensitive data and non-sensitive data, and take corresponding processing measures. The non-sensitive data is updated to the desensitized database in real time through master-slave synchronization, ensuring the availability and consistency of the data; while the sensitive data is isolated and further processed through the middleware, such as desensitization or encryption, increasing the security. This method not only improves the efficiency of data management, but also reduces the risk of leakage of sensitive information, and is suitable for data processing environments that require high security and privacy protection.

[0107] In an embodiment of the present invention, the tag generation system includes:

[0108] Data analysis system: analyze the type of each original field in the sensitive data, and the types include numbers, strings, dates, and keywords;

[0109] Marking system: generate corresponding desensitization methods according to different original field types, and mark each generated desensitization method with a corresponding desensitization method tag, and the desensitization method tags include DML desensitization and DDL desensitization.

[0110] The working principle and effects of the above technical solution are as follows: Analyze the original types of each field in the sensitive data to identify the basic structure of the data. According to the obtained original field types, generate corresponding desensitization methods for each data type. Add corresponding labels to each generated desensitization method. The desensitization method labels include DML desensitization or DDL desensitization. By analyzing the sensitive data field types and generating corresponding desensitization methods, this process not only improves the accuracy of data privacy protection but also enhances flexibility and applicability. Identifying the field types enables the system to apply specific desensitization strategies for each data type, effectively reducing the risk of information leakage while maintaining the usability and accuracy of the data. This method of managing desensitization strategies by tagging is convenient for maintenance and update, helps to better ensure the compliance and security of sensitive data processing, and provides a robust and adjustable data privacy solution for enterprises.

[0111] In an embodiment of the present invention, the fake data replacement system includes:

[0112] Desensitization method judgment system: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and judges the desensitization method to be adopted for each field according to the desensitization method label;

[0113] Fake data generation system: According to the judgment result, generate fake data of the corresponding type and send the generated fake data to the desensitization library to replace the original sensitive data.

[0114] Among them, the specific judgment method includes:

[0115] The first step: The real-time desensitization program obtains sensitive data from the middleware, and the desensitization method judgment system judges whether the desensitization method label is DML desensitization or DDL desensitization;

[0116] The second step: If it is judged that the desensitization method label is DML desensitization, encrypt the sensitive data and generate a table containing plaintext columns; if it is judged that the desensitization method label is DDL desensitization, reload the desensitization method and perform secondary desensitization on the DDL.

[0117] The working principle and effects of the above technical solution are as follows: The real-time desensitization program extracts sensitive data sent by TiCDC from Kafka. The desensitization method judgment system analyzes the desensitization method tags of each field in the data. The judgment tag is used to determine whether the processing path is for desensitization of DML (Data Manipulation Language) operations or for desensitization of DDL (Data Definition Language) operations. After the judgment system confirms that the desensitization method tag belongs to DML, it will encrypt all sensitive data marked as DML. A table containing both the ciphertext of the sensitive data and the original plaintext column is generated, which provides a basis for subsequent data review or further analysis. If the tag indicates that DDL desensitization is required, the system reloads the desensitization method related to DDL. The desensitization information of the original DDL is processed again to ensure that sensitive information in the structure is not leaked. The fake data replacement system effectively improves data security, prevents unauthorized access and data leakage, while maintaining flexibility and adaptability in data processing in different environments. It supports dynamic adjustment of desensitization policies to meet diverse business needs, and ensures data consistency and repeatability, providing assurance for the compliant processing of sensitive information. By seamlessly integrating with the existing technology stack, the fake data replacement system greatly simplifies the implementation difficulty and cost, and its automation ability enhances the data stream processing efficiency, enabling enterprises to flexibly use data for business analysis and decision-making while ensuring data privacy.

[0118] In one embodiment of the present invention, the recording system includes:

[0119] Data conversion system: Perform MD5 hashing on each original field, and the original field is converted into a hash value after the operation;

[0120] Pseudo-random number generation system: Use the hash value as the seed of the pseudo-random number, and through the pseudo-random number generator, divide the hash value into several fixed-length numbers according to the preset number length;

[0121] Combination system: Piece together several fixed-length numbers to form a complete desensitized data record.

[0122] The working principle and effects of the above technical solution are as follows: First, the data conversion system performs MD5 hashing on each original field. MD5 is an encryption hashing function used to generate a 128-bit fixed-length hash value. After the input value of the original field is converted by MD5, regardless of its original length and content, a fixed hash value will be generated. The generated MD5 hash value is used as the seed for the pseudo-random number generator. This approach enables the same output to be consistently produced for the same input field. Using the pseudo-random number generator, according to the specified scheme and requirements, the generated hash value is further converted into one or more fixed-length numbers. These numbers are the so-called pseudo-random numbers, which logically replace the original sensitive data. And the length of each number is set as needed to match the business requirements. Finally, the combination system is responsible for splicing these independent fixed-length numbers and recombining them into a complete desensitized data record. The pieced-together data is consistent with the original data in structure for continued use in downstream system applications. The formed desensitized data record has a sense of reality and consistency but does not contain any original sensitive information. The recording system ensures that the generated desensitized data is both consistent and does not expose the original information by performing MD5 hashing on the original field and then using the pseudo-random number generation mechanism. The system uses the MD5 hash value as the seed to generate fixed-length pseudo-random numbers, making the desensitized data predictable and consistent each time it is generated, suitable for multiple tests and analyses. In addition, the combination system that integrates various numbers to form a complete record ensures that the desensitized data remains structurally complete and logically coherent. This method not only enhances data privacy protection but also provides realistic and fully functional alternative data for business operations.

[0123] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. A real-time desensitization method for database data, characterized in that: The method comprises: S1: The data in the online main database is divided into sensitive data and non-sensitive data. The non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. The sensitive data is collected and sent to the middleware. S2: Generate a corresponding desensitization method label based on the type of each original field in the sensitive data; S3: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the determination result, generates corresponding type of false data according to the determination result, and sends the false data to the desensitization library. In addition, S3 includes: S31: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; S32: Generate corresponding type of false data according to the judgment result, and send the generated false data to the desensitization library to replace the original sensitive data. The generation method of the false data is obtained by the following formula: A: Get sensitive data P, and the string of P is N, and the fake data replacement system introduces a high-dimensional density matrix , ,... , generate a confusion vector of length N , generate a natural number permutation vector containing 1 to N , introducing multiple nonlinear functions , ..., ; B: According to the matrix vector function introduced in step A, the sensitive data is replaced with false data, and the replacement process is as follows: B1: Convert P into the corresponding ASCII value sequence to obtain a vector ; B2: vector Perform nonlinear transformation with the above multiple nonlinear functions to obtain intermediate change results ,and, The calculation formula is as follows: ； ； ； B3: Combine the intermediate change result with the confusion vector Phase conversion, resulting in confusing changes ,and, The calculation formula is as follows: ； B4: Perform multi-matrix transformation on the mixed change result to obtain the application replacement vector , and the application of the permutation vector The calculation formula is as follows: ； C: Apply the permutation vector As the replaced data, and the vector Send to the desensitization library; S4: Calculate the MD5 value of the original field, use the calculation result as a seed of a pseudo-random number, and generate a fixed number based on the seed of the pseudo-random number, and piece together the generated fixed numbers to form a complete desensitized data record.

2. A method for real-time desensitization of database data according to claim 1, characterized in that: The S1 includes: S11: Preset a sensitive data classification standard, embed the sensitive data classification standard into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standard; S12: Synchronize non-sensitive data to the desensitizing database in real time through master-slave synchronization, and transfer sensitive data to the middleware located between the online master database and the desensitizing database.

3. A method for real-time desensitization of database data according to claim 1, characterized in that: The S2 includes: S21: analyzing the type of each original field in the sensitive data, where the type includes numbers, strings, dates, and keywords; S22: Generate corresponding desensitization methods according to different original field types, and mark each generated desensitization method with a corresponding desensitization method label.

4. A method for real-time desensitization of database data according to claim 1, characterized in that: The S4 includes: S41: Perform MD5 hash operation on each original field, and convert the original field into a hash value after the operation; S42: Using the hash value as a seed of a pseudo-random number, and dividing the hash value into a number of fixed-length numbers according to a preset number length through a pseudo-random number generator; S43: Put together several numbers of fixed length to form a complete anonymized data record.

5. A real-time database data desensitization system, characterized in that: The system comprises: Data transmission system: divides the data in the online main database into sensitive data and non-sensitive data. Non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. Sensitive data is collected and sent to the middleware. Label generation system: Generates corresponding desensitization method labels based on the type of each original field in sensitive data; False data replacement system: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the judgment result, and generates corresponding type of false data according to the judgment result, and sends the false data to the desensitization library, and the false data replacement system includes: Desensitization method judgment system: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; False data generation system: Generates false data of corresponding type according to the judgment result, and sends the generated false data to the desensitization library to replace the original sensitive data. The generation method of the false data is obtained by the following formula: A: Get sensitive data P, and the string of P is N, and the fake data replacement system introduces a high-dimensional density matrix , ,... , generate a confusion vector of length N , generate a natural number permutation vector containing 1 to N , introducing multiple nonlinear functions , ..., ; B: According to the matrix vector function introduced in step A, the sensitive data is replaced with false data, and the replacement process is as follows: B1: Convert P into the corresponding ASCII value sequence to obtain a vector ; B2: vector Perform nonlinear transformation with the above multiple nonlinear functions to obtain intermediate change results ,and, The calculation formula is as follows: ； ； ； B3: Combine the intermediate change result with the confusion vector Phase conversion, resulting in confusing changes ,and, The calculation formula is as follows: ； B4: Perform multi-matrix transformation on the mixed change result to obtain the application replacement vector , and the application of the permutation vector The calculation formula is as follows: ； C: Apply the permutation vector As the replaced data, and the vector Send to the desensitization library; Recording system: calculates the MD5 value of the original field, uses the calculation result as the seed of the pseudo-random number, generates a fixed number based on the seed of the pseudo-random number, and pieced together the generated fixed numbers to form a complete anonymized data record.

6. A database data real-time desensitization system according to claim 5, characterized in that: The data transmission system comprises: Data classification system: preset sensitive data classification standards, embed the sensitive data classification standards into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standards; Data synchronization system: non-sensitive data is synchronized to the desensitizing database in real time through master-slave synchronization, and sensitive data is transmitted to the middleware located between the online main database and the desensitizing database.

7. A database data real-time desensitization system according to claim 5, characterized in that: The label generation system comprises: Data analysis system: analyzes the type of each original field in sensitive data, including numbers, strings, dates, and keywords; Labeling system: Generates corresponding desensitization methods according to different original field types, and marks each generated desensitization method with the corresponding desensitization method label.

8. A database data real-time desensitization system according to claim 5, characterized in that: The recording system comprises: Data conversion system: performs MD5 hash operation on each original field, and the original field is converted into a hash value after the operation; Pseudo-random number generation system: The hash value is used as the seed of the pseudo-random number, and the pseudo-random number generator is used to divide the hash value into several fixed-length numbers according to the preset number length; Combination system: several fixed-length numbers are pieced together to form a complete anonymized data record.

Images