Database data real-time desensitization method and system

Abstract

The invention provides a real-time database data desensitization method and system, relates to the technical field of database information security, and aims to desensitize data in real time according to data produced on line and keep the desensitized data in multiple tables consistent through an algorithm so as to solve the problems that in the prior art, the data are inconsistent before and after desensitization, and the desensitization efficiency is low. The problems that data synchronized to a Perf environment cannot be used due to the fact that the data cannot be desensitized in real time and encryption keys used in the production environment are inconsistent with encryption keys used in the Perf environment in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of database information security, and in particular to a method and system for real-time desensitization of database data. Background Art

[0002] In modern information management, data security and privacy protection have become one of the focuses of attention for enterprises and individuals. With the growth of data and the continuous increase in data flows, data desensitization technology has emerged. Data desensitization is a technology that protects data privacy by replacing, masking, or transforming original sensitive information. Especially in the scenario of troubleshooting in the pre-release environment simulating the production environment, how to ensure the privacy and consistency of data during the use of the pre-production environment has become an important technical challenge; The existing technologies face the following major problems in the process of data desensitization: 1. Data consistency issue: In data scenarios involving multiple tables or multiple systems, although the original data is consistent, the desensitization processing methods in different tables are not unified, resulting in inconsistency in the same data after desensitization, which brings problems to subsequent data analysis and use.

[0003] 2. Performance bottleneck: For scenarios where large amounts of data need to be processed in real time, existing desensitization solutions often desensitize the data once and for all, which cannot meet real-time requirements.

[0004] The encryption keys of the production and Perf environments are inconsistent: The encryption keys used by the production and Perf environments are inconsistent, resulting in the data synchronized to the Perf environment being unusable. Summary of the invention

[0005] In view of the above technical problems, the present invention proposes a method and system for real-time desensitization of database data, and the technical solutions adopted are as follows: A real-time desensitization method for database data, the method comprising: S1: The data in the online main database is divided into sensitive data and non-sensitive data. The non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. The sensitive data is collected and sent to the middleware. S2: Generate a corresponding desensitization method label based on the type of each original field in the sensitive data; S3: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the determination result, generates corresponding type of false data according to the determination result, and sends the false data to the desensitization library; S4: Calculate the MD5 value of the original field, use the calculation result as a seed of a pseudo-random number, and generate a fixed number based on the seed of the pseudo-random number, and piece together the generated fixed numbers to form a complete desensitized data record.

[0006] Preferably, the S1 includes: S11: Preset a sensitive data classification standard, embed the classification label into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standard; S12: Synchronize non-sensitive data to the desensitizing database in real time through master-slave synchronization, and transfer sensitive data to the middleware located between the online master database and the desensitizing database.

[0007] Preferably, the S2 includes: S21: analyzing the type of each original field in the sensitive data, where the type includes numbers, strings, dates, and keywords; S22: Generate corresponding desensitization methods according to different original field types, and mark each generated desensitization method with a corresponding desensitization method label.

[0008] Preferably, S3 includes: S31: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; S32: Generate corresponding type of false data according to the judgment result, and send the generated false data to the desensitization library to replace the original sensitive data. The generation method of the false data is obtained by the following formula: A: Get sensitive data P, and the string of P is N, and the fake data replacement system introduces a high-dimensional density matrix, , ..., Generate a confusion vector of length N , generate a natural number permutation vector containing 1 to N , introducing multiple nonlinear functions , ..., ; B: According to the matrix vector function introduced in step A, the sensitive data is replaced with false data, and the replacement process is as follows: B1: Convert P into the corresponding ASCII value sequence to obtain a vector ; B2: vector Perform nonlinear transformation with the above multiple nonlinear functions to obtain intermediate change results ,and, The calculation formula is as follows: ; ; ; B3: Combine the intermediate change result with the confusion vector Phase conversion, resulting in confusing changes ,and, The calculation formula is as follows: ; B4: Perform multi-matrix transformation on the mixed change result to obtain the application replacement vector , and the application of the permutation vector The calculation formula is as follows: ; C: Apply the permutation vector As the replaced data, and the vector Send to the desensitization library.

[0009] Preferably, the S4 includes: S41: Perform MD5 hash operation on each original field, and convert the original field into a hash value after the operation; S42: Using the hash value as a seed of a pseudo-random number, and dividing the hash value into a number of fixed-length numbers according to a preset number length through a pseudo-random number generator; S43: Put together several numbers of fixed length to form a complete anonymized data record.

[0010] A real-time desensitization system for database data, the system comprising: Data transmission system: divides the data in the online main database into sensitive data and non-sensitive data. Non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. Sensitive data is collected and sent to the middleware. Label generation system: Generates corresponding desensitization method labels based on the type of each original field in sensitive data; False data replacement system: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the determination result, generates corresponding type of false data according to the determination result, and sends the false data to the desensitization library; Recording system: calculates the MD5 value of the original field, uses the calculation result as the seed of the pseudo-random number, generates a fixed number based on the seed of the pseudo-random number, and pieced together the generated fixed numbers to form a complete anonymized data record.

[0011] Preferably, the data transmission system comprises: Data classification system: Preset sensitive data classification standards, embed the classification labels into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standards; Data synchronization system: non-sensitive data is synchronized to the desensitizing database in real time through master-slave synchronization, and sensitive data is transmitted to the middleware located between the online main database and the desensitizing database.

[0012] Preferably, the label generation system comprises: Data analysis system: analyzes the type of each original field in sensitive data, including numbers, strings, dates, and keywords; Labeling system: Generates corresponding desensitization methods according to different original field types, and marks each generated desensitization method with the corresponding desensitization method label.

[0013] Preferably, the false data replacement system comprises: Desensitization method judgment system: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; False data generation system: Generates false data of the corresponding type based on the judgment results, and sends the generated false data to the desensitizing library to replace the original sensitive data.

[0014] Preferably, the recording system comprises: Data conversion system: performs MD5 hash operation on each original field, and the original field is converted into a hash value after the operation; Pseudo-random number generation system: The hash value is used as the seed of the pseudo-random number, and the pseudo-random number generator is used to divide the hash value into several fixed-length numbers according to the preset number length; Combination system: several fixed-length numbers are pieced together to form a complete anonymized data record.

[0015] The beneficial effects of the present invention are as follows: the data can be desensitized in real time according to the data produced online, and the consistency of the desensitized data in multiple tables is guaranteed to ensure the normal operation of the business in the desensitized library environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A real-time database data desensitization system according to the present invention; Figure 2 This is the desensitization method judgment system described in the present invention. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0018] One embodiment of the present invention provides a real-time database data desensitization method, the method comprising: S1: The data in the online main database is divided into sensitive data and non-sensitive data. The non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. The sensitive data is collected and sent to the middleware. S2: Generate a corresponding desensitization method label based on the type of each original field in the sensitive data; S3: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the determination result, generates corresponding type of false data according to the determination result, and sends the false data to the desensitization library; S4: Calculate the MD5 value of the original field, use the calculation result as a seed of a pseudo-random number, and generate a fixed number based on the seed of the pseudo-random number, and piece together the generated fixed numbers to form a complete desensitized data record.

[0019] The working principle and effect of the above technical solution are as follows: sensitive data includes personal identity information, financial data, customer records, etc. Once leaked, these data may pose a threat to personal privacy or corporate security. Non-sensitive data refers to data that will not pose a major threat to individuals or enterprises even if leaked. Non-sensitive data is synchronized directly from the online main database to the desensitizing database in real time through MySQL master-slave synchronization technology. Non-sensitive data is synchronized to the desensitizing database in real time through MySQL master-slave synchronization technology, reducing the complexity of data migration and processing. Sensitive data is collected from TiDB by TiCDC and sent to Kafka. Sensitive data is collected and sent to Kafka through TiCDC, which realizes data isolation and encrypted transmission, reducing the risk of data leakage. Kafka, as a message middleware, is responsible for real-time transmission of data streams to ensure real-time collection and transmission of sensitive data. For each identified sensitive data field, its data type and purpose are analyzed. A desensitization method label is generated according to the type of field and business needs. The desensitization method label includes DML desensitization or DDL desensitization. The desensitized data will be written to the downstream desensitization library. In order to ensure that the desensitized data is consistent when generating false data, the present invention calculates the MD5 value according to the original field. The MD5 value is used as the seed of a pseudo-random number to generate a fixed number. The selected false data is pieced together to form a desensitized data record. By using the MD5 value as the seed of a pseudo-random number, it can be ensured that the false data generated under the same original field is consistent. The present invention can desensitize the data in real time according to the data produced online, and ensures the consistency of the desensitized data in multiple tables to ensure that the business runs normally in the desensitized library environment.

[0020] In one embodiment of the present invention, the S1 includes: S11: Preset a sensitive data classification standard, embed the classification label into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standard; S12: Synchronize non-sensitive data to the desensitizing database in real time through master-slave synchronization, and transfer sensitive data to the middleware located between the online master database and the desensitizing database.

[0021] The working principle and effect of the above technical solution are as follows: preset sensitive data classification standards, embed the set sensitive data classification standards into the online main library, and the main database uses the pre-defined standards to divide the data into two categories: sensitive data and non-sensitive data. For data classified as non-sensitive, the system will synchronize these data to the desensitization library in real time through the MySQL master-slave synchronization technology. The desensitization library is a database specifically used to store non-sensitive data. For data classified as sensitive, the system will not directly synchronize it to the desensitization library, but transfer it to the middleware Kafka located between the online main library and the desensitization library. By presetting the classification standards in the main library, the system can accurately identify and separate sensitive data from non-sensitive data, and take corresponding processing measures. Non-sensitive data is updated to the desensitization library in real time through master-slave synchronization, ensuring the availability and consistency of data; while sensitive data is isolated and further processed by the middleware, such as desensitization or encryption, which increases security. This method not only improves the efficiency of data management, but also reduces the risk of sensitive information leakage, and is suitable for data processing environments that require high security and privacy protection.

[0022] In one embodiment of the present invention, S2 includes: S21: analyzing the type of each original field in the sensitive data, where the type includes numbers, strings, dates, and keywords; S22: Generate corresponding desensitization methods according to different original field types, and mark each generated desensitization method with a corresponding desensitization method label.

[0023] The working principle and effect of the above technical solution are as follows: the original type of each field in the sensitive data is analyzed to identify the basic structure of the data. According to the original field type obtained by the analysis, a corresponding desensitization method is generated for each data type. A corresponding label is added to each generated desensitization method. The desensitization method label includes DML desensitization or DDL desensitization. By analyzing the sensitive data field type and generating the corresponding desensitization method, this process not only improves the accuracy of data privacy protection, but also enhances flexibility and applicability. Identifying the field type enables the system to apply a specific desensitization strategy for each data type, effectively reducing the risk of information leakage while maintaining the availability and accuracy of the data. This method manages desensitization strategies in a labeled manner, facilitates maintenance and updating, helps to better ensure the compliance and security of sensitive data processing, and provides enterprises with a robust and adjustable data privacy solution.

[0024] In one embodiment of the present invention, S3 includes: S31: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; S32: Generate corresponding type of false data according to the judgment result, and send the generated false data to the desensitization library to replace the original sensitive data. The generation method of the false data is obtained by the following formula: A: Get sensitive data P, and the string of P is N, and the fake data replacement system introduces a high-dimensional density matrix, , ..., Generate a confusion vector of length N , generate a natural number permutation vector containing 1 to N , introducing multiple nonlinear functions , ..., ; B: According to the matrix vector function introduced in step A, the sensitive data is replaced with false data, and the replacement process is as follows: B1: Convert P into the corresponding ASCII value sequence to obtain a vector ; B2: vector Perform nonlinear transformation with the above multiple nonlinear functions to obtain intermediate change results ,and, The calculation formula is as follows: ; ; ; B3: Combine the intermediate change result with the confusion vector Phase conversion, resulting in confusing changes ,and, The calculation formula is as follows: ; B4: Perform multi-matrix transformation on the mixed change result to obtain the application replacement vector , and the application of the permutation vector The calculation formula is as follows: ; C: Apply the permutation vector As the replaced data, and the vector Send to the desensitization library.

[0025] The working principle and effect of the above technical solution are: the replacement process is performed using a replacement algorithm. The core of this algorithm is to convert the original data into an encrypted data that is difficult to reversely analyze through a series of mathematical transformation processes. In this process, mathematical tools such as high-dimensional density matrices, confusion vectors, and nonlinear functions are introduced to ensure that the encrypted data is both safe and difficult to crack. The specific encryption process includes the following steps: highly sensitive data (set as P, whose string length is N) is converted into a corresponding ASCII value sequence to obtain a vector. This vector is nonlinearly transformed with a series of nonlinear functions to generate an intermediate change result. The intermediate change result is combined with the confusion vector for further confusion transformation. The obfuscated result is subjected to a multi-matrix transformation, and finally an application replacement vector is obtained, which is the replaced data.

[0026] Through the above replacement processing, even if the data is leaked, it is difficult for attackers to obtain the original sensitive information from it. The system uses a shielding algorithm for processing. The core of this algorithm is to mask or replace the key parts of sensitive information so that the original information cannot be directly identified. For example, when processing ID card information, only some numbers or characters can be retained, and the rest can be replaced with asterisks * or other characters. Similar masking strategies can also be used when processing sensitive texts such as medical records or passwords. The system will create a special highly sensitive database based on the highly sensitive data after desensitization. This database is set as the data output end to store and manage desensitized sensitive data. At the same time, the system will also perform detailed parameter configuration for this database, including primary keys, foreign keys, data formats and types, and performance optimization parameters.

[0027] These parameter configurations can ensure the stable operation and efficient access of the database, and also facilitate the management and maintenance of desensitized data by data managers. By statically desensitizing highly sensitive data, it can be ensured that these sensitive information will not be leaked during the storage process, thereby greatly improving the security of the data. Even if the database is attacked or leaked, the attacker cannot directly obtain the original highly sensitive data, but can only obtain encrypted or masked data, which greatly reduces the risk of data leakage. During the desensitization process, the system retains the original characteristics and structure of the data as much as possible, so that the desensitized data can still maintain its usability to a certain extent and meet the needs of data analysis, mining, etc. The system supports different desensitization processes for different types of sensitive data. In one embodiment of the present invention, the S4 includes: S41: Perform MD5 hash operation on each original field, and convert the original field into a hash value after the operation; S42: Using the hash value as a seed of a pseudo-random number, and dividing the hash value into a number of fixed-length numbers according to a preset number length through a pseudo-random number generator; S43: Put together several numbers of fixed length to form a complete anonymized data record.

[0028] The working principle and effect of the above technical solution are as follows: First, the data conversion system performs MD5 hash operation on each original field. MD5 is a cryptographic hash function used to generate a 128-bit fixed-length hash value. After the input value of the original field is converted by MD5, a fixed hash value will be generated regardless of its original length and content. The above-generated MD5 hash value is used as the seed of the pseudo-random number generator. This approach allows the same output to be generated consistently for the same input field. Using the pseudo-random number generator, the generated hash value is further converted into one or more fixed-length numbers according to the specified scheme and requirements. These numbers are so-called pseudo-random numbers, which logically replace the original sensitive data. And the length of each number is set as needed to match the business requirements. Finally, the combination system is responsible for splicing these independent fixed-length numbers and recombining them into a complete desensitized data record. The spliced ​​data is structurally consistent with the original data so that it can continue to be used in downstream system applications. The desensitized data record formed in this way has a sense of reality and consistency, but does not contain any original sensitive information. The recording system performs MD5 hash operations on the original fields and then uses a pseudo-random number generation mechanism to ensure that the generated desensitized data is consistent and does not expose the original information. The system uses the MD5 hash value as a seed to generate a pseudo-random number of a fixed length, so that the desensitized data remains predictable and consistent each time it is generated, which is suitable for multiple tests and analyses. In addition, a combination system that integrates various numbers to form a complete record ensures that the desensitized data maintains structural integrity and logical coherence. This method not only improves data privacy protection, but also provides realistic and fully functional alternative data for business operations.

[0029] One embodiment of the present invention provides a real-time database data desensitization system, the system comprising: Data transmission system: divides the data in the online main database into sensitive data and non-sensitive data. Non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. Sensitive data is collected and sent to the middleware. Label generation system: Generates corresponding desensitization method labels based on the type of each original field in sensitive data; False data replacement system: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the determination result, generates corresponding type of false data according to the determination result, and sends the false data to the desensitization library; Recording system: calculates the MD5 value of the original field, uses the calculation result as the seed of the pseudo-random number, generates a fixed number based on the seed of the pseudo-random number, and pieced together the generated fixed numbers to form a complete anonymized data record.

[0030] The working principle and effect of the above technical solution are as follows: sensitive data includes personal identity information, financial data, customer records, etc. Once leaked, these data may pose a threat to personal privacy or corporate security. Non-sensitive data refers to data that will not pose a major threat to individuals or enterprises even if leaked. Non-sensitive data is synchronized directly from the online main database to the desensitizing database in real time through MySQL master-slave synchronization technology. Non-sensitive data is synchronized to the desensitizing database in real time through MySQL master-slave synchronization technology, reducing the complexity of data migration and processing. Sensitive data is collected from TiDB by TiCDC and sent to Kafka. Sensitive data is collected and sent to Kafka through TiCDC, which realizes data isolation and encrypted transmission, reducing the risk of data leakage. Kafka, as a message middleware, is responsible for real-time transmission of data streams to ensure real-time collection and transmission of sensitive data. For each identified sensitive data field, its data type and purpose are analyzed. A desensitization method label is generated according to the type of field and business needs. The desensitization method label includes DML desensitization or DDL desensitization. The desensitized data will be written to the downstream desensitization library. In order to ensure that the desensitized data is consistent when generating false data, the present invention calculates the MD5 value according to the original field. The MD5 value is used as the seed of a pseudo-random number to generate a fixed number. The selected false data is pieced together to form a desensitized data record. By using the MD5 value as the seed of a pseudo-random number, it can be ensured that the false data generated under the same original field is consistent. The present invention can desensitize the data in real time according to the data produced online, and ensures the consistency of the desensitized data in multiple tables to ensure that the business runs normally in the desensitized library environment.

[0031] In one embodiment of the present invention, the data transmission system comprises: Data classification system: Preset sensitive data classification standards, embed the classification labels into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standards; Data synchronization system: non-sensitive data is synchronized to the desensitizing database in real time through master-slave synchronization, and sensitive data is transmitted to the middleware located between the online main database and the desensitizing database.

[0032] The working principle and effect of the above technical solution are as follows: preset sensitive data classification standards, embed the set sensitive data classification standards into the online main library, and the main database uses the pre-defined standards to divide the data into two categories: sensitive data and non-sensitive data. For data classified as non-sensitive, the system will synchronize these data to the desensitization library in real time through the MySQL master-slave synchronization technology. The desensitization library is a database specifically used to store non-sensitive data. For data classified as sensitive, the system will not directly synchronize it to the desensitization library, but transfer it to the middleware Kafka located between the online main library and the desensitization library. By presetting the classification standards in the main library, the system can accurately identify and separate sensitive data from non-sensitive data, and take corresponding processing measures. Non-sensitive data is updated to the desensitization library in real time through master-slave synchronization, ensuring the availability and consistency of data; while sensitive data is isolated and further processed by the middleware, such as desensitization or encryption, which increases security. This method not only improves the efficiency of data management, but also reduces the risk of sensitive information leakage, and is suitable for data processing environments that require high security and privacy protection.

[0033] In one embodiment of the present invention, the label generation system includes: Data analysis system: analyzes the type of each original field in sensitive data, including numbers, strings, dates, and keywords; Marking system: Generate corresponding desensitization methods according to different original field types, and mark each generated desensitization method with a corresponding desensitization method label, and the desensitization method label includes DML desensitization and DDL desensitization.

[0034] The working principle and effect of the above technical solution are as follows: the original type of each field in the sensitive data is analyzed to identify the basic structure of the data. According to the original field type obtained by the analysis, a corresponding desensitization method is generated for each data type. A corresponding label is added to each generated desensitization method. The desensitization method label includes DML desensitization or DDL desensitization. By analyzing the sensitive data field type and generating the corresponding desensitization method, this process not only improves the accuracy of data privacy protection, but also enhances flexibility and applicability. Identifying the field type enables the system to apply a specific desensitization strategy for each data type, effectively reducing the risk of information leakage while maintaining the availability and accuracy of the data. This method manages desensitization strategies in a labeled manner, facilitates maintenance and updating, helps to better ensure the compliance and security of sensitive data processing, and provides enterprises with a robust and adjustable data privacy solution.

[0035] In one embodiment of the present invention, the false data replacement system comprises: Desensitization method judgment system: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; False data generation system: Generates false data of the corresponding type based on the judgment results, and sends the generated false data to the desensitizing library to replace the original sensitive data.

[0036] The specific judgment methods include: Step 1: The real-time desensitization program obtains sensitive data from the middleware, and the desensitization method determination system determines whether the desensitization method label is DML desensitization or DDL desensitization; Step 2: If the desensitization method label is DML desensitization, the sensitive data is encrypted and a table with a plaintext column is generated; if the desensitization method label is DDL desensitization, the desensitization method is reloaded and the DDL is desensitized again.

[0037] The working principle and effect of the above technical solution are as follows: The real-time desensitization program extracts sensitive data sent by TiCDC from Kafka. The desensitization method judgment system analyzes the desensitization method label of each field in the data. The label is judged to determine whether the processing path is for desensitization of DML (Data Manipulation Language) operations or for desensitization of DDL (Data Definition Language) operations. After the judgment system confirms that the desensitization method label belongs to DML, it will encrypt all sensitive data marked as DML. A table containing both sensitive data ciphertext and original plaintext columns is generated, which provides a basis for subsequent data review or further analysis. If the label indicates that DDL desensitization is required, the system reloads the desensitization method related to DDL. The desensitization information of the original DDL is processed twice to ensure that sensitive information in the structure will not be leaked. The fake data replacement system effectively improves data security, prevents unauthorized access and data leakage, and maintains flexibility and adaptability in data processing in different environments. It supports dynamic adjustment of desensitization strategies to meet diverse business needs, ensures data consistency and repeatability, and provides guarantees for the compliance processing of sensitive information. By seamlessly integrating with the existing technology stack, the fake data replacement system greatly simplifies the implementation difficulty and cost. Its automation capabilities enhance the efficiency of data stream processing, enabling enterprises to flexibly use data for business analysis and decision-making while ensuring data privacy.

[0038] In one embodiment of the present invention, the recording system comprises: Data conversion system: performs MD5 hash operation on each original field, and the original field is converted into a hash value after the operation; Pseudo-random number generation system: The hash value is used as the seed of the pseudo-random number, and the hash value is divided into several fixed-length numbers according to the preset number length through the pseudo-random number generator; Combination system: several fixed-length numbers are pieced together to form a complete anonymized data record.

[0039] The working principle and effect of the above technical solution are as follows: First, the data conversion system performs MD5 hash operation on each original field. MD5 is a cryptographic hash function used to generate a 128-bit fixed-length hash value. After the input value of the original field is converted by MD5, a fixed hash value will be generated regardless of its original length and content. The above-generated MD5 hash value is used as the seed of the pseudo-random number generator. This approach allows the same output to be generated consistently for the same input field. Using the pseudo-random number generator, the generated hash value is further converted into one or more fixed-length numbers according to the specified scheme and requirements. These numbers are so-called pseudo-random numbers, which logically replace the original sensitive data. And the length of each number is set as needed to match the business requirements. Finally, the combination system is responsible for splicing these independent fixed-length numbers and recombining them into a complete desensitized data record. The spliced ​​data is structurally consistent with the original data so that it can continue to be used in downstream system applications. The desensitized data record formed in this way has a sense of reality and consistency, but does not contain any original sensitive information. The recording system performs MD5 hash operations on the original fields and then uses a pseudo-random number generation mechanism to ensure that the generated desensitized data is consistent and does not expose the original information. The system uses the MD5 hash value as a seed to generate a pseudo-random number of a fixed length, so that the desensitized data remains predictable and consistent each time it is generated, which is suitable for multiple tests and analyses. In addition, a combination system that integrates various numbers to form a complete record ensures that the desensitized data maintains structural integrity and logical coherence. This method not only improves data privacy protection, but also provides realistic and fully functional alternative data for business operations.

[0040] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time desensitization method for database data, characterized in that: The method comprises: S1: The data in the online main database is divided into sensitive data and non-sensitive data. The non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. The sensitive data is collected and sent to the middleware. S2: Generate a corresponding desensitization method label based on the type of each original field in the sensitive data; S3: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the determination result, generates corresponding type of false data according to the determination result, and sends the false data to the desensitization library; S4: Calculate the MD5 value of the original field, use the calculation result as a seed of a pseudo-random number, and generate a fixed number based on the seed of the pseudo-random number, and piece together the generated fixed numbers to form a complete desensitized data record.

2. A method for real-time desensitization of database data according to claim 1, characterized in that: The S1 includes: S11: Preset a sensitive data classification standard, embed the classification label into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standard; S12: Synchronize non-sensitive data to the desensitizing database in real time through master-slave synchronization, and transfer sensitive data to the middleware located between the online master database and the desensitizing database.

3. A method for real-time desensitization of database data according to claim 1, characterized in that: The S2 includes: S21: analyzing the type of each original field in the sensitive data, where the type includes numbers, strings, dates, and keywords; S22: Generate corresponding desensitization methods according to different original field types, and mark each generated desensitization method with a corresponding desensitization method label.

4. A method for real-time desensitization of database data according to claim 1, characterized in that: The S3 includes: S31: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; S32: Generate corresponding type of false data according to the judgment result, and send the generated false data to the desensitization library to replace the original sensitive data. The generation method of the false data is obtained by the following formula: A: Get sensitive data P, and the string of P is N, and the fake data replacement system introduces a high-dimensional density matrix, , ..., Generate a confusion vector of length N , generate a natural number permutation vector containing 1 to N , introducing multiple nonlinear functions , ..., ; B: According to the matrix vector function introduced in step A, the sensitive data is replaced with false data, and the replacement process is as follows: B1: Convert P into the corresponding ASCII value sequence to obtain a vector ; B2: vector Perform nonlinear transformation with the above multiple nonlinear functions to obtain intermediate change results ,and, The calculation formula is as follows: ； ； ； B3: Combine the intermediate change result with the confusion vector Phase conversion, resulting in confusing changes ,and, The calculation formula is as follows: ； B4: Perform multi-matrix transformation on the mixed change result to obtain the application replacement vector , and the application of the permutation vector The calculation formula is as follows: ； C: Apply the permutation vector As the replaced data, and the vector Send to the desensitization library.

5. A method for real-time desensitization of database data according to claim 1, characterized in that: The S4 includes: S41: Perform MD5 hash operation on each original field, and convert the original field into a hash value after the operation; S42: Using the hash value as a seed of a pseudo-random number, and dividing the hash value into a number of fixed-length numbers according to a preset number length through a pseudo-random number generator; S43: Put together several numbers of fixed length to form a complete anonymized data record.

6. A real-time database data desensitization system, characterized in that: The system comprises: Data transmission system: divides the data in the online main database into sensitive data and non-sensitive data. Non-sensitive data is synchronized from the online main database to the desensitizing database in real time through master-slave synchronization. Sensitive data is collected and sent to the middleware. Label generation system: Generates corresponding desensitization method labels based on the type of each original field in sensitive data; False data replacement system: The real-time desensitization program obtains sensitive data from the middleware, and determines the desensitization method of each field according to the desensitization method label, obtains the determination result, generates corresponding type of false data according to the determination result, and sends the false data to the desensitization library; Recording system: calculates the MD5 value of the original field, uses the calculation result as the seed of the pseudo-random number, generates a fixed number based on the seed of the pseudo-random number, and pieced together the generated fixed numbers to form a complete anonymized data record.

7. A database data real-time desensitization system according to claim 6, characterized in that: The data transmission system comprises: Data classification system: Preset sensitive data classification standards, embed the classification labels into the online main database, and the online main database divides the original data in the online main database into sensitive data and non-sensitive data according to the classification standards; Data synchronization system: non-sensitive data is synchronized to the desensitizing database in real time through master-slave synchronization, and sensitive data is transmitted to the middleware located between the online main database and the desensitizing database.

8. A database data real-time desensitization system according to claim 6, characterized in that: The label generation system comprises: Data analysis system: analyzes the type of each original field in sensitive data, including numbers, strings, dates, and keywords; Labeling system: Generates corresponding desensitization methods according to different original field types, and marks each generated desensitization method with the corresponding desensitization method label.

9. A database data real-time desensitization system according to claim 6, characterized in that: The false data replacement system comprises: Desensitization method judgment system: The real-time desensitization program obtains sensitive data from the middleware, encrypts the sensitive data, and determines the desensitization method to be adopted for each field based on the desensitization method label; False data generation system: Generates false data of the corresponding type based on the judgment results, and sends the generated false data to the desensitizing library to replace the original sensitive data.

10. A database data real-time desensitization system according to claim 6, characterized in that: The recording system includes: Data conversion system: performs MD5 hash operation on each original field, and the original field is converted into a hash value after the operation; Pseudo-random number generation system: The hash value is used as the seed of the pseudo-random number, and the pseudo-random number generator is used to divide the hash value into several fixed-length numbers according to the preset number length; Combination system: several fixed-length numbers are pieced together to form a complete anonymized data record.

Images