Incremental updating method, system and equipment based on real-time change detection and medium

Abstract

The invention discloses an incremental updating method, system and device based on real-time change detection and a medium. Comprising the steps that a binary log Binlog of a database is monitored, a DDL operation statement of the database is obtained in real time, and the DDL operation statement is analyzed to generate a field change event; based on the field change event, creating and executing an incremental scanning task for the change field; determining a database name, a table name and a field name in the field change event and an operation timestamp associated with the field change event as retrieval keywords, and retrieving a change approval work order within a preset time range before and after the operation timestamp in a change management system to obtain a change approval work order associated with a change field; and determining the semantic relevancy between the field sensitive type in the classification and grading result and the application reason text recorded in the change approval work order, and if the semantic relevancy reaches a preset relevancy threshold, automatically confirming the classification and grading result. The data security of the sensitive data can be improved.

Description

Technical Field

[0001] This application relates to the fields of data security and database management technology, and in particular to an incremental update method, system, device and medium based on real-time change detection. Background Technology

[0002] In the digital economy era, data has become a core production factor, and its security management is directly related to personal privacy, corporate core competitiveness, and even national security. The leakage of sensitive data (such as personal identification information, financial account details, and trade secrets) can lead to serious economic losses, legal risks, and reputational damage. Therefore, in the context of the internet, strict compliance requirements have been imposed on data processing activities, and enterprises must ensure that sensitive data is effectively protected throughout its entire lifecycle, from collection and storage to transmission, use, and destruction.

[0003] Currently, mainstream sensitive data management solutions mainly rely on traditional data classification and grading methods. Their core implementation methods include two categories: periodic full scans and manual rule maintenance. Periodic full scans, which perform periodic (daily, weekly, or monthly) scans of the entire database, while covering the entire database, suffer from long scan cycles, leading to delays and ineffective, untimely data security protection. Manual rule maintenance, on the other hand, heavily relies on human experience, making it prone to oversights and errors. Furthermore, manual operation struggles to handle high-frequency change scenarios, potentially causing classification results to misalign with actual business intent, resulting in false positives or false negatives, thus failing to effectively protect data security. Summary of the Invention

[0004] In view of this, embodiments of this application provide an incremental update method, system, device, medium, and product based on real-time change detection, which can improve the efficiency and accuracy of sensitive data identification, thereby enhancing its data security.

[0005] In a first aspect, embodiments of this application provide an incremental update method based on real-time change detection. The method includes: monitoring the binary log (Binlog) of a database to obtain data definition language (DDL) operation statements in real time; parsing the DDL operation statements to extract the database name, table name, field name, and change type corresponding to the changed field, generating a field change event; creating and executing an incremental scan task for the changed field based on the field change event, and calling a sensitive data identification algorithm to output the classification and grading results of the changed field; and combining the database name, table name, field name, and operation timestamp associated with the field change event into the data definition language (DDL) operation statements. The system identifies the search keywords and retrieves change approval work orders within a preset time range before and after the operation timestamp in the change management system based on these keywords, obtaining the change approval work orders associated with the change fields. It then determines the semantic relevance between the sensitivity type of the fields in the classification and grading results and the application reason text recorded in the change approval work orders. If the semantic relevance reaches a preset relevance threshold, the classification and grading results are automatically confirmed and written into the metadata corresponding to the change fields in the database. If the semantic relevance is lower than the preset relevance threshold, the classification and grading results are marked as pending confirmation, and an alarm notification is generated and sent to the data security administrator for manual review.

[0006] Secondly, embodiments of this application provide an incremental update system based on real-time change detection. This system includes: a generation module, used to acquire data definition language (DDL) operation statements from the database in real time by monitoring the database's binary log (Binlog), and extract the database name, table name, field name, and change type corresponding to the changed field by parsing the DDL operation statements to generate a field change event; a creation and execution module, used to create and execute an incremental scan task for the changed field based on the field change event, and to call a sensitive data identification algorithm to output the classification and grading results of the changed field; and a retrieval module, used to retrieve the database name, table name, field name, and field change event information from the field change event. The operation timestamp is used as the retrieval keyword, and the change approval work orders within a preset time range before and after the operation timestamp are retrieved in the change management system based on the retrieval keyword to obtain the change approval work orders associated with the change field. The result confirmation module is used to determine the semantic relevance between the field sensitivity type in the classification and grading results and the application reason text recorded in the change approval work order. If the semantic relevance reaches the preset relevance threshold, the classification and grading results are automatically confirmed and written into the metadata corresponding to the change field in the database. If the semantic relevance is lower than the preset relevance threshold, the classification and grading results are marked as pending confirmation and an alarm notification is generated and sent to the data security administrator for manual review.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the steps of the incremental update method based on real-time change detection as described in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the incremental update method based on real-time change detection as described in the first aspect.

[0009] Fifthly, embodiments of this application provide a computer program product stored in a non-volatile storage medium, which, when executed by a processor, implements the steps of the incremental update method based on real-time change detection as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the steps of the incremental update method based on real-time change detection as described in the first aspect.

[0011] This application provides an incremental update method, system, device, medium, and product based on real-time change detection. It utilizes MySQL's built-in binlog service to achieve low-intrusive monitoring of real-time changes to data table structures and improves the coverage of change monitoring. This enables immediate detection of database structure changes, mitigating the lag issues of traditional periodic full scans and shortening the risk window for sensitive data exposure. Furthermore, it creates and executes incremental scan tasks targeting changed fields based on field change events. This task only scans and identifies sensitive data in the changed fields, avoiding the computationally intensive and time-consuming full scans of traditional methods, thus reducing system resource consumption and improving detection efficiency. Furthermore, by linking with database change approval records, an automated verification mechanism is introduced. This mechanism retrieves associated change approval work orders from the change management system and determines the semantic relevance between the sensitive types of fields in the classification and grading results and the application reason text recorded in the change approval work orders. This allows for cross-verification of automated detection results with application records. If the semantic relevance reaches a preset threshold, the classification and grading results are automatically confirmed; otherwise, they are marked as pending confirmation, triggering an alarm notification for manual review. This process effectively reduces false alarm rates and improves the accuracy of classification and grading results. Finally, by writing the confirmed results into the metadata corresponding to the database change fields, accurate evidence is provided for the real-time effectiveness of data security policies. Thus, this application, through real-time sensing, incremental scanning, automated verification, and result write-back technologies, not only improves the efficiency and accuracy of sensitive data identification but also enhances the data security of sensitive data throughout its entire lifecycle from multiple dimensions, including shortening the risk window, reducing false alarm interference, and providing precise policy basis. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings of the embodiments of this application will be briefly described below.

[0013] Figure 1 This is a flowchart illustrating an incremental update method based on real-time change detection provided in an embodiment of this application;

[0014] Figure 2 This is an exemplary flowchart of an incremental update method based on real-time change detection provided in an embodiment of this application;

[0015] Figure 3 This is a schematic diagram of the structure of an incremental update system based on real-time change detection provided in an embodiment of this application;

[0016] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The principles and spirit of this application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided to make the principles and spirit of this application clearer and more thorough, enabling those skilled in the art to better understand and implement the principles and spirit of this application. The exemplary embodiments provided herein are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described herein without inventive effort are within the scope of protection of this application.

[0018] In this document, terms such as first, second, and third are used only to distinguish one entity (or operation) from another, and are not intended to require or imply any order or relationship between these entities (or operations).

[0019] The following is a brief description of the concepts and technical terms that may be involved in the embodiments of this application.

[0020] Binary log is a core service provided by the MySQL database engine. It continuously records all operations that modify the database content or structure in binary form, such as INSERT, UPDATE, DELETE, ALTERTABLE, etc.

[0021] Data Definition Language (DDL) is a language within SQL responsible for defining data structures and database objects. Common commands include: CREATE (create databases, tables, indexes, etc.), ALTER (modify the structure of existing database objects), DROP (delete database objects), and TRUNCATE (clear table contents).

[0022] Port mirroring is a feature provided by network switches that can copy all network traffic from a specified port to another monitoring port, thereby enabling lossless capture and analysis of network traffic without affecting the forwarding of the original service traffic.

[0023] An auditing plugin is a software component installed on a database server that can hook all database operations and record this information in a log or forward it directly to the auditing system.

[0024] COMMENT refers to a metadata annotation feature provided by a database management system. It allows descriptive text to be added to databases, tables, or fields. This text is stored in system tables as part of the metadata and does not directly affect the data itself, but it can be queried and displayed.

[0025] Metadata is used to describe the attributes of data itself. In the database field, it typically includes information such as table name, field name, data type, length, constraints, and comments.

[0026] Before describing the technical solutions provided in the embodiments of this application, in order to facilitate understanding of the embodiments of this application, this application first specifically explains the problems existing in the related technologies:

[0027] Currently, mainstream sensitive data management solutions mainly rely on traditional data classification and grading methods, with core implementation methods including regular full-volume scanning and manual rule maintenance.

[0028] Regular full-database scan solution: This solution performs a full database scan periodically (e.g., daily, weekly, or monthly), using a rule engine or machine learning model to identify sensitive data fields and update the classification and grading results. While this method covers the entire database, it has drawbacks: First, the scan cycle is long, making it unable to respond in real-time to frequent changes in the database structure (e.g., adding fields, modifying field types, etc.). Sensitive data identification lags behind database field changes, resulting in insufficient real-time timeliness of classification and grading results, affecting the accuracy of data security strategies. This leads to a "risk window" of several hours or even days between field creation and identification and control, during which sensitive data is unprotected and has low security. Second, it consumes enormous resources, especially for large databases (e.g., terabytes of data). Each full scan requires significant computing resources and time (e.g., some enterprises take up to two weeks per scan), resulting in severe inefficiency and impacting business system performance.

[0029] Manual rule maintenance solution: This solution relies on security administrators manually monitoring database change logs or change approval forms and manually updating classification rules or policies. While this method offers some flexibility, it heavily depends on human experience and is prone to oversights and errors. Furthermore, manual operation struggles to handle high-frequency change scenarios and cannot promptly respond to real-time changes to database fields (such as adding fields or modifying field usage), leading to delayed classification results, affecting the accuracy of data security policies. The lack of automated verification mechanisms also makes it easy for classification results to contradict actual business intent, resulting in false positives or false negatives.

[0030] These shortcomings collectively affect the accuracy and efficiency of sensitive data management solutions, thereby impacting the security of sensitive data.

[0031] In view of the inventors’ above-mentioned research findings, the embodiments of this application provide an incremental update method, system, device, medium and product based on real-time change detection, which aims to improve the efficiency and accuracy of sensitive data identification, thereby enhancing its data security.

[0032] The incremental update method based on real-time change detection provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0033] Figure 1 This is a flowchart illustrating an incremental update method based on real-time change detection provided in an embodiment of this application. The executing entity of this incremental update method based on real-time change detection can be an incremental update system (hereinafter referred to as "the system").

[0034] The following example, using the incremental update system as the execution entity of the incremental update method based on real-time change detection, illustrates the incremental update method based on real-time change detection of this application. It should be noted that the aforementioned execution entity and application scenario do not constitute a limitation on this application.

[0035] like Figure 1 As shown, the incremental update method based on real-time change detection provided in this application embodiment may include steps 110-140.

[0036] Step 110: By listening to the database's binary log (Binlog), the database's Data Definition Language (DDL) operation statements are obtained in real time. By parsing the DDL operation statements, the database name, table name, field name, and change type corresponding to the changed fields are extracted to generate field change events.

[0037] Step 120: Create and execute an incremental scan task for the changed field based on the field change event, so as to call the sensitive data identification algorithm to output the classification and grading results of the changed field;

[0038] Step 130: Determine the database name, table name, field name, and operation timestamp associated with the field change event as search keywords, and search the change approval work orders within a preset time range before and after the operation timestamp in the change management system based on the search keywords to obtain the change approval work orders associated with the changed field.

[0039] Step 140: Determine the semantic relevance between the sensitive types of fields in the classification and grading results and the application reason text recorded in the change approval work order. If the semantic relevance reaches the preset relevance threshold, the classification and grading results are automatically confirmed and written into the metadata corresponding to the changed fields in the database. If the semantic relevance is lower than the preset relevance threshold, the classification and grading results are marked as pending confirmation and an alarm notification is generated and sent to the data security administrator for manual review.

[0040] The incremental update method based on real-time change detection provided in this application utilizes MySQL's built-in binlog service to achieve low-intrusive monitoring of real-time changes to the data table structure and improve the coverage of change monitoring. This enables immediate awareness of database structure changes, mitigating the lag issues of traditional periodic full scans and shortening the risk window for sensitive data exposure. Furthermore, an incremental scan task is created and executed based on field change events. This task only scans and identifies sensitive data in the changed fields, avoiding the computationally intensive and time-consuming full scans of traditional methods, thus reducing system resource consumption and improving detection efficiency. Furthermore, by linking with database change approval records, an automated verification mechanism is introduced. This mechanism retrieves associated change approval work orders from the change management system and determines the semantic relevance between the sensitive types of fields in the classification and grading results and the application reason text recorded in the change approval work orders. This allows for cross-verification of automated detection results with application records. If the semantic relevance reaches a preset threshold, the classification and grading results are automatically confirmed; otherwise, they are marked as pending confirmation, triggering an alarm notification for manual review. This process effectively reduces false alarm rates and improves the accuracy of classification and grading results. Finally, by writing the confirmed results into the metadata corresponding to the database change fields, accurate evidence is provided for the real-time effectiveness of data security policies. Thus, this application, through real-time sensing, incremental scanning, automated verification, and result write-back technologies, not only improves the efficiency and accuracy of sensitive data identification but also enhances the data security of sensitive data throughout its entire lifecycle from multiple dimensions, including shortening the risk window, reducing false alarm interference, and providing precise policy basis.

[0041] The specific implementation of the above steps will be described in detail below with reference to specific embodiments.

[0042] In step 110, by listening to the database's binary log (Binlog), the database's Data Definition Language (DDL) operation statements are obtained in real time. By parsing the DDL operation statements, the database name, table name, field name, and change type corresponding to the changed fields are extracted to generate field change events.

[0043] In step 110, the field change event is used to uniformly represent database table structure change operations (DDL operations). Its core data fields may include, but are not limited to: database_name (database name), table_name (table name), column_name (column name), and change_type (change type, such as ADD_COLUMN, ALTER_COLUMN, etc.). Here, parsing the DDL operation statement refers to the process of performing lexical and syntactic analysis on the original DDL statements recorded in the database binary log (Binlog) to extract the above-mentioned key semantic information (i.e., database name, table name, field name, and change type).

[0044] It should be noted that since the binary log records all statements that affect changes to the database state, including not only statements that change the table structure, but also DDL operation statements such as CREATE INDEX, further filtering is required to extract the DDL operation statements that involve changes to the data tables.

[0045] Based on this, in some embodiments of this application, optionally, step 110 above, by listening to the database's binary log (Binlog) to obtain the database's Data Definition Language (DDL) operation events in real time, may specifically include: starting the database's Binlog service and configuring its recording format to STATEMENT format; consuming the Binlog log stream through the mysqlbinlog tool or a custom log parsing module, parsing the binary log records into readable SQL statements; and identifying and filtering DDL operation statements containing target keywords from the SQL statements based on a predefined rule engine.

[0046] Specifically, the database's Binlog service is enabled by setting the value of the `sql_log_bin` field to 1 in the database configuration file, and the value of the `binlog_format` field is set to `STATEMENT`. The `STATEMENT` format is a record format for MySQL database Binlog. In this format, the Binlog records the raw SQL statements (such as `ALTER TABLE`) issued by the client, which is different from the `ROW` format (which records changes to each row of data) and the `MIXED` format (mixed mode).

[0047] Next, the system uses open-source tools (such as mysqlbinlog) or a custom log parsing module to read and parse the Binlog file stream in real time, converting the binary log records into readable SQL statements. Then, a predefined rule engine scans all SQL statements line by line, identifying and filtering statements containing the target keywords, and outputs these as DDL operation statements to subsequent processes. The subsequent parser extracts the database name, table name, field name, and change type from each DDL operation statement and assembles them into a complete field change event object.

[0048] The predefined rule engine can include at least one of the grep command and regular expression matching engines. The target keyword refers to a predefined set of strings used to filter and identify DDL operations in the SQL statement text, and the target keyword includes at least one of ALTER, CREATE, and DROP.

[0049] In some embodiments, multiple grep commands can be connected using pipes to progressively filter all SQL statements. The filtering logic is as follows: the output of the previous grep command is used as the input of the next grep command, and the target keywords contained in the SQL statement text are matched sequentially. Alternatively, the grep-E command can be used to enable extended regular expressions, and multiple DDL operation patterns to be matched can be defined in a single expression using logical OR operators. Alternatively, a series of regular expression patterns can be predefined in the rule engine. The regular expression patterns are used to match the characteristics of DDL statements that represent changes in table structure. The rule engine matches the SQL statement text with the predefined rules in real time. If the text content matches a certain rule, the statement is determined to be a DDL operation event.

[0050] In this embodiment, by configuring the Binlog to STATEMENT format and using tools or custom modules to parse it, it is possible to ensure that complete and readable SQL statements are captured. Then, the rule engine performs precise filtering based on target keywords, which enables efficient and accurate real-time acquisition of DDL operation statements from massive database operation logs, providing a reliable data input source for the entire process.

[0051] To address scenarios where Binlog is disabled, such as when binlog is configured with sql_log_bin=0 and therefore cannot monitor changes, network devices in the system can simultaneously send a mirrored copy of the database traffic to the auditing system. This allows the auditing plugin to capture the same DDL traffic packets.

[0052] Based on this, in some embodiments of this application, optionally, before extracting field change events containing the database name, table name, field name, and change type corresponding to the changed field by parsing DDL operation events, the method may further include: obtaining mirrored traffic packets of the database port through the port mirroring function of the network switch or the auditing plugin of the database server, and obtaining database operation statements by parsing the mirrored traffic packets; and identifying and filtering DDL operation statements containing target keywords from the database operation statements based on a predefined rule engine.

[0053] Specifically, the traffic of the database server is copied using the port mirroring function of a network switch, or SQL traffic is captured directly using a database auditing plugin. The traffic analysis system (or auditing system) receives this mirrored traffic packet, parses it (e.g., parses the MySQL protocol), and reconstructs the database operation statements. Then, similar to the SQL statement filtering process described above, the same predefined rule engine identifies and filters out DDL operation statements containing the target keywords from these database operation statements.

[0054] In this embodiment, mirrored traffic packets are obtained through port mirroring or an auditing plugin as another data source, forming a dual guarantee with the aforementioned Binlog listening channel. This ensures that even if Binlog is not enabled or cannot be parsed under certain special circumstances, DDL operation statements can still be successfully captured, improving the coverage and reliability of change awareness.

[0055] In some embodiments of this application, optionally, by parsing DDL operation statements, the database name, table name, field name, and change type corresponding to the changed field are extracted to generate field change events. This includes: uniformly formatting each DDL operation event obtained from binary logs and mirrored traffic packets into a field change event containing the database name, table name, field name, change type, and operation timestamp, to obtain each field change event; calculating the hash value corresponding to each field change event; and deduplicating field change events with the same hash value within a preset time window to obtain the final field change event.

[0056] The hash value is generated using the database name, table name, field name, change type, and operation timestamp as input sources. The hash value refers to a unique identifier calculated by hash algorithms such as MD5 and SHA-1. The same input will always produce the same hash value, so it can be used to quickly determine whether two events are exactly the same.

[0057] Specifically, the system converts all DDL operation events from different data sources, such as Binlog and mirror traffic, into a unified JSON format message (i.e., field change event). This message must contain the core fields: database_name, table_name, column_name, change_type, and operation_timestamp.

[0058] Next, for each standardized field change event, a hash value is calculated using all its core fields as input. The system checks for event messages with the same hash value within a preset time window (e.g., 5 seconds). For duplicate messages, indicating duplicate reports of the same DDL statement from both Binlog and mirror traffic, the system retains only the first one and discards subsequent duplicates, thus achieving aggregation and deduplication, ultimately generating a unique field change event.

[0059] In this embodiment, by uniformly formatting event messages and calculating their hash values, a unique identifier is generated for each change event. Then, by deduplicating field change events with the same hash value within a preset time window, the problem of duplicate events triggering multiple processing due to the dual-channel listening mechanism is effectively avoided. This ensures that each subsequent incremental scan task and verification process is executed based on a unique and accurate change event, thereby improving the accuracy and stability of the system.

[0060] Step 120: Create and execute an incremental scan task for the changed field based on the field change event, so as to call the sensitive data identification algorithm to output the classification and grading results of the changed field.

[0061] In step 120, the incremental scan task refers to the data scan and sensitive data identification task initiated only for the single field that has changed. Compared with the traditional "full database scan", its scope is accurate to the field level and has low resource consumption. The sensitive data identification algorithm refers to the set of rules or models used to determine the sensitivity of data, which may include regular expression-based rule matching (such as ID card and mobile phone number patterns), natural language processing (NLP) models, or machine learning classifiers.

[0062] Specifically, incremental scan tasks can be created based on the database name, table name, field name, and change type contained in the field change event. For example, for the changed field id_card, the task scope is accurate to user_db.users.id_card, and the meta information of this task explicitly limits its scan targets to: database_name.user_db, table_name.users, column_name.id_card.

[0063] The incremental scan task calls the built-in sensitive data identification algorithm to analyze the sample data corresponding to the changed fields and outputs the classification and grading results of the changed fields. The classification and grading results may include at least: sensitivity data type (indicating the sensitive category or subject area to which the data belongs), field sensitivity type (more precise than sensitivity, clearly indicating what kind of data the field is), sensitivity level (indicating the degree of sensitivity or importance of the data in this category), and may also include confidence (indicating the algorithm's certainty or probability in making this classification and grading result, a value between 0 and 1).

[0064] For example, when sampling data from the `id_card` field, 100 values ​​of this field are randomly selected. Then, a sensitive data identification algorithm is called to analyze the sample data. The algorithm performs calculations based on the data characteristics. For instance, if 95% of the sampled data conforms to the format of an ID card number, the algorithm outputs the classification and grading result as: {"sensitivity":"PII","level":"L3","type":"ID_Card","confidence":0.95}.

[0065] In some embodiments of this application, optionally, creating and executing an incremental scanning task for the changed field based on a field change event includes: pushing the field change event to a data classification and grading system through a second API interface, so that the data classification and grading system creates an incremental scanning task based on the database name, table name, and field name in the field change event, wherein the configuration scope of the incremental scanning task is limited to performing data sampling and sensitive content identification analysis only on the field names in the table names under the database name.

[0066] Here, the second API interface refers to the programming interface provided by the data classification and grading system, used to receive external event messages and trigger the creation of scan tasks. This interface differs from the first API interface in its target audience; the former is geared towards the classification and grading system, while the latter is geared towards security devices. An incremental scan task refers to a scan task whose scan scope is precisely defined. Its scan target is not the entire database or table, but is limited to the specific database, table, and field specified in the field change event.

[0067] Specifically, the aforementioned system may include an event center and a classification and grading system. After the event center completes step 110 to generate a field change event, it can immediately push the field change event as a push message to the classification and grading system by calling the second API interface of the data classification and grading system (e.g., POST / api / v1 / scan / task). Upon receiving the message, the classification and grading system parses the parameters such as database_name, table_name, and column_name in the message body, and then creates a new scan task in the task management module. For example, for the changed field id_card, the core configuration item of this task limits its scan scope to only sampling and sensitive content identification analysis of the id_card field in the users table under the user_db database, without touching any other unrelated fields.

[0068] In this embodiment, an event is pushed and an incremental scan task is created via a second API interface. The task's configuration scope is limited to sampling and analyzing sensitive content only for the field names in the table under the database name. This allows computing resources to be highly focused on the smallest data unit that has changed. Compared to traditional periodic full database scans, this avoids a large amount of unnecessary repetitive calculations, greatly reduces system resource consumption, and improves scanning efficiency, making it particularly suitable for ultra-large database environments.

[0069] Step 130: Determine the database name, table name, field name, and operation timestamp associated with the field change event as search keywords, and search the change management system for change approval work orders within a preset time range before and after the operation timestamp based on the search keywords, to obtain the change approval work orders associated with the changed field.

[0070] In step 130, after configuring the MySQL Binlog service and setting its record format to STATEMENT, Binlog automatically appends a timestamp to each DDL statement when recording it. This timestamp is the operation timestamp, recording the moment the statement was executed on the database server. This operation timestamp can be used as one field in a field change event, or it can be retrieved independently from the Binlog after the field change event is generated. The search keyword is used to accurately query relevant approval records in the Change Management System (ITSM). It is not a single keyword but a composite key. The change approval work order refers to the application and approval voucher for database change operations, which is recorded in a process-oriented manner in ITSM systems (such as Jira and ServiceNow). The "Reason for Application" text field is the key basis for semantic comparison.

[0071] Specifically, the system accesses the change management system, initiates a query using pre-assembled keywords, and retrieves change approval work orders involving the same database, table, and fields within a certain time range (e.g., ±2 hours) before and after the operation timestamp. The ITSM system then returns the matching work order information. For example, for the search keywords generated based on the field change event corresponding to the changed field id_card, a work order record is returned, with the "Reason for Application" field containing: "Adding the ID number field required for user real-name authentication".

[0072] Step 140: Determine the semantic relevance between the sensitive types of fields in the classification and grading results and the application reason text recorded in the change approval work order. If the semantic relevance reaches the preset relevance threshold, the classification and grading results are automatically confirmed and written into the metadata corresponding to the changed fields in the database. If the semantic relevance is lower than the preset relevance threshold, the classification and grading results are marked as pending confirmation and an alarm notification is generated and sent to the data security administrator for manual review.

[0073] In step 140, semantic relevance can refer to the degree of semantic association between the field sensitivity type and the application reason text. Its value is a numerical value between 0 and 1, which can be calculated by a natural language processing (NLP) model. The preset relevance threshold is used as a benchmark for automated judgment. Its value can be set according to specific needs, such as 0.75, 0.8 or other values.

[0074] Specifically, the system inputs the sensitive field type (such as "PII_ID_Card") from the classification and grading results and the application reason text (such as "ID number field required for new user real-name authentication") from the approval work order into the semantic relevance calculation model (such as a sentence embedding model based on BERT). The model outputs a relevance score to obtain the semantic relevance, and compares the semantic relevance with a preset relevance threshold. The preset relevance threshold can be set according to specific needs, such as 0.8, 0.9 or other values. This application does not make specific limitations on this.

[0075] If the semantic relevance value is greater than or equal to the preset relevance threshold: for example, the semantic relevance between the sensitive field type "ID number" and the application reason "ID number field required for new user real-name authentication" is greater than the preset relevance threshold of 0.8, the system automatically determines that the classification and grading result is accurate and automatically confirms the result; if the semantic relevance value is less than the preset relevance threshold: for example, the semantic relevance between the sensitive field type "ID number" and the application reason "new user hobby field" is less than 0.8, the system marks this result as "pending confirmation" and automatically generates an alarm notification (such as email or SMS) to send to the data security administrator, prompting them to conduct a manual review.

[0076] In related technologies, the classification and grading results are stored independently in the classification and grading system. The database itself cannot mark the results of tables and fields. The decoupling of the classification and grading results from the database and applications makes it impossible for the classification and grading results to be used in practice.

[0077] Based on this, in some embodiments of this application, optionally, writing the classification and grading results into the metadata corresponding to the changed field in the database may specifically include: writing the classification and grading results into the metadata of the changed field in the form of a COMMENT through a data classification and grading system or database management tool.

[0078] Specifically, the classification and grading results can be confirmed by the event center and then pushed to the classification and grading system, or directly confirmed by the data classification and grading system. After confirming the classification and grading results, the data classification and grading system or database management tools (such as MySQL Workbench, pt-online-schema-change) will construct a specific ALTER TABLE DDL statement. This DDL statement, in addition to including the MODIFY COLUMN or CHANGE COLUMN clauses, adds a COMMENT clause. Through this COMMENT clause, the classification and grading results are persisted as comments in the metadata of that field, so that all subsequent applications or tools accessing that field can directly perceive its sensitivity.

[0079] For example, for the field user_db.users.id_card, whose classification result is "Personal Sensitive Information, Level L3", the system will generate and execute the following SQL command:

[0080] "ALTER TABLE user_db.users MODIFY COLUMN id_card varchar(18)COMMENT'Sensitivity:PII_ID_Card,Level:L3';)";

[0081] This statement modifies the field and permanently writes the classification and grading results as key-value pairs into the field's metadata comment. Subsequently, the sensitive attribute labels for this field can be viewed by querying the information_schema.COLUMNS system table or using the SHOW FULLCOLUMNS command.

[0082] In this embodiment, by directly writing the classification and grading results into the metadata of the database field in the form of COMMENT, the database itself and all applications accessing the database can directly perceive the sensitivity of the data through a standard interface. This provides an accurate and reliable data foundation for the subsequent implementation of dynamic data security strategies based on metadata (such as real-time desensitization), and solves the problem of decoupling the classification and grading results from the application system.

[0083] In some embodiments of this application, optionally, after writing the classification and grading results into the metadata corresponding to the changed field in the database in step 140 above, the following steps may also be included: adding an access control policy for the changed field in the data security policy; and distributing the data security policy to the data security gateway and the database firewall through the first application programming API interface, so that the data security gateway and the database firewall, upon receiving the data security policy, immediately load it into the runtime policy rule set and execute the access control policy for all subsequent access requests for the changed field.

[0084] Here, access control policies refer to the set of rules configured on a data security gateway (DSG) or database firewall (DBF) to control access to specific data resources. These policies can include at least one of the following: dynamic masking (real-time masking of query results), access auditing (logging all access requests), and access blocking (rejecting illegal or high-risk access requests). The first API interface refers to the programming interface provided by the data security gateway / firewall vendor, used to receive external instructions to remotely and automatically create, update, and delete its internal security policies; it is typically a RESTful API.

[0085] Specifically, the system automatically generates or triggers the generation of a new data security policy based on the classification and grading results written to the metadata. For example, for the id_card field marked as Sensitivity:PII, Level:L3, a policy is generated: "Desensitize the query results for this field (e.g., display only the last four digits) and record a full audit log." Then, by calling the first API interface of the data security gateway / firewall, the above policy content is encapsulated in an HTTP request in a format such as JSON or XML and sent to the target device. After receiving the policy, the data security gateway / firewall loads it into the policy rule engine in memory in real time, and it takes effect immediately. Subsequently, all database requests flowing through the target device will be matched and processed by this rule, achieving real-time protection of the changed field.

[0086] In this embodiment, by adding an access control policy for the changed field to the data security policy and distributing the data security policy to the data security gateway and database firewall via a first API interface, the deployment of the security policy and the data change awareness, classification, and grading processes are fully automated. The policy is immediately loaded into the runtime policy rule set upon receiving it from the device, achieving real-time effect of access control for the changed field within minutes. This improves upon the latency issues of traditional manual policy configuration and shortens the exposure window for sensitive data.

[0087] As a concrete example, Figure 2 This is an exemplary flowchart of an incremental update method based on real-time change detection provided in an embodiment of this application. The following is a detailed description in conjunction with... Figure 2 This application describes the specific implementation process:

[0088] Step 1, Low-Intrusive Data Table Structure Change Detection: When a database administrator executes a DDL statement, this application detects the change in real time through a dual mechanism:

[0089] (1) Binlog Channel: Enable the database binlog service and configure it to STATEMENT format; output binlog log stream in real time; then execute the binlog log file parsing process: parse the logs using the mysqlbinlog tool; filter statements based on predefined DDL operation filtering rules; parse DDL operation SQL statements, extract changed database, table, and field information, and generate field change events;

[0090] (2) Mirrored traffic channel: Since some sessions are configured with sql_log_bin=0, which leads to monitoring bypass and binlog is disabled, traffic mirroring is used to supplement and improve coverage. Specifically, network mirroring traffic is configured to capture database communication; mirrored statements are obtained through the database auditing plugin; and then the audit record statement parsing process is executed: DDL operation statements are filtered based on the rule engine; DDL operation SQL statements are parsed to extract the changed database, table and field information and generate field change events.

[0091] Step 2: Information Aggregation and System Linkage. The field change events obtained from dual-channel parsing are aggregated to form a unified set of field change events, and then the dual-system linkage mechanism is activated:

[0092] (1) Data classification and grading system linkage: This data classification and grading system only performs tasks on tables that have been changed, starts incremental scanning and analysis processes, and outputs the classification and grading results of the changed fields;

[0093] (2) Linkage with the change approval system: Since there is likely no sample data for the fields that have just been changed, the classification and grading results will be inaccurate and feature matching based on the sampled data will not be possible. Therefore, change application records in the database are collected, the changed database, table and field information are parsed, the Chinese functional description of the changed fields is parsed, and the classification and grading of the changed fields marked in the change application records are output.

[0094] Step 3, Result Cross-validation: When the field change event is the same as the classification and grading result in the change application record, proceed directly to the next process; when there is a difference, the manual review and marking mechanism is activated, and the final classification and grading result is manually reviewed and confirmed.

[0095] Step 4, Result Output and Policy Update: After verification, the final classification and grading results are output in near real-time. The classification and grading results are written into the metadata of the changed field in the database in the form of COMMENT, and the data security policy in the data security gateway is updated to complete the entire change processing flow.

[0096] This embodiment fully demonstrates the entire process from dual-channel change perception, information aggregation, system linkage, cross-validation to result implementation. Dual-channel perception ensures the integrity of change capture, dual-system linkage enables automated verification, and finally, result writing and security policy updates achieve closed-loop management, demonstrating the advantages of this application in terms of real-time performance, reliability, and accuracy.

[0097] Corresponding to the method embodiments of this application, this application also provides an incremental update system based on real-time change detection, which is applied to the client.

[0098] Figure 3 This is a schematic diagram of the structure of an incremental update system based on real-time change detection provided in an embodiment of this application. Figure 3 As shown, the incremental update system 300 based on real-time change detection may include: a generation module 310, a creation and execution module 320, a retrieval module 330, and a result confirmation module 340.

[0099] The generation module 310 is used to acquire the database's Data Definition Language (DDL) operation statements in real time by listening to the database's binary log (Binlog), and extract the database name, table name, field name, and change type corresponding to the changed field by parsing the DDL operation statements to generate a field change event. The creation and execution module 320 is used to create and execute an incremental scan task for the changed field based on the field change event, and call a sensitive data identification algorithm to output the classification and grading results of the changed field. The retrieval module 330 is used to determine the database name, table name, field name, and operation timestamp associated with the field change event as retrieval keywords. The system retrieves change approval work orders within a preset time range before and after the operation timestamp in the change management system based on search keywords, and obtains the change approval work orders associated with the change fields. The result confirmation module 340 is used to determine the semantic relevance between the field sensitivity type in the classification and grading results and the application reason text recorded in the change approval work order. If the semantic relevance reaches the preset relevance threshold, the classification and grading results are automatically confirmed and written into the metadata corresponding to the change fields in the database. If the semantic relevance is lower than the preset relevance threshold, the classification and grading results are marked as pending confirmation and an alarm notification is generated and sent to the data security administrator for manual review.

[0100] The incremental update system based on real-time change detection provided in this application utilizes MySQL's built-in binlog service to achieve low-intrusive monitoring of real-time changes to the data table structure and improve the coverage of change monitoring. This enables immediate awareness of database structure changes, mitigating the lag issues of traditional periodic full scans and shortening the risk window for sensitive data exposure. Furthermore, it creates and executes incremental scan tasks targeting changed fields based on field change events. These tasks only scan and identify sensitive data in the changed fields, avoiding the computationally intensive and time-consuming full scans of traditional methods, thus reducing system resource consumption and improving detection efficiency. Furthermore, by linking with database change approval records, an automated verification mechanism is introduced. This mechanism retrieves associated change approval work orders from the change management system and determines the semantic relevance between the sensitive types of fields in the classification and grading results and the application reason text recorded in the change approval work orders. This allows for cross-verification of automated detection results with application records. If the semantic relevance reaches a preset threshold, the classification and grading results are automatically confirmed; otherwise, they are marked as pending confirmation, triggering an alarm notification for manual review. This process effectively reduces false alarm rates and improves the accuracy of classification and grading results. Finally, by writing the confirmed results into the metadata corresponding to the database change fields, accurate evidence is provided for the real-time effectiveness of data security policies. Thus, this application, through real-time sensing, incremental scanning, automated verification, and result write-back technologies, not only improves the efficiency and accuracy of sensitive data identification but also enhances the data security of sensitive data throughout its entire lifecycle from multiple dimensions, including shortening the risk window, reducing false alarm interference, and providing precise policy basis.

[0101] In some embodiments of this application, the result confirmation module is specifically used to: write the classification and grading results into the metadata of the change field in the form of a COMMENT through a data classification and grading system or database management tool.

[0102] In some embodiments of this application, the system further includes: a policy update module, configured to add an access control policy for the changed field in the data security policy after writing the classification and grading results into the metadata corresponding to the changed field in the database, wherein the access control policy includes at least one of dynamic desensitization, access auditing, and access blocking; and a policy distribution module, configured to distribute the data security policy to the data security gateway and the database firewall through a first application programming API interface, so that the data security gateway and the database firewall, upon receiving the data security policy, immediately load it into the runtime policy rule set and execute the access control policy on all subsequent access requests for the changed field.

[0103] In some embodiments of this application, the creation execution module is specifically used to: push field change events to the data classification and grading system through the second API interface, so that the data classification and grading system can create an incremental scan task based on the database name, table name, and field name in the field change event. The configuration scope of the incremental scan task is limited to performing data sampling and sensitive content identification analysis only on the field names in the table names under the database name.

[0104] In some embodiments of this application, the generation module includes: a configuration unit for enabling the database's Binlog service and configuring its record format to STATEMENT format; a parsing unit for consuming the Binlog log stream through the mysqlbinlog tool or a custom log parsing module, and parsing the binary log records into readable SQL statements; and an identification and filtering unit for identifying and filtering DDL operation statements containing target keywords from the SQL statements based on a predefined rule engine, wherein the target keywords include at least one of ALTER, CREATE, and DROP.

[0105] In some embodiments of this application, the system further includes: a parsing unit, which is further configured to obtain mirrored traffic packets of the database port through the port mirroring function of the network switch or the auditing plugin of the database server before extracting field change events containing the database name, table name, field name and change type corresponding to the changed field by parsing the DDL operation events, and obtain database operation statements by parsing the mirrored traffic packets; and an identification and filtering unit, which is further configured to identify and filter DDL operation statements containing target keywords from the database operation statements based on a predefined rule engine.

[0106] In some embodiments of this application, the parsing unit is specifically used to: uniformly format each DDL operation event obtained from binary logs and mirrored traffic packets into a field change event containing database name, table name, field name, change type, and operation timestamp, thereby obtaining each field change event; calculate the hash value corresponding to each field change event, wherein the hash value is generated using database name, table name, field name, change type, and operation timestamp as input sources; and within a preset time window, deduplicate field change events with the same hash value to obtain the final field change event.

[0107] The incremental update system based on real-time change detection provided in this application embodiment can achieve... Figure 1-2 The various processes implemented in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0108] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0109] like Figure 4 As shown, the electronic device 400 includes a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402.

[0110] In one example, the processor 402 described above may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0111] Memory 401 may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the incremental update method based on real-time change detection according to the embodiments of the first aspect of this application.

[0112] The processor 402 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 401, in order to implement the incremental update method based on real-time change detection in the embodiment of the first aspect described above.

[0113] In some examples, electronic device 400 may also include communication interface 403 and bus 410. For example, Figure 4 As shown, the memory 401, processor 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0114] The communication interface 403 is mainly used to enable communication between various modules, systems, units, and / or devices in the embodiments of this application. Input devices and / or output devices can also be connected through the communication interface 403.

[0115] Bus 410 includes hardware, software, or both, that couples components of electronic device 400 together. For example, and not limitingly, bus 410 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0116] The electronic device provided in this application embodiment is capable of achieving Figure 1-2 The various processes implemented in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0117] In conjunction with the incremental update method based on real-time change detection in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement the steps of any of the incremental update methods based on real-time change detection in the above embodiments.

[0118] In conjunction with the incremental update method based on real-time change detection in the above embodiments, this application embodiment can provide a computer program product to implement it. This (computer) program product is stored in a non-volatile storage medium, and when executed by at least one processor, it implements the steps of any of the incremental update methods based on real-time change detection in the above embodiments.

[0119] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described incremental update method embodiment based on real-time change detection, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0120] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0121] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0122] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0123] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or systems. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0124] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing system to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing system, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0125] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for incremental update based on real-time change detection, characterized in that, The application comprises the following steps: Real-time acquisition of database data definition language (DDL) operation statements by monitoring the database binary log (Binlog), and extraction of the database name, table name, field name and change type corresponding to the changed field from the DDL operation statements by analyzing the DDL operation statements to generate a field change event; Creation and execution of an incremental scanning task for the changed field based on the field change event to call a sensitive data identification algorithm to output a classification and grading result of the changed field; Determination of the database name, table name, field name in the field change event and the operation timestamp associated with the field change event as a retrieval keyword, and retrieval of a change approval work order in a change management system within a preset time range before and after the operation timestamp based on the retrieval keyword to obtain a change approval work order associated with the changed field; Determination of the semantic relevance of the field sensitive type in the classification and grading result to the application reason text recorded in the change approval work order, automatic confirmation of the classification and grading result if the semantic relevance reaches a preset relevance threshold, and writing of the classification and grading result into the metadata corresponding to the changed field in the database, and marking of the classification and grading result as a to-be-confirmed state and generating an alarm notification to send to a data security administrator for manual review if the semantic relevance is lower than the preset relevance threshold.

2. The method of claim 1, wherein, Writing the classification and grading result into the metadata corresponding to the changed field in the database comprises the following steps: Writing the classification and grading result in the form of COMMENT into the metadata of the changed field through a data classification and grading system or a database management tool.

3. The method of claim 1, wherein, After writing the classification and grading result into the metadata corresponding to the changed field in the database, the following steps are further included: Adding an access control policy for the changed field in the data security policy, wherein the access control policy comprises at least one of dynamic desensitization, access audit and access blocking; Downlinking the data security policy to a data security gateway and a database firewall through a first application programming API interface, so that the data security gateway and the database firewall load the data security policy into a runtime policy rule set after receiving the data security policy, and execute the access control policy on all subsequent access requests for the changed field.

4. The method of claim 1, wherein, Creation and execution of an incremental scanning task for the changed field based on the field change event comprises the following steps: Pushing the field change event to a data classification and grading system through a second API interface, so that the data classification and grading system creates the incremental scanning task based on the database name, table name and field name in the field change event, wherein the configuration range of the incremental scanning task is limited to data sampling and sensitive content identification analysis of only the field name in the table name under the database name.

5. The method of claim 1, wherein, Real-time acquisition of database data definition language (DDL) operation events by monitoring the database binary log (Binlog) comprises the following steps: Starting the Binlog service of the database and configuring its record format as STATEMENT format; Consuming the Binlog log stream through the mysqlbinlog tool or a custom log parsing module, the binary log record is parsed into readable SQL statements; Based on a predefined rule engine, identifying and filtering DDL operation statements containing target keywords from the SQL statements, wherein the target keywords include at least one of ALTER, CREATE, and DROP.

6. The method of claim 5, wherein, Before extracting the field change event containing the database name, table name, field name, and change type corresponding to the changed field from the parsed DDL operation event, the method further comprises: Obtaining the mirror traffic packet of the database port through the port mirroring function of the network switch or the audit plug-in of the database server, and obtaining the database operation statement by parsing the mirror traffic packet; Based on the predefined rule engine, identifying and filtering DDL operation statements containing the target keywords from the database operation statement.

7. The method of claim 6, wherein, Generating a field change event by parsing the DDL operation statement to extract the database name, table name, field name, and change type corresponding to the changed field, comprising: Uniformly formatting each DDL operation event obtained from the binary log and the mirror traffic packet into a field change event containing the database name, table name, field name, change type, and operation timestamp, to obtain various field change events; Calculating the hash value corresponding to each field change event, wherein the hash value is generated based on the database name, table name, field name, change type, and operation timestamp as input sources; In a preset time window, de-duplicating each field change event with the same hash value to obtain the final field change event.

8. A delta update system based on real-time change detection, characterized in that, Comprising: A generation module for obtaining the data definition language (DDL) operation statement of the database in real time by listening to the binary log (Binlog) of the database, and generating a field change event by parsing the DDL operation statement to extract the database name, table name, field name, and change type corresponding to the changed field; A creation and execution module for creating and executing an incremental scanning task for the changed field based on the field change event to call a sensitive data identification algorithm to output a classification and grading result of the changed field; A retrieval module for determining the database name, table name, field name, and operation timestamp associated with the field change event as retrieval keywords, and retrieving a change approval work order within a preset time range before and after the operation timestamp in a change management system based on the retrieval keywords to obtain a change approval work order associated with the changed field; A result confirmation module for determining the semantic relevance of the field sensitive type in the classification and grading result to the application reason text recorded in the change approval work order, if the semantic relevance reaches a preset relevance threshold, automatically confirming the classification and grading result, and writing the classification and grading result into the metadata corresponding to the changed field in the database, if the semantic relevance is lower than the preset relevance threshold, marking the classification and grading result as a to-be-confirmed state, and generating an alarm notification to send to a data security administrator for manual review.

9. An electronic device, comprising: The electronic device comprises a processor and a memory storing computer program instructions; the electronic device implements the method according to any one of claims 1-7 when executing the computer program instructions.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the method according to any one of claims 1-7.

Images