A data change awareness method, device and medium based on application layer non-invasive access

Abstract

This application discloses a data change perception method, device, and medium based on non-intrusive application-layer access, relating to the field of data processing. The method includes: intercepting update-type SQL statements sent from the application layer to the database; identifying the database operation type and condition information in the update-type SQL statements; generating a first query statement based on the database operation type and condition information; executing the first query statement to obtain the changed data range corresponding to the update-type SQL statement and the data before the change within the changed data range; and after the business transaction to which the update-type SQL statement belongs is successfully committed, executing a second query statement based on the changed data range to obtain the changed data within the changed data range. This application achieves data change perception by intercepting the execution process of database write operation SQL at the application layer, parsing the semantics and constructing a reverse query before SQL execution to determine the changed data range, and determining the timing of data capture and reporting after SQL execution based on the transaction status.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data change sensing method, device and medium based on application-layer non-intrusive access. Background Technology

[0002] Currently, in the fields of data governance, metadata management, and data lineage analysis, the capture of changes in table data in business databases typically adopts a change data capture (CDC) solution based on database logs (Binlog).

[0003] In one approach using related technologies, the Canal middleware is deployed to monitor the Binlog logs of a MySQL database. When insert, update, or delete operations occur on a business table, Canal asynchronously pushes the corresponding change events to the downstream metadata service. The metadata service receives the events through a pre-defined callback interface and builds SQL parsers based on different data stack application types to parse the change statements, thereby identifying the dependencies between tables and ultimately persistently storing the lineage information in a relational database.

[0004] However, this solution heavily relies on MySQL's unique Binlog logging mechanism, making it difficult to directly adapt to other database systems that do not support or are not fully compatible with the Binlog protocol. With the increasing adoption of domestically developed databases in critical sectors, continuing to use this CDC solution, which heavily relies on underlying database logs, often necessitates restructuring the data acquisition chain and adapting to new log parsing middleware during database migration or heterogeneous deployment. This results in high system transformation costs, significant technical risks, and an inability to achieve a smooth, seamless replacement of existing business systems. Summary of the Invention

[0005] This application provides a data change awareness method, device, and medium based on application-layer non-intrusive access, which can avoid dependence on the underlying database log mechanism, thereby adapting to multiple database types and enabling smooth and seamless replacement of existing systems without modifying business code.

[0006] Firstly, this application provides a data change detection method based on application-layer non-intrusive access, the data change detection method based on application-layer non-intrusive access comprising: Before the data from the application layer is sent to the database, intercept the update SQL statements destined for the database; The intercepted update SQL statements are parsed to identify their corresponding database operation types and condition information; A first query statement is generated based on the database operation type and the condition information; Execute the first query statement to obtain the range of changed data corresponding to the update SQL statement and the data before the change within the range of changed data; After the business transaction to which the update SQL statement belongs is successfully committed, a second query statement is executed based on the changed data range to obtain the changed data within the changed data range; Data change information is generated based on the data before the change and the data after the change.

[0007] In an optional implementation of the first aspect, the data change awareness method based on application-layer non-intrusive access further includes: Determine whether the execution of the update SQL statement is within a transaction environment; If in a transactional environment, the step of executing the second query statement will be registered as a transaction synchronization step. A callback is triggered after the business transaction is committed.

[0008] In an optional implementation of the first aspect, after determining whether the execution of the update SQL statement is in a transaction environment, the method further includes: If not in a transaction environment, the second query statement will be executed after the update SQL statement has been executed.

[0009] In an optional implementation of the first aspect, generating the first query statement based on the database operation type and the condition information in the update-type SQL statement includes: If the database operation type is a modification operation or a deletion operation, then the first query statement is constructed based on the conditional clauses in the update-type SQL statement.

[0010] In an optional implementation of the first aspect, after generating data change information based on the data before the change and the data after the change, the method further includes: Based on the data change information and the parsing results of the update-type SQL statements, data lineage analysis is performed to obtain lineage information; The bloodline information is stored in the metadata management system.

[0011] In one alternative implementation of the first aspect, intercepting update-type SQL statements destined for the database before the application-layer data is sent to the database includes: A custom interceptor plugin based on the MyBatis framework is used to intercept the update operation methods of the database executor in order to capture the update-type SQL statements.

[0012] In one alternative implementation of the first aspect, the custom interceptor plugin is encapsulated as an independently integrateable module and registered to the application's data session via dependency import.

[0013] Secondly, this application provides a data change awareness system based on application-layer non-intrusive access, comprising: The interception module is used to intercept update SQL statements sent to the database before the data from the application layer is sent to the database. The parsing module is used to perform syntax parsing on the intercepted update-type SQL statements and identify their corresponding database operation types and condition information; The reverse query module is used to generate a first query statement based on the database operation type and the condition information; and to execute the first query statement to obtain the changed data range corresponding to the update SQL statement and the data before the change within the changed data range. The change query module is used to execute a second query statement based on the changed data range after the business transaction to which the update SQL statement belongs is successfully submitted, so as to obtain the changed data within the changed data range; The analysis module is used to generate data change information based on the data before the change and the data after the change.

[0014] Thirdly, this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the data change sensing method based on application-layer non-intrusive access as provided in the first aspect.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the data change perception method based on application-layer non-intrusive access provided in the first aspect.

[0016] The data change sensing method, system, device, and medium based on application-layer non-intrusive access provided in this application have the following beneficial effects: This application intercepts update SQL statements destined for the database before data is sent from the application layer. The intercepted SQL statements are parsed to identify their corresponding database operation types and conditions. A first query statement is generated based on the database operation type and conditions. This first query statement is executed to obtain the range of changed data corresponding to the update SQL statement and the data before the change within that range. After the business transaction to which the update SQL statement belongs is successfully committed, a second query statement is executed based on the range of changed data to obtain the data after the change within that range. Data change information is generated based on the data before and after the change. This application achieves accurate and consistent awareness of data changes by intercepting the execution of database write SQL operations at the application layer, parsing the semantics and constructing a reverse query before SQL execution to determine the range of changed data, and determining the timing of data capture and reporting after SQL execution based on the transaction status. This solution does not rely on specific logging mechanisms at the database level, can adapt to various database types, and can be integrated in a low-intrusion, modular manner, supporting lossless replacement of existing CDC systems. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] Figure 1 A flowchart illustrating a data change awareness method based on non-intrusive application-layer access provided in an embodiment of this application; Figure 2 Another flowchart illustrating a data change awareness method based on non-intrusive application-layer access provided in an embodiment of this application; Figure 3 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0020] Currently, in the fields of data governance, metadata management, and data lineage analysis, the capture of changes in table data in business databases typically adopts a change data capture (CDC) solution based on database logs (Binlog).

[0021] One implementation of this technology involves deploying Canal middleware to monitor the MySQL database's Binlog. When an INSERT, UPDATE, or DELETE operation occurs on a business table, Canal asynchronously pushes the corresponding change event to downstream systems. The downstream metadata service receives the event through a pre-defined callback interface and builds corresponding parsers based on different application types to parse the changed SQL, thereby identifying table dependencies and ultimately persistently storing the lineage information. This solution features low invasiveness to business systems and strong real-time performance.

[0022] However, the aforementioned Canal-based CDC technology solution has the following obvious shortcomings in practical applications: (1) Strong dependence on database type: This solution is deeply coupled with MySQL’s Binlog log mechanism, making it difficult to adapt to other database systems that do not support or are incompatible with the Binlog protocol.

[0023] (2) Not applicable to domestic database environments: With the widespread use of domestic databases such as DM and Kingbase, they usually do not provide or support the Binlog logging mechanism with MySQL, which makes the existing Canal-based listening solution unable to work.

[0024] (3) High system transformation cost and high replacement risk: When it is necessary to migrate from MySQL to a domestic database, if the existing solution is used, it is often necessary to reconstruct the entire CDC link and adapt to the new log parsing middleware. The transformation cost is high, the technical risk is high, and it is difficult to achieve a smooth and imperceptible system replacement.

[0025] (4) Difficulty in ensuring consistency of business semantics: Binlog is a physical log at the bottom layer of the database. Its event sequence is difficult to align precisely with the transaction logic of the business layer, which may cause the downstream received change messages to be out of order or contain intermediate data of rolled-back transactions, thus failing to ensure consistency between change perception and business semantics.

[0026] To address at least one of the aforementioned issues, this application provides a data change awareness method, system, device, and medium based on non-intrusive application-layer access. By intercepting the execution of database write operations (SQL) at the application layer, parsing the semantics and constructing a reverse query before SQL execution to determine the scope of changed data, and determining the timing of data capture and reporting after SQL execution based on the transaction status, accurate and consistent awareness of data changes is achieved. This solution does not rely on specific logging mechanisms at the database level, can adapt to various database types, and can be integrated in a low-intrusive, modular manner, supporting seamless replacement of existing CDC systems.

[0027] Please see Figure 1 , Figure 1 This application provides a data change awareness method based on non-intrusive application-layer access, comprising steps S10-S60: S10 intercepts update-type SQL statements destined for the database before the application layer data is sent to the database.

[0028] S20 performs syntax parsing on the intercepted update SQL statements to identify their corresponding database operation types and condition information.

[0029] S30, Generate the first query statement based on the database operation type and condition information.

[0030] S40, execute the first query statement to obtain the range of changed data corresponding to the update SQL statement and the data before the change within the range of changed data.

[0031] S50: After the business transaction to which the update SQL statement belongs is successfully committed, execute the second query statement based on the changed data range to obtain the changed data within the changed data range.

[0032] S60 generates data change information based on the data before and after the change.

[0033] In an optional implementation, S10 may include: By using a custom interceptor plugin based on the MyBatis framework, the update operation methods of the database executor can be intercepted to capture update-type SQL statements.

[0034] Custom interceptor plugins are encapsulated as independently integrateable modules and registered to the application's data session via dependency import.

[0035] Specifically, based on the Interceptor extension mechanism provided by MyBatis, an SQL interception module CdcMybatisInterceptor is built to intercept the update methods of the database executor, and is used to capture database operations of type INSERT, UPDATE and DELETE.

[0036] The SQL interception module is defined as a starter dependency in Spring Boot. Internally, it implements automatic interceptor registration logic through a custom `spring.factories` configuration file. Once the application includes this starter dependency in its `pom.xml` file, no additional registration code is required; the SQL interception module will automatically inject into the application's data session factory (`sqlSessionFactory`), completing the functional integration.

[0037] In addition, the SQL interception module has built-in full table listening logic and SQL parsing logic. Applications do not need to develop additional adaptation code, nor do they need to modify the original business logic or database access methods. They can quickly obtain data change interception capabilities by simply importing the dependency, which greatly reduces the cost and intrusion of replacing the original CDC solution.

[0038] When S20 performs syntax parsing on the intercepted update-type SQL statements, it can use self-developed or existing SQL parsing tools to perform syntax parsing on the intercepted update-type SQL statements. By identifying the key fields of the update-type SQL statements (such as INSERT, UPDATE, DELETE), it can determine the database operation type to which the current update-type SQL statement belongs. Specifically, it can be divided into three database operation types: add operation, modify operation, and delete operation.

[0039] In an optional implementation, when S30 is executed, if the database operation type is a modification or deletion operation, a first query statement is constructed based on the conditional clauses in the update-type SQL statement. If the database operation type is an insert operation, then it is not necessary to construct a first query statement. The first query statement is a SELECT query statement whose scope is completely equivalent to that of the update-type SQL statement.

[0040] Specifically, for update SQL statements of type UPDATE or DELETE, the conditional clauses, such as the conditional expressions contained in the WHERE clause, are first extracted from the string of the update SQL statement. Then, with the conditional clause as the core constraint, a SELECT query statement that is completely equivalent to the scope of influence of the original database operation type is constructed, which is the first query statement.

[0041] For example, if the intercepted original update SQL statement is "update table setname=1 where id=10", then the condition clause "id=10" is extracted, and the first query statement is constructed as "select from table whereid=10". Through this first query statement, the state of the data records affected by this modification operation before the change can be located and obtained.

[0042] In an optional implementation, step S40 above executes a first query statement before executing the update-type SQL statement. This first query statement retrieves specific data records from the database, defined by conditional information. The set of these records includes the range of changed data corresponding to this operation and the original state of all data within that range before the change, i.e., the data before the change.

[0043] In an optional implementation, before executing the second query statement based on the changed data range, step S50 determines whether the execution of the update-type SQL statement is within a transaction environment. If it is within a transaction environment, the step of executing the second query statement is registered as a transaction synchronization callback to be triggered after the business transaction is committed. If it is not within a transaction environment, the second query statement is executed after the update-type SQL statement is completed.

[0044] Specifically, if the determination result indicates that the transaction is in a transaction environment, the operation of executing the second query statement is registered as a transaction synchronization callback task based on the transaction management mechanism. This callback task is only triggered after the business transaction has been verified to be successfully committed, in order to avoid the problem of premature consumption of changed data due to transaction failure and rollback, and to ensure that the data capture result is completely consistent with the semantics of the business transaction.

[0045] It should be noted that both the first and second query statements are used to query the same data range, but their execution timing and purpose are different: the first query statement is executed before the update SQL statement, and its purpose is to obtain the original state before the data change; the second query statement is executed after the transaction is committed, and its purpose is to obtain the final state after the data change.

[0046] Through the aforementioned differentiated processing mechanism, precise control over the execution timing of the second query statement is achieved at the application layer. This ensures semantic consistency in data change perception under transactional environments and maintains high processing efficiency in non-transactional scenarios. Thus, without relying on the underlying database logs, the state before and after data changes can be captured completely and reliably.

[0047] In some implementations, after generating data change information based on the data before and after the change, the process may further include: Data lineage analysis is performed based on data change information and the parsing results of update-type SQL statements to obtain lineage information; Bloodline information is stored in a metadata management system.

[0048] The data change information includes, but is not limited to, the data before the change, the data after the change, the operation type, and the data table that was changed. The parsed results of update SQL statements may include, but are not limited to, the original SQL statement and conditional clauses.

[0049] Specifically, the parsing results of data change information and update-type SQL statements are encapsulated and written into a change event report. This change event report is provided to the downstream data lineage analysis module through defined interfaces (such as an internal event bus, callback functions, or message queues). The data lineage analysis module identifies the data dependencies and flow relationships between tables, and even between fields, by comparing the differences between the data before and after the change, and combining this with the semantics of the SQL (for example, parsing INSERT INTO table_a SELECT FROM table_b can identify the data flow from table_b to table_a), i.e., the data lineage information. The analyzed data lineage information is then persistently stored in the metadata management system.

[0050] For a detailed understanding of the embodiments of this application, please refer to [link / reference]. Figure 2 In a specific example, the data change awareness method based on non-intrusive application-layer access in this embodiment includes: By encapsulating predefined business logic and custom interceptors into a Starter module, which is then included as a dependency by sub-applications in their pom.xml files, this module automatically intercepts update SQL statements when database data changes occur in a sub-application. It then parses the SQL to obtain the conditions and inversely calculates the SELECT query statement (the first query statement). The module then executes the original database change operation and, upon completion, determines whether the application is in a transaction environment. If no transaction is active, it immediately constructs and sends the changed content to the metadata service for lineage resolution. If a transaction is active, it waits for the transaction's final state: if the transaction is successfully committed, the changed content is sent to the metadata service; if the transaction is rolled back, the sending process is terminated, and no change message is generated.

[0051] Through the above process, accurate and consistent perception of data changes is achieved at the application layer, and structured change information that can be used for lineage analysis is output downstream.

[0052] Compared with existing data change capture schemes based on database logs, this application has the following technical advantages: First, it reduces dependence on database types: by intercepting the SQL execution process at the application layer to capture data changes, it avoids relying on the underlying database log mechanism, enabling the solution to adapt to database types that do not support Binlog, including but not limited to domestically produced databases.

[0053] Secondly, it enables seamless replacement of existing systems. By integrating change monitoring capabilities as a Starter dependency, business systems can smoothly migrate the data lineage collection scheme without modifying their original business code or database access methods.

[0054] 3. Ensure the consistency of the order in which metadata changes are received. Canal may have message out-of-order issues, which can lead to duplicate message consumption or error messages in some business scenarios. The MyBatis plugin extension can guarantee message order.

[0055] This application also provides a data change awareness system based on application-layer non-intrusive access, including: The interception module is used to intercept update SQL statements sent to the database before the data from the application layer is sent to the database. The parsing module is used to perform syntax parsing on the intercepted update-type SQL statements and identify their corresponding database operation types and condition information; The reverse query module is used to generate a first query statement based on the database operation type and the condition information; and to execute the first query statement to obtain the changed data range corresponding to the update SQL statement and the data before the change within the changed data range. The change query module is used to execute a second query statement based on the changed data range after the business transaction to which the update SQL statement belongs is successfully submitted, so as to obtain the changed data within the changed data range; The analysis module is used to generate data change information based on the data before the change and the data after the change.

[0056] In an optional implementation, the interception module is specifically used for: By using a custom interceptor plugin based on the MyBatis framework, the update operation methods of the database executor can be intercepted to capture update-type SQL statements. The interceptor plugin is defined as an independently integrateable module and registered to the application's data session through dependency import.

[0057] In an optional implementation, the reverse lookup module is specifically used for: If the database operation type is an update or delete operation, the first query statement is constructed based on the conditional clauses in the update SQL statement. If the database operation type is an insert operation, there is no need to construct the first query statement.

[0058] In an optional implementation, the change query module is further configured to: Before executing the second query statement based on the changed data range, determine whether the execution of the update SQL statement is within a transaction environment; If in a transactional environment, the step of executing the second query statement will be registered as a transaction synchronization callback to be triggered after the business transaction is committed. If not in a transaction environment, the second query statement will be executed after the update SQL statement has been executed.

[0059] In an optional implementation, the analysis module is further configured to: Data lineage analysis is performed based on data change information and the parsing results of update-type SQL statements to obtain lineage information; Bloodline information is stored in a metadata management system.

[0060] The specific implementation principles and beneficial effects of each of the above modules correspond one-to-one with the data change awareness method based on application layer non-intrusive access in the above embodiments, and will not be repeated here.

[0061] Accordingly, see Figure 3 , Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. In this embodiment, the electronic device includes a memory 101, a processor 100, and a computer program 102 stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer program 102, it implements all the steps of the data change awareness method based on application-layer non-intrusive access provided in the above embodiment.

[0062] The electronic device can be a mobile computing device (such as a mobile phone), a desktop computer, a laptop, a PDA, and a cloud server, etc. This electronic device may include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will understand that... Figure 3 This is merely an example of an electronic device and does not constitute a limitation on electronic devices. It may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0063] The processor 100 may be a microcontroller unit (MCU) or a central processing unit (CPU). It may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0064] In some embodiments, memory 101 may be an internal storage unit of an electronic device, such as a hard disk or memory. In other embodiments, memory 101 may be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, memory 101 may include both internal and external storage units. Memory 101 is used to store operating systems, applications, bootloaders, data, and other programs, such as program code for computer programs. Memory 101 can also be used to temporarily store data that has been output or will be output.

[0065] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon. These computer-readable program instructions are used to execute the data change sensing method based on application-layer non-intrusive access in the above embodiments. The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. Program code contained on a computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (Radio Frequency), etc., or any suitable combination thereof. The aforementioned computer-readable storage medium may be contained within an electronic device; or it may exist independently, not assembled into an electronic device.

[0066] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0067] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Modules described in the embodiments of this application can be implemented in software or hardware. The names of modules do not, in some cases, constitute a limitation on the unit itself.

[0068] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described data change awareness method based on application-layer non-intrusive access. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those provided in the above embodiments, and will not be repeated here.

[0069] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A data change awareness method based on non-intrusive application-layer access, characterized in that, include: Before the data from the application layer is sent to the database, intercept the update SQL statements destined for the database; The intercepted update SQL statements are parsed to identify their corresponding database operation types and condition information; A first query statement is generated based on the database operation type and the condition information; Execute the first query statement to obtain the range of changed data corresponding to the update SQL statement and the data before the change within the range of changed data; After the business transaction to which the update SQL statement belongs is successfully committed, a second query statement is executed based on the changed data range to obtain the changed data within the changed data range; Data change information is generated based on the data before the change and the data after the change.

2. The data change awareness method based on non-intrusive application-layer access as described in claim 1, characterized in that, The data change awareness method based on application-layer non-intrusive access also includes: Determine whether the execution of the update SQL statement is within a transaction environment; If in a transactional environment, the step of executing the second query statement will be registered as a transaction synchronization step. A callback is triggered after the business transaction is committed.

3. The data change awareness method based on non-intrusive application-layer access as described in claim 2, characterized in that, After determining whether the execution of the update SQL statement is within a transaction environment, the method further includes: If not in a transaction environment, the second query statement will be executed after the update SQL statement has been executed.

4. The data change awareness method based on non-intrusive application-layer access as described in claim 1, characterized in that, The step of generating the first query statement based on the database operation type and the condition information in the update SQL statement includes: If the database operation type is a modification operation or a deletion operation, then the first query statement is constructed based on the conditional clauses in the update-type SQL statement.

5. The data change awareness method based on non-intrusive application-layer access as described in claim 1, characterized in that, After generating data change information based on the data before and after the change, the method further includes: Based on the data change information and the parsing results of the update-type SQL statements, data lineage analysis is performed to obtain lineage information; The bloodline information is stored in the metadata management system.

6. The data change awareness method based on non-intrusive application-layer access as described in claim 1, characterized in that, The interception of update-type SQL statements destined for the database before data from the application layer is sent to the database includes: A custom interceptor plugin based on the MyBatis framework is used to intercept the update operation methods of the database executor in order to capture the update-type SQL statements.

7. The data change awareness method based on non-intrusive application-layer access as described in claim 6, characterized in that, The custom interceptor plugin is encapsulated as an independently integrateable module and registered to the application's data session via dependency import.

8. A data change sensing system based on application-layer non-intrusive access, characterized in that, include: The interception module is used to intercept update SQL statements sent to the database before the data from the application layer is sent to the database. The parsing module is used to perform syntax parsing on the intercepted update-type SQL statements and identify their corresponding database operation types and condition information; The reverse query module is used to generate a first query statement based on the database operation type and the condition information. And, execute the first query statement to obtain the range of changed data corresponding to the update SQL statement and the data before the change within the range of changed data; The change query module is used to execute a second query statement based on the changed data range after the business transaction to which the update SQL statement belongs is successfully submitted, so as to obtain the changed data within the changed data range; The analysis module is used to generate data change information based on the data before the change and the data after the change.

9. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the data change awareness method based on application-layer non-intrusive access as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the data change awareness method based on non-intrusive application-layer access as described in any one of claims 1 to 7.

Images