Database table tracing method and device

Abstract

The invention relates to a database table tracing method and device, and the method comprises the steps: carrying out the real-time monitoring of the operation state of a target database, and obtaining monitoring data; when the monitoring data is abnormal, triggering a restarting process of the target database; after the target database is restarted, recovering the data from the nearest database backup point to obtain the target database of which the data is recovered; extracting watermarks contained in a database table in the target database of the recovered data to obtain watermark information; and determining copyright information and a source identifier of a database table in the target database according to the watermark information, and determining a user and a propagation path of the database table in the target database in combination with an access record of the database table in the target database. Therefore, the consistency and integrity of the database table data can be ensured in a high-concurrency application scene, and the data processing efficiency and reliability are effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a database table tracing method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of data processing capabilities, there has emerged a technology for tracing the source of various data in the operation of the power grid signal monitoring automation data center, that is, verifying the integrity and consistency of the data by inserting special tags into the data.

[0003] In traditional technology, the private key of the database copyright owner is generally used to convert the watermark plaintext into a binary sequence; the watermark embedding procedure is performed separately for each attribute value as a non-numeric attribute column in natural language; and the database table with the embedded watermark is published. When a copyright dispute occurs, it is necessary to extract the watermark from the database table in dispute and use it as an effective proof of copyright ownership.

[0004] However, in high-concurrency scenarios, the amount of data surges, and the above traditional methods are difficult to maintain data consistency and integrity, and the data processing efficiency is low and the reliability is poor. Summary of the invention

[0005] Based on this, it is necessary to provide a database table tracing method, device, equipment, storage medium and program product that can ensure data consistency and integrity in high-concurrency application scenarios and effectively improve data processing efficiency and reliability in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for tracing the source of a database table, comprising: monitoring the operating status of a target database in real time to obtain monitoring data; when an abnormality exists in the monitoring data, triggering a restart process of the target database; after the target database completes the restart, restoring the data from the most recent database backup point to obtain a target database of restored data; extracting a watermark contained in a database table in the target database of restored data to obtain watermark information; determining the copyright information and source identification of the database table in the target database based on the watermark information, and determining the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

[0007] In one of the embodiments, before real-time monitoring of the operating status of the target database is performed to obtain monitoring data, the method further includes: performing watermark embedding processing on the database table to obtain a database table containing the watermark; completely inserting the database table containing the watermark into the end of the target database; and performing integrity verification on the database table containing the watermark inserted into the end of the target database.

[0008] In one embodiment, the watermark embedding process is performed on the database table to obtain the database table containing the watermark, including: obtaining the floating-point value field to be embedded with the watermark from the database table, and converting the floating-point value field to the frequency domain by Fourier transform to obtain the frequency domain coefficient matrix; performing block processing on the frequency domain coefficient matrix, and selecting the intermediate frequency region coefficient as the watermark embedding position; encoding the preset watermark information containing copyright information and timestamp into a binary sequence; embedding the binary sequence into the selected intermediate frequency region coefficient, and modulating the embedded binary sequence by quantization index; if the watermark embedding is completed, converting the frequency domain coefficient matrix containing the watermark information back to the time domain by inverse Fourier transform to obtain the numerical sequence containing the watermark information; rounding the numerical sequence containing the watermark information according to preset requirements to obtain the processed numerical sequence containing the watermark information; rewriting the processed numerical sequence containing the watermark information into the corresponding field of the original database table by SQL (Structured Query Language) update statement to obtain the database table containing the watermark.

[0009] In one embodiment, the step of inserting the database table containing the watermark completely into the tail of the target database includes: reading the data to be inserted from the database table containing the watermark, and obtaining the target database table structure information through an SQL statement; creating a temporary table according to the target database table structure information, wherein the temporary table is used to store the insertion status, and the insertion status includes: insertion progress, amount of inserted data, and breakpoint position field; constructing a batch insert statement, wherein the batch insert statement is generated according to the target database table structure information; setting the data amount threshold for each batch to be inserted to a preset number of records, and The method comprises the following steps: executing a statement insert operation in a pool; setting an insert operation once at a preset interval in a timed task mode, and monitoring the insertion process; if an insertion interruption is detected, reading the latest insertion status from the temporary table, obtaining the breakpoint position information, and continuing to execute the insertion operation from the breakpoint after reconstructing the insert statement; calculating the checksum of the inserted statement, wherein the checksum is calculated using the MD5 hash algorithm; comparing the data consistency of the source database table and the target database table based on the obtained checksum; if the check results are consistent, deleting the temporary table, and completing the insertion operation of the database table containing the watermark into the tail of the target database.

[0010] In one embodiment, the integrity verification of the database table containing the watermark inserted into the tail of the target database includes: obtaining all data of the database table containing the watermark in the target database, and reading all the data in a block reading manner, wherein the block reading manner refers to reading fixed rows of data each time; calculating a hash value based on the data read in blocks, wherein the hash value is obtained by a SHA-256 hash algorithm; connecting the hash values ​​corresponding to the data read in blocks to obtain a connected hash value; performing a hash calculation on the connected hash value to obtain a final hash value of the database table containing the watermark; obtaining a pre-stored original hash value from a hardware security module, wherein the hardware security module is connected to the target database; The databases communicate with each other through the TLS protocol; the final hash value of the database table containing the watermark and the original hash value are converted into byte arrays, and compared through a byte-by-byte XOR operation; if the results of the byte-by-byte XOR operation are all zero, the hash values ​​are determined to be consistent, and the comparison result is marked as verification passed; if the results of the byte-by-byte XOR operation are not all zero, the hash values ​​are determined to be inconsistent, and the comparison result is marked as verification failed; when the comparison result is marked as verification failed, the security monitoring process of the target database is triggered, and the abnormal event information is recorded, wherein the abnormal event information includes: timestamp, table name, original hash value, current hash value; an alarm containing the abnormal event information is sent to a designated receiving end through a preset alarm channel.

[0011] In one of the embodiments, when the monitoring data has an abnormality, the restart process of the target database is triggered, including: collecting the operating indicators of the target database, the operating indicators including: CPU usage, memory usage, number of connections, query response time; comparing the operating indicators of the target database with preset thresholds; if any operating indicator of the target database exceeds the preset threshold within the preset time, it is determined that an abnormality exists; when it is determined that an abnormality exists, the fault handling process of the target database is triggered, and a restart instruction is sent to the target database management program; wherein the restart instruction is used to instruct the target database management program to perform the following operations: close the current database instance; clean up the cache; reload the configuration file and start the target database instance; confirm that the target database instance has been successfully started and responds to query requests normally.

[0012] In one of the embodiments, after the target database completes restart, data is restored from the most recent database backup point to obtain a target database for restored data, including: after the target database completes restart, obtaining the most recent full backup and the incremental backup after the full backup; applying the full backup and the incremental backup to the target database in chronological order until the database table containing the watermark is restored to the state before the exception occurs.

[0013] In one of the embodiments, the watermark contained in the database table in the target database of the recovered data is extracted to obtain the watermark information, including: obtaining all the data of the database table containing the watermark from the target database, and converting the all the data to the frequency domain through Fourier transform to obtain a frequency domain coefficient matrix; processing the frequency domain coefficient matrix using a quantization index modulation algorithm, and extracting the watermark bits according to the quantization values ​​of adjacent coefficients; converting the extracted watermark bits to obtain a binary sequence, and performing verification on the binary sequence; decoding the binary sequence according to a predefined watermark information format to restore the metadata containing the copyright holder identifier and timestamp information; parsing the metadata to extract the copyright holder identifier and timestamp information; if the copyright holder identifier exists in the copyright information database, obtaining the complete copyright ownership information corresponding to the copyright holder identifier.

[0014] In one of the embodiments, the copyright information and source identification of the database table in the target database are determined based on the watermark information, and the user and dissemination path of the database table in the target database are determined in combination with the access record of the database table in the target database, including: obtaining the access record of the database table containing the watermark from the target database, the access record including: access time, access user, access operation type; generating a chronological access sequence based on the access record, and mining the access item set whose occurrence frequency exceeds the upper limit threshold to obtain the target access item set; generating association rules based on the target access item set, and calculating the confidence and lift of the association rules; if the confidence If the value of the watermark is greater than a preset confidence threshold and the lift is greater than a preset lift threshold, the association rule is determined to be a strong association rule; the main users of the database table containing the watermark and the potential propagation paths are determined according to the strong association rule, wherein the main users refer to the users with the highest frequency of occurrence, and the user sequences in the target access item set are used to characterize the potential propagation paths; a graph database is established in combination with the copyright information and source identification of the database table containing the watermark, wherein the nodes created in the graph database are used to characterize the users, and the edges created are used to characterize the potential propagation paths; an association graph is generated according to the main users, the potential propagation paths, the copyright information, and the source identification.

[0015] On the second aspect, the present application also provides a database table tracing device, including: a monitoring module, which is used to monitor the operating status of the target database in real time to obtain monitoring data; a trigger module, which is used to trigger the restart process of the target database when there is an abnormality in the monitoring data; a recovery module, which is used to restore data from the most recent database backup point after the target database completes the restart, and obtain the target database of the restored data; an extraction module, which is used to extract the watermark contained in the database table in the target database of the restored data to obtain watermark information; a tracing module, which is used to determine the copyright information and source identification of the database table in the target database based on the watermark information, and determine the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

[0016] In a third aspect, the present application also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program: real-time monitoring of the operating status of a target database to obtain monitoring data; when an abnormality exists in the monitoring data, triggering a restart process of the target database; after the target database completes the restart, restoring the data from the most recent database backup point to obtain a target database of restored data; extracting a watermark contained in a database table in the target database of restored data to obtain watermark information; determining the copyright information and source identification of the database table in the target database based on the watermark information, and determining the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

[0017] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the following steps: monitoring the operating status of a target database in real time to obtain monitoring data; when an abnormality exists in the monitoring data, triggering a restart process of the target database; after the target database completes the restart, restoring the data from the most recent database backup point to obtain a target database of restored data; extracting a watermark contained in a database table in the target database of restored data to obtain watermark information; determining the copyright information and source identification of the database table in the target database based on the watermark information, and determining the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

[0018] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the following steps when executed by a processor: monitor the operating status of the target database in real time to obtain monitoring data; when there is an abnormality in the monitoring data, trigger the restart process of the target database; after the target database completes the restart, restore the data from the most recent database backup point to obtain the target database of the restored data; extract the watermark contained in the database table in the target database of the restored data to obtain watermark information; determine the copyright information and source identification of the database table in the target database based on the watermark information, and determine the users and dissemination channels of the database table in the target database in combination with the access records of the database table in the target database.

[0019] The above-mentioned database table tracing method, device, computer equipment, computer-readable storage medium and computer program product obtain monitoring data by real-time monitoring of the running status of the target database; thus, the running status of the database can be monitored in real time to facilitate timely discovery of abnormal conditions. When the monitoring data is abnormal, the restart process of the target database is triggered; thus, the abnormal state can be responded to in a timely manner, the restart process is triggered, and the normal operation of the database is ensured. After the target database completes the restart, the data is restored from the most recent database backup point to obtain the target database of the restored data; thus, the data in the database can be quickly restored through the database backup point. The watermark contained in the database table in the target database of the restored data is extracted to obtain the watermark information; based on the watermark information, the copyright information and source identification of the database table in the target database are determined, and the user and dissemination path of the database table in the target database are determined in combination with the access record of the database table in the target database. Thus, the consistency and integrity of the database table data can be ensured in high-concurrency application scenarios, and the data processing efficiency and reliability can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] FIG1 is an application environment diagram of a database table tracing method in one embodiment;

[0022] FIG2 is a schematic diagram of a process of tracing a database table in one embodiment;

[0023] FIG3 is a schematic flow chart of a method for tracing the origin of a database table in another embodiment;

[0024] Figure 4 A schematic diagram of a method flow for watermark embedding and verification in one embodiment;

[0025] FIG5 is a schematic diagram of a method flow for performing hash verification on an embedded watermark in one embodiment;

[0026] FIG6 is a structural block diagram of a database table tracing device in one embodiment;

[0027] FIG7 is a structural block diagram of a database table tracing device in another embodiment;

[0028] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] The database table tracing method provided in the embodiment of the present application can be applied to the application environment shown in FIG. 1 . In which, the database server 110 is directly or indirectly connected to a plurality of storage volumes storing the database 120. One or more application objects storing data in the database 120 can access the database 120. The database server 110 is connected to a network 130, which provides access to and from the database server. The network 130 can be an enterprise-level network for an organization, and can include but is not limited to a local area network, a metropolitan area network, a wide area network, and similar networks. The network 130 can be supported by standard network hardware and protocols (including, for example, Ethernet routers, optical cables, etc.). The network 130 can also connect computer devices (for example, clients 140 (1)-(N)) in one or more locations of the enterprise, thereby providing communication from those clients to the database server 110. The network shown in FIG. 1 is also connected to a network server 150. The network server 150 can provide user access to an application, which is associated with one or more application objects. The network 130 may have one or more network servers 150, each of which provides a separate application or provides multiple network servers that distribute the same application. The network server 150 provides an application object model by communicating with the database server 110 to access the data stored in the database 120. Such communication between the network server 150 and the database server 110 may be through an application program interface (API), which is shared by multiple application objects, which are served by the network server 150 and the database server 110. The backup server 160 is connected to the network 130 and is configured to store backup images of the data stored on the network 130 in one or more backup storage volumes 170. The backup server 160 can be configured to run multiple types of backup utilities, which provide multiple snapshots of the data on the network when performing backups. For example, the backup server 160 can back up the database 120 by directly accessing the data via the database server 110.

[0031] In an exemplary embodiment, as shown in FIG2 , a method for tracing the source of a database table is provided, and the method is applied to the network architecture in FIG1 as an example for description, including the following steps 201 to 205. Among them:

[0032] Step 201: monitor the operating status of the target database in real time to obtain monitoring data.

[0033] In this embodiment, by monitoring the running status of the target database in real time, when a database abnormality is detected, the database restart process is triggered, and the data is restored from the most recent database backup point after restart to ensure the integrity and consistency of the database table containing the watermark inserted at the end of the target database.

[0034] Step 202: When there is an abnormality in the monitoring data, a restart process of the target database is triggered.

[0035] In this embodiment, the operating indicators of the target database are collected, and the operating indicators include: CPU usage, memory usage, number of connections, and query response time; the operating indicators of the target database are compared with preset thresholds; if any operating indicator of the target database exceeds the preset threshold within a preset time, it is determined that an abnormality exists; when it is determined that an abnormality exists, the fault handling process of the target database is triggered, and a restart instruction is sent to the target database management program; wherein the restart instruction is used to instruct the target database management program to perform the following operations: close the current database instance; clean up the cache; reload the configuration file and start the target database instance; confirm that the target database instance has been successfully started and responds to query requests normally.

[0036] Exemplarily, the database operation indicators (including CPU usage, memory usage, number of connections, query response time and other parameters) can be collected in real time through the database monitoring agent, and the collected operation indicators are compared with the preset thresholds. According to the historical operation data, it is assumed that the CPU usage threshold is set to 85%, the memory usage threshold is set to 90%, the connection number threshold is set to 95% of the maximum number of connections, and the query response time threshold is set to 500 milliseconds. The database operation status is judged according to the comparison results. If any indicator exceeds the preset threshold for 5 consecutive minutes, it is determined to be an abnormal state, triggering the database fault handling process, and sending a restart instruction to the database management program. After receiving the restart instruction, the database management program performs a series of operations such as closing the current database instance, clearing the cache, reloading the configuration file, and starting the database instance, and confirms that the database instance has been successfully started and responds to query requests normally through the heartbeat detection mechanism.

[0037] Step 203: after the target database is restarted, data is restored from the most recent database backup point to obtain a target database of restored data.

[0038] In this embodiment, after the target database completes restart, the most recent full backup and the incremental backup after the full backup are obtained; the full backup and the incremental backup are applied to the target database in chronological order until the database table containing the watermark is restored to the state before the exception occurs.

[0039] For example, after the database is restarted, the most recent full backup and subsequent incremental backups are selected, and these backups are applied in chronological order until the latest state before the exception occurs is restored, and the database table containing watermark information is restored to the state before the exception occurs, thereby ensuring the integrity and consistency of the data. For exceptions such as query performance degradation, database cache cleanup and index reconstruction are performed first; for exceptions such as insufficient storage space, the automatic expansion mechanism is triggered. During the database fault detection and recovery process, a monitoring agent named DBMonitor (data visualization monitoring software) is deployed to collect database operation indicators every 5 seconds. Taking the PostgreSQL database as an example, the CPU usage, memory usage, number of connections, and query response time are obtained through the pg_stat_activity view. Set the CPU usage threshold to 85%, the memory usage threshold to 90%, the number of connections threshold to 950, assuming that the maximum number of connections is 1000, and the query response time threshold to 500 milliseconds. DBMonitor uses a sliding window algorithm to calculate the average value of indicators in the past 5 minutes. If an exception is found, such as the CPU usage exceeds 85% for 5 consecutive minutes, the fault handling process is triggered. Send a restart instruction to the database management program DBManager through the SSH protocol. DBManager executes the pg_ctlstop command to stop the database, cleans up temporary files in the / tmp directory, reloads the postgresql.conf configuration file, and then executes pg_ctlstart to start the database. Use the pg_isready command to check the database status every second until it returns "accepting connections". After the restart is complete, DBManager calls the pg_basebackup tool (a built-in backup tool) to restore the most recent full backup, and then uses pg_receivewal to apply the WAL log for incremental recovery. For query performance degradation, execute the VACUUMANALYZE command to clean up the cache and update statistics; for insufficient storage space, call the AWS API to automatically expand the EBS volume capacity.

[0040] Step 204: extract the watermark contained in the database table in the target database for restoring data to obtain watermark information.

[0041] In this embodiment, all data of a database table containing a watermark is obtained from a target database, and the all data is converted to the frequency domain through Fourier transform to obtain a frequency domain coefficient matrix; the frequency domain coefficient matrix is ​​processed by a quantization index modulation algorithm, and watermark bits are extracted according to the quantization values ​​of adjacent coefficients; the extracted watermark bits are converted to obtain a binary sequence, and the binary sequence is verified; according to a predefined watermark information format, the binary sequence is decoded to restore metadata containing a copyright holder identifier and timestamp information; the metadata is parsed to extract the copyright holder identifier and timestamp information; if the copyright holder identifier exists in a copyright information database, the complete copyright ownership information corresponding to the copyright holder identifier is obtained.

[0042] Exemplarily, all data including the watermark database table are read from the target database, and the data is converted to the frequency domain by Fourier transform to obtain the frequency domain coefficient matrix, and the coefficients in the intermediate frequency region are selected for watermark extraction, because these regions are most sensitive to watermark embedding and extraction. The frequency domain coefficient matrix is ​​processed by a quantization index modulation algorithm, and the quantization step is set to 0.1. The watermark bits are extracted by comparing the quantization values ​​of adjacent coefficients, and the extracted watermark information is converted into a binary sequence. The extracted binary sequence is checked and calculated to ensure that the extracted watermark information is complete and correct. According to the predefined watermark information format, the binary sequence is decoded, and it is assumed that the watermark information format is: 32-bit copyright holder identifier + 64-bit timestamp + 32-bit checksum, and metadata including copyright holder information and timestamp is restored. The restored metadata is parsed, the copyright holder identifier and timestamp information are extracted, and the complete copyright ownership information and the generation source of the database table are obtained by querying the copyright information database. Among them, the copyright information database contains fields such as copyright holder identifier, copyright holder name, contact information, etc., and index query is performed through the copyright holder identifier. In the process of digital watermark extraction, all data is read from the "watermarked_table" table of the target database, a total of 1 million records. The data is converted to the frequency domain using the FFT algorithm (Fast Fourier Transform Algorithm) to obtain a 1 million x 1 million complex matrix. The intermediate frequency coefficients of the 10000th to 20000th rows and columns in the matrix are selected for watermark extraction. The quantization index modulation algorithm is used, and the quantization step is set to 0.1, and the quantization values ​​of adjacent coefficients are compared. For example, if the difference between the quantization values ​​of the coefficients at positions (10000, 10000) and (10001, 10001) is greater than 0, the extracted watermark bit is 1, otherwise it is 0. Repeat this process to extract a 128-bit binary sequence. Calculate the CRC-32 checksum for the extracted sequence to ensure integrity. Decoded according to the predefined format, the first 32 bits "0x1A2B3C4D" represent the copyright holder identifier, the next 64 bits "1627372800" represent the Unix timestamp (2021-07-27 12:00:00 UTC), and the last 32 bits are the checksum. To query the copyright information database table "copyright_info", use the SQL statement:

[0043] "SELECT*FROMcopyright_infoWHEREidentifier='0x1A2B3C4D'"

[0044] Get complete information. The result shows that the copyright owner is "AcmeCorporation" and is sent to the mailbox of the relevant personnel, realizing the extraction and verification of copyright information of the database table containing watermarks.

[0045] Step 205, determining the copyright information and source identifier of the database table in the target database based on the watermark information, and determining the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

[0046] In this embodiment, access records of a database table containing a watermark are obtained from a target database, wherein the access records include: access time, access user, and access operation type; an access sequence arranged in chronological order is generated based on the access records, and an access item set whose occurrence frequency exceeds an upper threshold is mined to obtain a target access item set; association rules are generated based on the target access item set, and the confidence and lift of the association rules are calculated; if the confidence is greater than a preset confidence threshold, and the lift is greater than a preset lift threshold, the association rule is determined to be a strong association rule; the main user of the database table containing the watermark and the potential propagation path are determined based on the strong association rule, wherein the main user refers to the user with the highest frequency of occurrence, and the user sequence in the target access item set is used to characterize the potential propagation path; a graph database is established in combination with the copyright information and source identification of the database table containing the watermark, wherein the nodes created in the graph database are used to characterize the users, and the edges created are used to characterize the potential propagation paths; an association graph is generated based on the main users, the potential propagation paths, the copyright information, and the source identification.

[0047] Exemplarily, the access records of the watermarked database table are extracted from the target database (including access time, access user, access operation type; access operation type includes query, insert, update, delete), and these access records are arranged in chronological order to form an access sequence. The Apriori algorithm is used to process the access sequence. According to the amount of data, the minimum support threshold is set to 1%, and the frequently occurring access item sets are mined to obtain the common usage pattern of the database table. Association rules are generated according to the frequent item sets, and the confidence and lift are calculated. The confidence threshold is set to 0.6 and the lift threshold is set to 1.5, and strong association rules are screened to identify the main users and potential propagation paths of the database table. The user with the highest frequency is marked as the main user, and the user sequence in the frequent item set is regarded as a potential propagation path. Combined with the copyright information and source identification of the database table, the mined usage pattern and propagation path are annotated, and the association relationship is stored and visualized using a graph database, where nodes represent users and edges represent data propagation, and an association graph containing copyright information, usage patterns and propagation paths is generated.

[0048] For example, in the database table usage pattern analysis, we first extract the access records of the last 30 days from the access_log table of the target database, a total of 1 million records. Each record contains a timestamp, user ID, and operation type (1-query, 2-insert, 3-update, 4-delete). Use the SQL statement:

[0049] "SELECT * FROM access_log WHERE table_name = 'watermarked_table' AND access_time > = DATE_SUB (CURDATE (), INTERVAL30DAY) ORDER BY access_time" Get the ordered access sequence. Apply the Apriori algorithm, set the minimum support to 1%, that is, at least 10,000 times, and mine frequent itemsets. For example, it is found that the support of {user A-query, user B-update} is 1.5%. When generating association rules, calculate the confidence conf(A→B) = support(A∪B) / support(A), if it is greater than 0.6, it is retained; calculate the lift lift(A→B) = conf(A→B) / support(B), if it is greater than 1.5, it is considered a strong rule. Identify user ID 12345 as the most frequent visitor, with 50,000 appearances, and mark it as the main user. The frequent item set {12345-query, 67890-update, 24680-query} is considered as a potential propagation path.

[0050] Finally, use the Neo4j graph database to store the association relationship and create nodes:

[0051] "CREATE(u:User{id:12345,role:'main user'})"

[0052] Create edges:

[0053] "CREATE(u1)-[:TRANSFER{frequency:1000}]->(u2)" indicates data propagation.

[0054] Through the Cypher query language:

[0055] "MATCHp=(u1:User)-[:TRANSFER*]->(u2:User)WHEREu1.role='Main user'RETURNp" visualizes the propagation path to form a complete usage pattern and propagation analysis map.

[0056] The above-mentioned database table tracing method, device, computer equipment, computer-readable storage medium and computer program product obtain monitoring data by real-time monitoring of the running status of the target database; thus, the running status of the database can be monitored in real time to facilitate timely discovery of abnormal conditions. When the monitoring data is abnormal, the restart process of the target database is triggered; thus, the abnormal state can be responded to in a timely manner, the restart process is triggered, and the normal operation of the database is ensured. After the target database completes the restart, the data is restored from the most recent database backup point to obtain the target database of the restored data; thus, the data in the database can be quickly restored through the database backup point. The watermark contained in the database table in the target database of the restored data is extracted to obtain the watermark information; based on the watermark information, the copyright information and source identification of the database table in the target database are determined, and the user and dissemination path of the database table in the target database are determined in combination with the access record of the database table in the target database. Thus, the consistency and integrity of the database table data can be ensured in high-concurrency application scenarios, and the data processing efficiency and reliability can be effectively improved.

[0057] In another exemplary embodiment, as shown in FIG3 , a method for tracing the source of a database table is provided, and the method is applied to the network architecture in FIG1 as an example for description, including the following steps 301 to 308 . Among them:

[0058] Step 301: Perform watermark embedding processing on the database table to obtain a database table containing the watermark.

[0059] In this embodiment, a watermark embedding process is performed on the table to be inserted into the database, a digital watermark containing copyright information and timestamp metadata is embedded in the database table, and the database table is converted to the frequency domain through Fourier transform, and the corresponding frequency components are selected in the frequency domain to embed the watermark information. After embedding, an inverse Fourier transform is performed to obtain the database table containing the watermark.

[0060] Exemplarily, a floating-point numerical field to be embedded with a watermark is obtained from a database table, and the floating-point numerical field is converted to the frequency domain by Fourier transform to obtain a frequency domain coefficient matrix; the frequency domain coefficient matrix is ​​processed in blocks, and the intermediate frequency region coefficients are selected as the watermark embedding position; the preset watermark information containing copyright information and timestamp is encoded into a binary sequence; the binary sequence is embedded into the selected intermediate frequency region coefficients, and the embedded binary sequence is modulated by a quantization index; if the watermark embedding is completed, the frequency domain coefficient matrix containing the watermark information is converted back to the time domain by inverse Fourier transform to obtain a numerical sequence containing the watermark information; the numerical sequence containing the watermark information is rounded according to preset requirements to obtain a processed numerical sequence containing the watermark information; the processed numerical sequence containing the watermark information is rewritten into the corresponding field of the original database table by an SQL update statement to obtain a database table containing the watermark.

[0061] Exemplarily, the database table is preprocessed according to a preset watermark algorithm, and the floating-point numerical field to be embedded with the watermark is extracted from the database table. The extracted numerical sequence is converted to the frequency domain by Fourier transform to obtain a frequency domain coefficient matrix. The frequency domain coefficient matrix is ​​divided into blocks, and the intermediate frequency region coefficient is selected as the watermark embedding position, and the watermark information containing copyright information and timestamp is encoded into a binary sequence. The quantization step size and embedding strength parameters required for quantization index modulation are prepared, and the quantization step size is adaptively calculated according to the data distribution characteristics and the preset watermark strength. For the selected intermediate frequency region coefficient, the quantization index modulation is used to embed the encoded watermark information, and the quantization step size is adjusted to balance the robustness and imperceptibility of the watermark, and the frequency domain coefficients after the watermark is embedded are inversely transformed. The frequency domain data containing the watermark is converted back to the time domain by inverse Fourier transform, and the converted numerical sequence is rounded according to the original field format and precision requirements, and then rewritten into the corresponding field of the original database table by SQL update statement, completing the embedding process of the digital watermark, and generating a database table containing an invisible watermark.

[0062] For example, Figure 4As shown in the figure, in the process of watermark embedding in the database table, firstly, a floating-point numerical field, such as a commodity price column, is extracted from the table. The extracted numerical sequence is Fourier transformed to convert it into a frequency domain coefficient matrix. For example, a sequence containing 1000 price data is fast Fourier transformed to obtain a 1000×1000 complex matrix. Then the matrix is ​​divided into 8×8 sub-blocks, and the intermediate frequency region (such as the 3rd to 6th rows and columns) is selected as the watermark embedding position. The watermark information contains a 32-bit copyright identifier and a 64-bit timestamp, which is encoded as a 96-bit binary sequence. According to the data distribution characteristics, the initial quantization step size is set to 0.01 and the embedding strength coefficient is set to 0.1. The watermark information is embedded in the selected intermediate frequency coefficient using the quantization index modulation method. The specific operation is: if the watermark bit to be embedded is 1, the coefficient is adjusted to the nearest odd multiple of the quantization step size value; if it is 0, it is adjusted to the nearest even multiple of the quantization step size value. During the embedding process, the quantization step size is dynamically adjusted to float between 0.005 and 0.02 by monitoring the watermark strength and data distortion. After the embedding is completed, the modified frequency domain coefficients are inverse Fourier transformed to convert the data back to the time domain. Finally, the numerical sequence containing the watermark is rounded to retain the original precision (such as retaining two decimal places), and the processed data is updated back to the corresponding fields of the original table using the SQLUPDATE statement, thereby generating a database table containing the invisible watermark.

[0063] Step 302: insert the database table containing the watermark completely into the end of the target database.

[0064] In this embodiment, a database table tail insertion method is used to insert a watermarked database table into a target database. During the insertion process, the insertion status is monitored in real time, and the insertion progress and inserted data volume information are recorded. When the insertion process is interrupted, the insertion operation is continued from the breakpoint according to the recorded progress information to complete the complete insertion of the watermarked database table into the tail of the target database.

[0065] For example, Figure 4As shown, further, the data to be inserted is read from the database table containing the watermark, and the target database table structure information is obtained through an SQL statement; a temporary table is created according to the target database table structure information, wherein the temporary table is used to store the insertion status, and the insertion status includes: insertion progress, amount of inserted data, and breakpoint position field; a batch insert statement is constructed, wherein the batch insert statement is generated according to the target database table structure information; the data amount threshold for each batch of insertion is set to a preset number of records, and the statement insertion operation is executed through the connection pool of the target database; a scheduled task is used to set an insert operation to be executed once every preset time interval, and the insertion process is monitored; if an insertion interruption is detected, the latest insertion status is read from the temporary table, the breakpoint position information is obtained, and after the insert statement is reconstructed, the insert operation is continued from the breakpoint; the checksum of the inserted statement is calculated, wherein the checksum is calculated using the MD5 hash algorithm; according to the obtained checksum, the data consistency of the source database table and the target database table is compared; if the check results are consistent, the temporary table is deleted to complete the insertion operation of the database table containing the watermark to the tail of the target database.

[0066] Exemplarily, the data to be inserted is read from the watermarked database table, the target database table structure information is obtained through the SQL statement DESCTABLE_NAME, and a temporary table is created to store the insertion status (including fields such as insertion progress, amount of data inserted, and breakpoint position). According to the target database table structure, a batch insert statement is constructed, the data volume threshold for each batch of insertion is set to 1,000 records, the maximum number of connections in the connection pool is configured to be 20, and the minimum number of idle connections is configured to be 5. The insertion operation is performed through the database connection pool, and the insertion status information in the temporary table is updated at the same time. A scheduled task is used to execute every 30 seconds to monitor the insertion process. If an insertion interruption is detected, the latest insertion status is read from the temporary table, the breakpoint location information is obtained, and the insert statement is reconstructed to continue the insertion operation from the breakpoint. After each batch of insertion is completed, the checksum of the inserted data is calculated and stored in the temporary table.

[0067] Furthermore, the maximum ID (identity) value of the target table is obtained through the SQL statement SELECTMAX(ID)FROMTABLE_NAME as the starting ID of the newly inserted data to achieve tail insertion. The MD5 hash algorithm is used to calculate the data checksum and compare the data consistency of the source database table and the target database table. If the check result is consistent, the temporary table is deleted to complete the tail insertion process of the watermarked database table into the target database. In the process of inserting the tail of the database table, the data to be inserted is first read from the watermarked database table "source_table", and the target database table structure information is obtained through the SQL statement "DESCtarget_table". Create a temporary table named "insert_status", which contains the fields "progress" (insertion progress), "inserted_count" (amount of inserted data) and "breakpoint" (breakpoint position).

[0068] Furthermore, according to the target table structure, construct a batch insert statement:

[0069] "INSERTINTOtarget_tableSELECT*FROMsource_tableWHEREid>?LIMIT1000".

[0070] Configure the HikariCP connection pool, set the maximum number of connections to 20, and the minimum number of idle connections to 5. Use Java's ScheduledExecutorService to create a scheduled task and perform a monitoring operation every 30 seconds. During the insertion process, the SQL statement "SELECTMAX(id)FROMtarget_table" is used to obtain the maximum ID value of the target table as the starting ID of the newly inserted data to achieve tail insertion. After inserting 1000 records in each batch, use Java's MessageDigest class to calculate the MD5 checksum and store the result in the "checksum" field of the insert_status table. If the insertion is interrupted, read the breakpoint position from the insert_status table, reconstruct the insert statement and continue to execute. After all data is inserted, compare the MD5 checksums of source_table and target_table. If they are consistent, delete the insert_status table to complete the entire insertion process. In the whole process, SQL statements are executed through Java's PreparedStatement to avoid SQL injection risks, and batch processing is used to improve insertion efficiency.

[0071] Step 303: perform integrity verification on the database table containing the watermark inserted into the end of the target database.

[0072] In this embodiment, an integrity check is performed on the watermarked database table that has been inserted into the end of the target database by calculating the hash value of the database table and comparing it with the pre-stored hash value. If the hash values ​​are consistent, it is determined that the database table has not been tampered with and the integrity verification passes; if the hash values ​​are inconsistent, it is determined that the database table has been tampered with and an integrity verification failure alarm is triggered.

[0073] Exemplarily, all data of a database table containing a watermark in a target database are obtained, and all the data are read in a block reading manner, wherein the block reading manner refers to reading fixed rows of data each time; a hash value is calculated based on the data read in blocks, and the hash value is obtained by a SHA-256 hash algorithm; the hash values ​​corresponding to the data read in blocks are connected to obtain a connected hash value; a hash calculation is performed on the connected hash value to obtain a final hash value of the database table containing the watermark; a pre-stored original hash value is obtained from a hardware security module, wherein the hardware security module communicates with the target database through a TLS protocol; The final hash value of the watermarked database table and the original hash value are converted into byte arrays and compared through byte-by-byte XOR operation; if the results of the byte-by-byte XOR operation are all zero, the hash values ​​are determined to be consistent, and the comparison result is marked as verification passed; if the results of the byte-by-byte XOR operation are not all zero, the hash values ​​are determined to be inconsistent, and the comparison result is marked as verification failed; when the comparison result is marked as verification failed, the security monitoring process of the target database is triggered, and the abnormal event information is recorded, wherein the abnormal event information includes: timestamp, table name, original hash value, current hash value; an alarm containing the abnormal event information is sent to a designated receiving end through a preset alarm channel.

[0074] Exemplarily, as shown in FIG5 , all data of the watermarked database table are read from the target database. For large database tables, block reading and calculation are adopted. Each time 1 million rows of data are read for SHA-256 hash algorithm calculation. Finally, the hash values ​​of all blocks are connected and hash calculation is performed again to obtain the final hash value of the current database table. The original hash value stored in advance is read from the hardware security module, and the confidentiality and integrity of the reading process are ensured through the TLS1.3 protocol. The hash value currently calculated and the pre-stored hash value are converted into a byte array, and compared by byte-by-byte XOR operation. If all byte XOR results are 0, the hash values ​​are determined to be completely consistent, and the comparison result is marked as "verification passed"; otherwise, the verification failure process is transferred to. If the hash value comparison results are inconsistent, the comparison result is marked as "verification failed". At the same time, the database security monitoring module is triggered to record abnormal events (including timestamp, table name, original hash value and current hash value), and send alarm information containing abnormal details to the designated receiving end through the preset alarm channel. In the process of database integrity verification, JDBC (a standard application interface for connecting and operating databases in Java) is first used to connect to the target database and read the watermarked database table named "watermarked_table". Since the table contains 50 million rows of data, a block processing strategy is adopted to read 1 million rows each time. The SHA-256 algorithm is implemented using Java's MessageDigest class to calculate the hash value for each block of data. For example, the hash value of the first block of data is "a1b2c3...", the second block is "d4e5f6...", and so on. The hash values ​​of all blocks are concatenated into a string "a1b2c3...d4e5f6...", and the final hash value "7890ab..." is calculated again. Subsequently, a connection is made to the hardware security module (HSM) through the secure socket layer (SSL), and the original hash value "7890ab..." is read from the HSM using the predefined key ID "HASH_KEY_001". The two 64-byte hash values ​​are converted into byte arrays and compared byte by byte using Java's bitwise XOR operator "^". If the XOR result of all bytes is 0, it is marked as "verification passed". Otherwise, the verification failure process is triggered, and the preset DatabaseSecurityMonitor class is called to record the abnormal event, including the current timestamp "2024-07-0914:30:25", the table name "watermarked_table", the original hash value "7890ab..." and the current hash value "7890cd...". Finally, an alarm email is sent to the mailbox of the security manager through the configured JavaMailAPI (JAVA mail service application interface), and the email content contains the above abnormal details.

[0075] Step 304: monitor the operating status of the target database in real time to obtain monitoring data.

[0076] Step 305: When there is an abnormality in the monitoring data, a restart process of the target database is triggered.

[0077] Step 306: After the target database is restarted, data is restored from the most recent database backup point to obtain a target database of restored data.

[0078] Step 307: extract the watermark contained in the database table in the target database for restoring data to obtain watermark information.

[0079] Step 308, determining the copyright information and source identifier of the database table in the target database based on the watermark information, and determining the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

[0080] In this embodiment, the specific implementation process and technical effects of steps 304 to 308 can be found in the relevant descriptions of steps 201 to 205 shown in FIG. 2 , which will not be repeated here.

[0081] The method in this embodiment aims to cope with the challenge of balancing the surge in data volume with data security, system stability and high performance requirements; by embedding digital watermarks into database tables, and using Fourier transform to cleverly embed watermarks containing copyright information and timestamps in the frequency domain, effective identification and tamper-proofing of data copyright are achieved, while ensuring that the integrity and consistency of the original data are not affected, solving the limitations of traditional data insertion methods in copyright protection and data integrity verification, and is particularly suitable for power grid monitoring systems with high security requirements. Secondly, an optimization strategy for inserting the tail of the database table is adopted, supplemented by a real-time monitoring mechanism to ensure the efficiency and stability of the data writing process. Even if an interruption occurs during the data insertion process, it can be seamlessly continued from the breakpoint, ensuring the continuity of the data and the smoothness of the system operation. At the same time, by calculating and comparing the hash values ​​of the database table in real time, signs of data tampering can be immediately discovered, an alarm is triggered and a data recovery plan is initiated, effectively avoiding the performance bottleneck caused by locking the entire table, and ensuring the atomicity and consistency of transactions in high concurrency scenarios; once again, the database operation status is monitored in real time, and once an abnormality is detected, the automatic recovery process is immediately started to restore the data from the most recent backup point to ensure the consistency and integrity of the data. In addition, by extracting watermark information, analyzing copyright ownership and generation sources, and combining access records and association rule mining algorithms, it is also possible to gain insight into data usage patterns and dissemination paths, identify major user groups and data dissemination channels, and provide valuable analytical basis for data management and decision-making.

[0082] This embodiment, by integrating efficient digital watermarking technology with advanced database management, not only enhances data security and copyright protection capabilities, but also greatly improves the efficiency and reliability of data processing, providing strong technical support for the stable operation and data security protection of the power grid signal monitoring automation data center. It has found an ideal balance between ensuring data security, system stability and high performance, meeting the high standards of data processing in modern power grid monitoring systems, and opening up a new path for the digital transformation and intelligent upgrading of the power industry.

[0083] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0084] Based on the same inventive concept, the embodiment of the present application also provides a database table tracing device for implementing the above-mentioned database table tracing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiment of the tracing device for one or more database tables provided below can refer to the limitations of the database table tracing method above, and will not be repeated here.

[0085] In an exemplary embodiment, as shown in FIG6 , a database table tracing device is provided, including: a monitoring module 601, a triggering module 602, a recovery module 603, an extraction module 604 and a tracing module 605, wherein:

[0086] The monitoring module 601 is used to monitor the operation status of the target database in real time and obtain monitoring data;

[0087] A trigger module 602 is used to trigger a restart process of the target database when there is an abnormality in the monitoring data;

[0088] A recovery module 603 is used to restore data from the most recent database backup point after the target database is restarted to obtain a target database for restoring data;

[0089] An extraction module 604 is used to extract the watermark contained in the database table in the target database of the restored data to obtain the watermark information;

[0090] The tracing module 605 is used to determine the copyright information and source identification of the database table in the target database according to the watermark information, and determine the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

[0091] Exemplarily, the trigger module 602 is specifically used to: collect operating indicators of the target database, the operating indicators including: CPU usage, memory usage, number of connections, and query response time; compare the operating indicators of the target database with preset thresholds; if any operating indicator of the target database exceeds the preset threshold within the preset time, it is determined that an abnormality exists; when it is determined that an abnormality exists, the fault handling process of the target database is triggered, and a restart instruction is sent to the target database management program; wherein the restart instruction is used to instruct the target database management program to perform the following operations: close the current database instance; clean up the cache; reload the configuration file and start the target database instance; confirm that the target database instance has been successfully started and responds to query requests normally.

[0092] Exemplarily, the recovery module 603 is specifically used to: after the target database completes restart, obtain the most recent full backup and the incremental backup after the full backup; apply the full backup and the incremental backup to the target database in chronological order until the database table containing the watermark is restored to the state before the exception occurs.

[0093] Exemplarily, the extraction module 604 is specifically used to: obtain all data of the database table containing the watermark from the target database, and convert all the data to the frequency domain through Fourier transform to obtain a frequency domain coefficient matrix; use a quantization index modulation algorithm to process the frequency domain coefficient matrix, and extract watermark bits according to the quantization values ​​of adjacent coefficients; convert the extracted watermark bits to obtain a binary sequence, and perform verification on the binary sequence; decode the binary sequence according to a predefined watermark information format to restore the metadata containing the copyright holder identifier and timestamp information; parse the metadata to extract the copyright holder identifier and timestamp information; if the copyright holder identifier exists in the copyright information database, obtain the complete copyright ownership information corresponding to the copyright holder identifier.

[0094] The traceability module 605 is specifically used to: obtain access records of the database table containing the watermark from the target database, and the access records include: access time, access user, and access operation type; generate an access sequence arranged in chronological order according to the access records, and mine the access item set whose occurrence frequency exceeds the upper limit threshold to obtain the target access item set; generate association rules according to the target access item set, and calculate the confidence and lift of the association rules; if the confidence is greater than a preset confidence threshold, and the lift is greater than a preset lift threshold, the association rule is determined to be a strong association rule; determine the main user of the database table containing the watermark and the potential propagation path according to the strong association rule, wherein the main user refers to the user with the highest frequency of occurrence, and the user sequence in the target access item set is used to characterize the potential propagation path; combine the copyright information and source identification of the database table containing the watermark to establish a graph database, wherein the nodes created in the graph database are used to characterize the users, and the edges created are used to characterize the potential propagation paths; generate an association graph according to the main users, the potential propagation paths, the copyright information, and the source identification.

[0095] In another exemplary embodiment, as shown in FIG. 7 , a database table tracing device is provided, which, based on the device shown in FIG. 6 , further includes: a watermark embedding module 606 , an insertion module 607 , and a verification module 608 ;

[0096] The watermark embedding module 606 is used to perform watermark embedding processing on the database table to obtain a database table containing the watermark;

[0097] The inserting module 607 is used to completely insert the database table containing the watermark into the end of the target database;

[0098] The verification module 608 is used to verify the integrity of the database table containing the watermark inserted into the end of the target database.

[0099] Exemplarily, the watermark embedding module 606 is specifically used to: obtain a floating-point numerical field to be embedded with a watermark from a database table, and convert the floating-point numerical field to the frequency domain using Fourier transform to obtain a frequency domain coefficient matrix; perform block processing on the frequency domain coefficient matrix, and select intermediate frequency region coefficients as watermark embedding positions; encode preset watermark information containing copyright information and timestamp into a binary sequence; embed the binary sequence into the selected intermediate frequency region coefficients, and modulate the embedded binary sequence using a quantization index; if the watermark embedding is completed, convert the frequency domain coefficient matrix containing the watermark information back to the time domain through an inverse Fourier transform to obtain a numerical sequence containing the watermark information; round the numerical sequence containing the watermark information according to preset requirements to obtain a processed numerical sequence containing the watermark information; rewrite the processed numerical sequence containing the watermark information into the corresponding field of the original database table through an SQL update statement to obtain a database table containing the watermark.

[0100] Exemplarily, the insertion module 607 is specifically used to: read the data to be inserted from the database table containing the watermark, and obtain the target database table structure information through SQL statements; create a temporary table according to the target database table structure information, wherein the temporary table is used to store the insertion status, and the insertion status includes: insertion progress, amount of inserted data, and breakpoint position field; construct a batch insert statement, wherein the batch insert statement is generated according to the target database table structure information; set the data volume threshold of each batch to a preset number of records, and execute the statement insertion operation through the connection pool of the target database; set the insertion operation to be performed once every preset time interval in a timed task mode, and monitor the insertion process; if an insertion interruption is detected, read the latest insertion status from the temporary table, obtain the breakpoint position information, and continue to execute the insertion operation from the breakpoint after reconstructing the insert statement; calculate the checksum of the inserted statement, wherein the checksum is calculated using the MD5 hash algorithm; compare the data consistency of the source database table and the target database table according to the obtained checksum; if the check results are consistent, delete the temporary table to complete the insertion operation of the database table containing the watermark to the end of the target database.

[0101] Exemplarily, the verification module 608 is specifically used to: obtain all the data of the database table containing the watermark in the target database, and read all the data in a block reading manner, wherein the block reading manner refers to reading fixed rows of data each time; calculating a hash value based on the data read in blocks, wherein the hash value is obtained by the SHA-256 hash algorithm; connecting the hash values ​​corresponding to the data read in blocks to obtain a connected hash value; performing a hash calculation on the connected hash value to obtain a final hash value of the database table containing the watermark; obtaining a pre-stored original hash value from the hardware security module, wherein the hardware security module and the target database are communicated via the TLS protocol. The method comprises the following steps: the first step is to perform a communication between the two database tables; the final hash value of the database table containing the watermark and the original hash value are converted into a byte array, and the two are compared by a byte-by-byte XOR operation; if the results of the byte-by-byte XOR operation are all zero, the hash values ​​are determined to be consistent, and the comparison result is marked as verification passed; if the results of the byte-by-byte XOR operation are not all zero, the hash values ​​are determined to be inconsistent, and the comparison result is marked as verification failed; when the comparison result is marked as verification failed, a security monitoring process of the target database is triggered, and abnormal event information is recorded, wherein the abnormal event information includes: timestamp, table name, original hash value, current hash value; and an alarm containing the abnormal event information is sent to a designated receiving end through a preset alarm channel.

[0102] Each module in the tracing device of the above database table can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0103] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for tracing the source of a database table is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0104] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0106] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0107] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0109] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0111] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for tracing the origin of a database table, characterized in that: The method comprises: Monitor the operating status of the target database in real time and obtain monitoring data; When the monitoring data is abnormal, a restart process of the target database is triggered; After the target database is restarted, data is restored from the most recent database backup point to obtain a target database for restoring data; Extracting the watermark contained in the database table in the target database of the recovered data to obtain the watermark information; The copyright information and source identification of the database table in the target database are determined based on the watermark information, and the users and dissemination channels of the database table in the target database are determined in combination with the access records of the database table in the target database.

2. The method according to claim 1, characterized in that: Before real-time monitoring of the operating status of the target database to obtain monitoring data, the method further includes: Perform watermark embedding processing on the database table to obtain a database table containing the watermark; Insert the database table containing the watermark completely into the end of the target database; The integrity of the database table containing the watermark inserted into the end of the target database is verified.

3. The method according to claim 2, characterized in that The watermark embedding process is performed on the database table to obtain a database table containing the watermark, including: Obtaining a floating-point value field to be embedded with a watermark from a database table, and converting the floating-point value field into a frequency domain by Fourier transform to obtain a frequency domain coefficient matrix; The frequency domain coefficient matrix is ​​processed in blocks, and the coefficients in the intermediate frequency region are selected as watermark embedding positions; Encode the preset watermark information including copyright information and timestamp into a binary sequence; embedding the binary sequence into the selected intermediate frequency region coefficients, and modulating the embedded binary sequence using a quantization index; If the watermark embedding is completed, the frequency domain coefficient matrix containing the watermark information is converted back to the time domain through inverse Fourier transform to obtain a numerical sequence containing the watermark information; Rounding the numerical sequence containing the watermark information according to a preset requirement to obtain a processed numerical sequence containing the watermark information; The processed numerical sequence containing the watermark information is rewritten into the corresponding field of the original database table through an SQL update statement to obtain a database table containing the watermark.

4. The method according to claim 2, characterized in that: The step of completely inserting the database table containing the watermark into the tail of the target database includes: Read the data to be inserted from the database table containing the watermark, and obtain the target database table structure information through SQL statements; Creating a temporary table according to the target database table structure information, wherein the temporary table is used to store the insertion status, and the insertion status includes: insertion progress, amount of inserted data, and breakpoint position field; Constructing a batch insert statement, wherein the batch insert statement is generated according to the target database table structure information; Set the data volume threshold for each batch of inserts to a preset number of records, and execute the statement insert operation through the connection pool of the target database; Use a scheduled task to set the insertion operation to be performed at a preset interval and monitor the insertion process; If an insertion interruption is detected, the latest insertion status is read from the temporary table, the breakpoint position information is obtained, and after the insert statement is reconstructed, the insert operation is continued from the breakpoint; Calculate the checksum of the inserted statement, wherein the checksum is calculated using the MD5 hash algorithm; Comparing the data consistency between the source database table and the target database table according to the obtained checksum; If the verification results are consistent, the temporary table is deleted, and the insertion operation of the database table containing the watermark into the tail of the target database is completed.

5. The method according to claim 2, characterized in that: The integrity verification of the database table containing the watermark inserted into the tail of the target database includes: Acquire all data of the database table containing the watermark in the target database, and read all the data in a block reading manner, wherein the block reading manner refers to reading fixed rows of data each time; Calculate a hash value based on the data read in blocks, where the hash value is obtained using the SHA-256 hash algorithm; Connect the hash values ​​corresponding to the data read in blocks to obtain the connected hash value; Perform hash calculation on the connected hash value to obtain the final hash value of the database table containing the watermark; Obtaining a pre-stored original hash value from a hardware security module, wherein the hardware security module communicates with a target database via a TLS protocol; Convert the final hash value of the database table containing the watermark and the original hash value into byte arrays and compare them by byte-by-byte XOR operation; If the byte-by-byte XOR operation results are all zero, the hash values ​​are determined to be consistent, and the comparison result is marked as verified; If the result of the byte-by-byte XOR operation is not all zero, the hash values ​​are determined to be inconsistent and the comparison result is marked as verification failure; When the comparison result is marked as verification failure, trigger the security monitoring process of the target database and record abnormal event information, wherein the abnormal event information includes: timestamp, table name, original hash value, current hash value; An alarm containing the abnormal event information is sent to a designated receiving end through a preset alarm channel.

6. The method according to any one of claims 1 to 5, characterized in that: When the monitoring data is abnormal, triggering the restart process of the target database includes: Collecting the operating indicators of the target database, including: CPU usage, memory usage, number of connections, and query response time; Comparing the operating index of the target database with a preset threshold; If the operating index of any of the target databases exceeds the preset threshold within the preset time, it is determined that an abnormality exists; When it is determined that an abnormality exists, the fault handling process of the target database is triggered, and a restart instruction is sent to the target database management program; wherein the restart instruction is used to instruct the target database management program to perform the following operations: Close the current database instance; Clear the cache; Reload the configuration file and start the target database instance; Confirm that the target database instance has been successfully started and is responding to query requests normally.

7. The method according to any one of claims 1 to 5, characterized in that: After the target database is restarted, data is restored from the most recent database backup point to obtain a target database for restoring data, including: After the target database is restarted, the most recent full backup and the incremental backups after the full backup are obtained; The full backup and the incremental backup are applied to the target database in chronological order until the database table containing the watermark is restored to a state before the exception occurs.

8. The method according to any one of claims 1 to 5, characterized in that: Extract the watermark contained in the database table in the target database of the recovered data to obtain the watermark information, including: Acquire all data of a database table containing a watermark from a target database, and convert all the data into a frequency domain by Fourier transform to obtain a frequency domain coefficient matrix; The frequency domain coefficient matrix is ​​processed by using a quantization index modulation algorithm, and watermark bits are extracted according to quantization values ​​of adjacent coefficients; Converting the extracted watermark bits to obtain a binary sequence, and performing verification on the binary sequence; Decoding the binary sequence according to a predefined watermark information format to restore metadata including the copyright holder identifier and timestamp information; Parsing the metadata to extract the copyright holder identifier and timestamp information; If the copyright holder identifier exists in the copyright information database, the complete copyright ownership information corresponding to the copyright holder identifier is obtained.

9. The method according to any one of claims 1 to 5, characterized in that: Determining the copyright information and source identifier of the database table in the target database according to the watermark information, and determining the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database, including: Acquire access records of a database table containing a watermark from a target database, wherein the access records include: access time, access user, and access operation type; Generate an access sequence arranged in chronological order according to the access records, and mine access item sets whose occurrence frequency exceeds an upper threshold to obtain a target access item set; Generate association rules according to the target access item set, and calculate the confidence and lift of the association rules; If the confidence is greater than a preset confidence threshold, and the lift is greater than a preset lift threshold, then the association rule is determined to be a strong association rule; Determine the main user of the database table containing the watermark and the potential propagation path according to the strong association rule, wherein the main user refers to the user with the highest frequency of occurrence, and the user sequence in the target access item set is used to characterize the potential propagation path; Combine the copyright information and source identification of the database table containing the watermark to establish a graph database, wherein the nodes created in the graph database are used to represent users, and the edges created are used to represent potential dissemination paths; A correlation map is generated based on the main users, the potential dissemination channels, the copyright information, and the source identifier.

10. A database table tracing device, characterized in that: The device comprises: The monitoring module is used to monitor the operating status of the target database in real time and obtain monitoring data; A trigger module, used to trigger a restart process of the target database when there is an abnormality in the monitoring data; A recovery module, used to restore data from the most recent database backup point after the target database is restarted, to obtain a target database for restoring data; An extraction module is used to extract the watermark contained in the database table of the target database for recovering data to obtain the watermark information; The traceability module is used to determine the copyright information and source identification of the database table in the target database according to the watermark information, and determine the user and dissemination path of the database table in the target database in combination with the access record of the database table in the target database.

Images