Watermark embedding methods, watermark tracing methods and systems in cross-media transmission processes

Abstract

This application provides a watermark embedding method, a watermark tracing method, and a system for cross-media transmission, belonging to the field of data processing technology. The method includes: responding to a watermark task creation instruction, acquiring data source configuration information, target source configuration information, watermark information, and watermark scheme information, wherein the data source configuration information is used to characterize a first database or a first storage file, and the target source configuration information is used to characterize a second database or a second storage file; according to the data source configuration information, streaming multiple batches of raw data from the first database or the first storage file; according to the watermark information and the watermark scheme information, determining the data rows to be embedded in each batch of raw data, and performing watermark processing on the data rows to be embedded to generate watermarked data rows; according to the target source configuration information, writing each batch of raw data containing the watermarked data rows into the second database or the second storage file. This application enables seamless data tracing in cross-media transmission scenarios.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a watermark embedding method, watermark tracing method and system in cross-media transmission process. Background Technology

[0002] In the field of data security and traceability technology, as enterprises continue to improve their level of informatization, sensitive data flows more and more frequently between various information systems. In particular, in cross-media transmission scenarios between various heterogeneous data sources such as database to database, database to file, and file to database, the cross-media transmission process involves multiple media forms such as SQL statement transmission, API calls, and file export, which makes data traceability face complex technical challenges.

[0003] Currently, data breach tracing primarily relies on encrypting static database files or exported files. This can be achieved by applying digital watermarks to the exported files or directly watermarking table data at the database level. However, these solutions typically assume the data remains in the same medium—either always within the database or always existing as a file. In cross-media transmission scenarios (e.g., data transferred directly from a source database to another heterogeneous database via SQL statements or API interfaces), directly encrypting or watermarking the entire source database can lead to traditional watermarks being lost, exposed, or rendered unrecognizable when data flows through non-file transmission channels (such as SQL statement streams). Furthermore, developing customized watermarking schemes for different transmission media (such as file streams, API responses, and database connections) incurs significant development costs and system complexity.

[0004] Therefore, how to achieve seamless data traceability in cross-media transmission scenarios has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a watermark embedding method, a watermark tracing method, and a system for cross-media transmission to solve the above-mentioned problems.

[0006] To achieve the above objectives, in a first aspect, this application proposes a watermark embedding method during cross-media transmission, the method comprising:

[0007] In response to the watermark task creation instruction, data source configuration information, target source configuration information, watermark information, and watermark scheme information are obtained. The data source configuration information is used to represent a first database or a first storage file, and the target source configuration information is used to represent a second database or a second storage file that is different from the data source configuration information.

[0008] Based on the data source configuration information, multiple batches of raw data are streamed from the first database or the first storage file.

[0009] Based on the watermark information and the watermark scheme information, determine the data rows to be embedded in each batch of original data, and perform watermark processing on the data rows to be embedded to generate watermarked data rows;

[0010] Based on the target source configuration information, each batch of original data containing the watermark data row is written into the second database or the second storage file.

[0011] In some implementations, the watermarking scheme information includes watermark density and a mapping relationship between field types and embedding algorithms. The step of determining the data rows to be embedded in each batch of original data based on the watermark information and the watermarking scheme information, and performing watermarking processing on the data rows to be embedded to generate watermarked data rows, includes:

[0012] Based on the watermark density, determine the data rows to be embedded in each batch of original data;

[0013] Based on the field type of each field in each row of data to be embedded, the corresponding embedding algorithm is determined according to the mapping relationship;

[0014] According to the embedding algorithm, the watermark information is segmented and embedded into each field to generate a watermark data row.

[0015] In some implementations, determining the data rows to be embedded in each batch of original data based on the watermark density includes:

[0016] Based on the watermark density and the total number of rows in each batch of original data, calculate the number of rows to be embedded in each batch of original data;

[0017] Using a preset pseudo-random number generator and the batch number of each batch of original data, a set of random row indices equal to the number of rows to be embedded is generated for each batch of original data.

[0018] Iterate through each batch of original data and determine the data rows whose row index belongs to the random row index set as the data rows to be embedded.

[0019] In some implementations, before the step of segmenting the watermark information and embedding it into the respective fields according to the embedding algorithm to generate the watermark data row, the method further includes:

[0020] The watermark information is encrypted using a preset encryption algorithm to obtain encrypted watermark data;

[0021] The encrypted watermark data is subjected to BCH error correction encoding to generate redundant encoded data;

[0022] The redundant encoded data is mapped into a quaternary symbol sequence to obtain the encoded watermark information;

[0023] The step of segmenting the watermark information and embedding it into the respective fields according to the embedding algorithm to generate a watermark data row includes:

[0024] According to the embedding algorithm, the encoded watermark information is segmented and embedded into each field to generate a watermark data row.

[0025] In some implementations, the step of segmenting the watermark information and embedding it into the respective fields includes:

[0026] According to the sorting order of the field names of each field, the watermark information is divided into multiple segments equal to the number of fields, wherein each segment corresponds to one field;

[0027] Check each fragment and its corresponding field sequentially to see if they meet the preset embedding conditions;

[0028] Fragments that meet the preset embedding conditions are embedded into the corresponding fields;

[0029] Fragments that do not meet the preset embedding conditions are split and cyclically embedded into the corresponding fields.

[0030] In some implementations, the step of streaming multiple batches of raw data from the first database or the first storage file according to the data source configuration information includes:

[0031] Based on a preset number of threads, the data in the first database or the first storage file is divided into multiple logical shards;

[0032] Start parallel processing threads corresponding to the preset number of threads, where each thread is responsible for reading data from one logical slice;

[0033] Within each thread, raw data is retrieved batch by batch from the first database or the first storage file at a preset batch size.

[0034] In some implementations, when the second database represented by the target source configuration information and the first database represented by the data source configuration information are heterogeneous databases, before writing each batch of original data containing the watermark data rows into the second database or the second storage file according to the target source configuration information, the following steps are included:

[0035] Read the table structure information of the source table in the first database;

[0036] Based on the table structure information and the syntax rules of the second database, a table creation statement is dynamically generated, and the target table is created in the second database using the table creation statement.

[0037] The step of writing each batch of original data containing the watermark data row into the second database or the second storage file according to the target source configuration information includes:

[0038] Based on the target source configuration information, each batch of original data containing the watermarked data row is written into the target table.

[0039] In some implementations, the data source configuration information, target source configuration information, watermark information, and watermark scheme information are configured and stored through a management interface, which includes:

[0040] The data source management module is used to configure and manage data source configuration information and target source configuration information;

[0041] The watermark information management module is used to configure watermark information and encode and store the watermark information.

[0042] The watermarking scheme management module is used to configure the embedding algorithm, watermark density, and data rows to be embedded;

[0043] The task scheduling module is used to create watermarking tasks based on the configured data source configuration information, target source configuration information, watermark information, and watermarking scheme information.

[0044] Secondly, to achieve the above objectives, this application also proposes a watermarking traceability method during cross-media transmission, the method comprising:

[0045] In response to the source tracing task creation command, obtain the configuration information to be traced;

[0046] Based on the source-tracing configuration information, multiple batches of data to be detected are streamed from a third database or a third storage file.

[0047] Iterate through each field of each row in each batch of data to be detected, and call the corresponding extraction algorithm according to the field type to extract watermark fragments from the field values;

[0048] Watermark fragments extracted from multiple fields in the same row are concatenated according to the preset sorting rules of the field names to obtain watermark encoding information;

[0049] The watermark encoding information is decoded. If the decoding is successful and the confidence level exceeds a preset threshold, the decoding result is output as the original watermark information.

[0050] Thirdly, to achieve the above objectives, this application also proposes a watermark embedding and tracing system, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the watermark embedding method and the watermark tracing method in the cross-media transmission process as described above.

[0051] Compared with the prior art, the beneficial effects of this application include:

[0052] Firstly, compared to existing technologies that assume data always exists in the same medium, such as always within a database or always in file form, watermark information is easily lost or becomes unrecognizable due to media conversion when data flows through non-file channels such as SQL statements and API interfaces. This application addresses this by setting the watermark embedding time during the transmission process after the original data is read from the data source and before it is written to the target source. This allows the watermark information to accompany the data across different media formats, whether from database to database, from database to file, or from file to database, ensuring the watermark information remains stable. This fundamentally solves the technical problem of broken traceability chains in cross-media transmission scenarios, achieving continuous traceability of data along the complete transmission path.

[0053] Secondly, compared to existing technologies that directly encrypt or watermark the source database as a whole, making traditional watermark information often exposed or unrecognizable when data is transferred through non-file transmission channels, this application only performs watermarking processing during the process of reading the original data and preparing to write it to the target source. This avoids the business risks that may be caused by directly modifying the source data of the data source, and also eliminates the need to customize watermarking schemes for different transmission media, significantly reducing the implementation complexity of the system. At the same time, it ensures the transparency of the watermark embedding process to the source business system, enabling the technical solution of this application to be easily applied to various existing data transfer scenarios.

[0054] Thirdly, this application integrates multiple elements such as data source, target source, watermark information, and watermarking scheme through a configuration-driven approach, providing a flexible and controllable technical architecture for large-scale data processing. Furthermore, by using unified task creation instructions, it centrally manages various configuration information, enabling the system to flexibly adjust the watermarking scheme according to different business needs without requiring redevelopment for each transmission scenario. This configuration-based and parameterized design not only reduces system operation and maintenance costs but also improves the execution efficiency of watermarking tasks, making it possible to achieve unified watermark protection in large-scale, diverse data flow environments. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0056] Figure 1 This is a flowchart illustrating a watermark embedding method during cross-media transmission in one embodiment.

[0057] Figure 2 This is a detailed flowchart illustrating the process of streaming multiple batches of raw data from the first database or the first storage file according to the data source configuration information in one embodiment.

[0058] Figure 3 In one embodiment, a detailed flowchart is shown in which the data rows to be embedded in each batch of original data are determined based on the watermark information and the watermark scheme information, and the data rows to be embedded are watermarked to generate watermarked data rows.

[0059] Figure 4 This is a schematic diagram of the functional modules of the management interface in one embodiment;

[0060] Figure 5 This is a flowchart illustrating a watermarking traceability method during cross-media transmission in one embodiment.

[0061] Figure 6 This is a schematic diagram of the watermark embedding and watermark tracing system involved in the watermark embedding method and watermark tracing method during cross-media transmission in the embodiments of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0064] For example, the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For instance, without departing from the scope of this application, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. Both the first element and the second element are elements, but they are not the same element.

[0065] For example, the terms "comprising" or "including" used in this application indicate the presence of features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0066] As mentioned earlier, current methods for tracing the source of data breaches primarily involve encrypting static database files or exported files. This can be achieved by applying digital watermarks to the exported files or by directly watermarking table data at the database level. However, these solutions typically assume that the data remains in the same medium—either always within the database or always existing as a file. In cross-media transmission scenarios (e.g., data is transferred directly from a source database to another heterogeneous database via SQL statements or API interfaces), if the source database is directly encrypted or watermarked, traditional watermark information is often lost, exposed, or becomes unrecognizable due to media conversion when the data flows through non-file transmission channels (such as SQL statement streams). Furthermore, developing customized watermarking schemes for different transmission media (such as file streams, API responses, and database connections) would incur significant development costs and system complexity.

[0067] Therefore, how to achieve seamless data traceability in cross-media transmission scenarios has become an urgent technical problem to be solved. To this end, this application proposes a watermark embedding method, a watermark traceability method, and a system for cross-media transmission. By positioning the watermark embedding time during the cross-media data flow process, and combining it with streaming processing and configuration-driven techniques, the traceability problem in cross-media transmission scenarios is effectively solved, achieving zero-intrusion protection of the data source, while reducing the complexity of system implementation and operation and maintenance.

[0068] like Figure 1 As shown in the figure, this application provides a watermark embedding method during cross-media transmission, the method including the following steps:

[0069] Step S10: In response to the watermark task creation instruction, obtain the data source configuration information, target source configuration information, watermark information, and watermark scheme information.

[0070] In this embodiment, the watermark task creation instruction is a trigger signal, which can be triggered by a user clicking a button on the operation interface or by another system calling it through an application programming interface. It is used to inform the watermark embedding and tracing system in this embodiment to begin a watermarking operation process. The data source configuration information is a set of parameters used to locate and connect the original data location, representing the first database or the first storage file. For example, when the data source is the first database, the data source configuration information may include the internet protocol address or domain name of the first database, communication port number, database name, access username, access password, and the table name or query statement to be processed. When the data source is the first storage file, the data source configuration information may include the path of the storage file, file server address, access protocol type, and filename or file matching pattern. The target source configuration information is another set of parameters used to locate and connect the data writing target. Its structure is similar to the data source configuration information, but it represents a second database or a second storage file of a different type than the data source configuration information. Watermark information refers to the identifying content that needs to be embedded in the data for subsequent tracing. It can be a string in text form, such as the distributor's name, the name of the user unit, the distribution timestamp, or custom remarks. Watermarking scheme information is a set of strategic parameters that control how watermarks are embedded. It defines the specific rules for watermark embedding, such as which embedding algorithm to use and the sparsity of the watermark.

[0071] Specifically, the system responds to the watermark task creation instruction and, based on the identifier or reference carried in the instruction, reads the pre-configured data source configuration information, target source configuration information, watermark information, and watermark scheme information from persistent storage media such as relational databases or configuration files.

[0072] In some implementations, after step S10, a connection can be established with the first database or the first storage file, and a connection can be established with the second database or the second storage file, based on the data source configuration information and the target source configuration information. If the connection is successful, network connectivity and the read / write status of the target medium are verified based on the connection relationship, and step S20 is executed after successful verification. If the connection fails, an error message is returned and the watermarking task is terminated to avoid wasting resources by discovering connection problems in subsequent processing stages.

[0073] Step S20: Based on the data source configuration information, read multiple batches of raw data in a streaming manner from the first database or the first storage file.

[0074] In this embodiment, the original data refers to the source data stored in the first database or the first storage file that has not yet been watermarked by this system. It can be a record row in a database table or a structured data row in a file, such as one or more lines of JSON format text in a CSV file.

[0075] In some implementations, when the data source is a primary database, the system can establish a connection to the primary database via a database driver such as JDBC or ODBC and create a streaming query statement. For example, in MySQL, streaming read mode can be enabled by setting the `useCursorFetch=true` and `defaultFetchSize` parameters, so that the query results are not returned all at once, but in batches using a cursor. When the data source is a primary storage file, the system can open the primary storage file via a file input stream and read it line by line using a parser corresponding to the file format of the primary storage file. For example, a CSV parser is used for CSV files, and a JSON parser is used for JSON Lines format files, reading a specified number of lines to form a batch of raw data.

[0076] Through the streaming and batch processing in step S20, the system does not need to load the entire dataset into memory. Even when faced with database tables containing hundreds of millions of data points or files in the hundreds of gigabytes range, it can maintain stable memory usage and avoid the risk of memory overflow.

[0077] In some implementations, such as Figure 2 As shown, step S20 includes:

[0078] Step S21: Divide the data in the first database or the first storage file into multiple logical shards according to the preset number of threads.

[0079] In this embodiment, the preset number of threads is a user-configurable integer value used to specify the number of worker threads that the system concurrently starts when performing data reading tasks. For example, it can be set to 8, 16, or 32. In some implementations, the system can determine a recommended number of threads based on the size of the data source, the number of CPU cores in the system, the input / output capabilities, and the write capabilities of the target source, for user reference. Logical sharding is a logical division of all data in the data source. Each shard represents a continuous subset of the entire dataset. All shards do not overlap and, when merged, can cover the complete dataset. The purpose of sharding is to allocate an independent working unit to each thread, allowing multiple threads to process different parts of the data simultaneously without interfering with each other. When the data source is a first database, logical sharding can be based on the primary key range, unique index range, or physical storage blocks; when the data source is a first storage file, logical sharding can be based on the size of the first storage file, the range of rows, or the byte offset.

[0080] In some implementations, the system can determine the data source type based on the data source configuration information, determine the corresponding sharding strategy based on the data source type, and divide the data in the first database or the first storage file into multiple logical shards based on the corresponding sharding strategy and the preset number of threads.

[0081] Specifically, when the data source is the first database, the corresponding sharding strategy is determined as the database sharding strategy. Based on the database sharding strategy, the total number of records in the first database is obtained. The expected number of records for each logical shard is determined based on the total number of records and the preset number of threads. For example, if the total number of records is 10 million rows and the number of threads is 8, then the expected number of records for each shard is 1.25 million rows. Based on the expected number of records for each logical shard, the data in the first database is divided into multiple logical shards.

[0082] If the total number of records in the first database cannot be obtained, the system can also generate sharding boundaries based on the range of primary keys. For example, the first shard corresponds to rows with primary key values ​​from 1 to 1.25 million, the second shard corresponds to rows with primary key values ​​from 1.25 million and 1 to 2.5 million, and so on. Thus, the data in the first database is divided into multiple logical shards according to the sharding boundaries.

[0083] If the total number of records or the primary key of the first database cannot be obtained, the system can also divide the data in the first database into multiple logical shards based on ROWID or physical storage blocks. For example, in an Oracle database, sharding can be based on a range of ROWIDs.

[0084] When the data source is the first storage file, the system can obtain the total size of the first storage file, for example, 800 megabytes. Dividing the total size by the preset number of threads yields the estimated size of each logical partition, for example, approximately 100 megabytes per partition. Based on the estimated size of each logical partition, the system locates the start and end offsets of each logical partition within the first storage file and divides the data in the first storage file into multiple logical partitions. Since lines in a text file cannot be truncated across partitions, the system needs to find the starting position of the first complete line from the start offset of each partition as the actual starting point of that partition. Simultaneously, it determines the end point of the current partition from the end offset of the previous partition, ensuring that each partition contains complete line data. After partitioning, the system can generate descriptive information for each partition, including a partition identifier and data source location information such as primary key range or file offset range.

[0085] Step S22: Start the parallel processing thread corresponding to the preset number of threads.

[0086] In this embodiment, a parallel processing thread is the smallest instruction sequence unit that the operating system can independently schedule and execute. Each thread is designed to execute the same data reading logic and is responsible for reading the data of a logical slice. Multiple threads can run simultaneously on a multi-core processor, thereby achieving parallel processing.

[0087] Specifically, the system can iterate through all logical slices. For each logical slice, the system encapsulates a read task into an executable object, such as a class implementing the Runnable interface or a lambda expression, and then submits the read task to the thread pool for execution. Upon receiving the read task, the thread pool allocates an idle thread from the pool to execute the task, passing the logical slice description information as a parameter to the corresponding idle thread. Since the size of the thread pool is equal to the number of logical slices, typically a preset number of threads, all threads will be activated simultaneously, and each thread will independently begin processing the logical slice assigned to it.

[0088] By launching multiple threads to work in parallel, the overall throughput of data reading can theoretically be increased by several times. For example, with sufficient input / output bandwidth, 16 threads reading in parallel can reduce the data reading time to one-sixteenth of the original. Furthermore, since each thread is only responsible for its own slice, there is no need for communication and synchronization between threads, avoiding the performance overhead caused by lock contention and context switching, thus maximizing parallel efficiency.

[0089] Step S23: Within each thread, raw data is retrieved batch by batch from the first database or the first storage file at a preset batch size.

[0090] In this embodiment, the preset batch size is a user-configurable integer value used to specify the number of data rows each thread retrieves from the data source in a single read operation. Batch fetching refers to threads not attempting to retrieve data in batches. Figure 1 Instead of loading the entire data of the shard into memory at once, a loop structure is used to fetch a batch of data at a time, process the batch, and then fetch the next batch, repeating this process until all the data in the shard has been read.

[0091] Specifically, each thread first constructs a data query or read instruction for the assigned logical shard based on its description information. When the data source is the first database, the thread constructs an SQL query statement containing shard boundary conditions and sets the start and end values ​​of the shard as parameters in the query statement. To support streaming reads, the thread needs to set relevant parameters of the database driver; for example, in MySQL, the cursor type needs to be set to ResultSet.TYPE_FORWARD_ONLY, and the retrieval size needs to be set to a preset batch size. After executing the query, the thread obtains a result set object, which is maintained using a cursor and does not load all matching data into memory at once. Subsequently, the thread enters a loop, repeatedly calling the next method of the result set. Each call retrieves one row of data. When the cumulative number of retrieved rows reaches the preset batch size, the rows in this batch are assembled into a data set, such as a list object, and then this batch of data is sent to the processing queue, waiting to execute step S30. The thread continues to loop, fetching the next batch of data, until the `next` method of the result set returns false, indicating that all data within that slice has been read. When the data source is the first storage file, the thread opens the file input stream based on the slice's start and end offsets and positions itself at the beginning. The thread uses a buffered reader to read the file content line by line, counting each line read. When the number of lines read reaches the preset batch size, this batch of lines is assembled into a data set and sent to the processing queue. The thread continues reading until it reaches the slice's end offset or the end of the file.

[0092] By employing a streaming, batch-fetching approach within each thread, the system can process arbitrarily large data shards with constant memory consumption, avoiding memory pressure caused by excessive data volume. Simultaneously, batch fetching reduces the number of interactions with the data source; for example, fetching 1000 rows at a time instead of fetching row by row significantly reduces network and database call overhead, improving overall read performance. Ultimately, all threads read data in parallel, streaming, and in batches, enabling a continuous flow of data from the entire data source into the processing pipeline with extremely high throughput.

[0093] Step S30: Based on the watermark information and the watermark scheme information, determine the data rows to be embedded in each batch of original data, and perform watermark processing on the data rows to be embedded to generate watermarked data rows.

[0094] In this embodiment, the data rows to be embedded refer to those data rows in a batch of original data that require watermark information to be embedded. Data rows not selected remain unchanged without any modification. Watermarking processing refers to the process of embedding watermark information into data rows in an invisible or imperceptible manner. A watermarked data row refers to the data row generated after watermarking processing, which contains all the information of the original data as well as the additionally embedded watermark information.

[0095] In some implementations, the watermarking scheme information includes watermark density and the mapping relationship between field types and embedding algorithms, such as... Figure 3 As shown, step S30 includes:

[0096] Step S31: Determine the data rows to be embedded in each batch of original data based on the watermark density.

[0097] In this embodiment, the watermark density is a preset parameter used to control the proportion of data rows with embedded watermarks in the entire dataset.

[0098] In some implementations, the number of rows to be embedded in each batch of original data can be calculated based on the watermark density and the total number of rows in each batch of original data. For example, if a batch of original data has 1000 rows and the watermark density is 10%, then the number of rows to be embedded is 100. Using a preset pseudo-random number generator and the batch number of each batch of original data, a set of random row indices equal to the number of rows to be embedded is generated for each batch of original data. These indices point to specific rows in that batch of data. Each batch of original data is traversed, and the data rows whose row indices belong to the set of random row indices are determined as the data rows to be embedded.

[0099] Step S31 significantly reduces the data bloat caused by the introduction of watermarks through sparse embedding. In large-scale data scenarios, watermarking only a portion of the rows can greatly reduce storage and transmission overhead. On the other hand, random selection enhances the concealment and anti-detection capabilities of the watermark. Attackers cannot determine which rows contain the watermark through statistical analysis, thereby improving the robustness of the watermark.

[0100] Step S32: Determine the corresponding embedding algorithm according to the mapping relationship based on the field type of each field in each data row to be embedded.

[0101] In this embodiment, field type refers to the data type of each column in a database table or file structure. Common field types include character types such as varchar, text, and char; numeric types such as int, float, double, and decimal; timestamp types such as date, time, datetime, and timestamp; and other types such as binary and boolean. The mapping relationship is a predefined configuration that associates each field type with one or more watermark embedding algorithms. This mapping relationship can be specified by the user in the watermarking scheme or automatically generated by the system based on the characteristics of the field type and the applicability of the watermarking algorithm. Embedding algorithms refer to the specific technical methods used to embed watermark information into field values. For different field types, embedding algorithms need to ensure the concealment of the watermark and its non-impact on business data. For example, for character fields, a zero-width character insertion algorithm can be used to insert the encoded watermark information into the original data of the field in the form of a zero-width character; for numeric fields, a least significant bit replacement algorithm can be used to replace the value after a preset decimal place in the original data of the field with the encoded watermark information; for timestamp fields, a time precision adjustment algorithm can be used to embed the encoded watermark information by adjusting the millisecond position of the original data of the field.

[0102] Specifically, the system iterates through all fields in each row of data to be embedded that require watermark embedding. For each field, the system obtains the field type through database metadata or data source description information, such as obtaining the field type code through JDBC's ResultSetMetaData. Based on the field type, the system queries a pre-defined mapping table to determine the corresponding embedding algorithm. This mapping table is a collection of key-value pairs, where the key is the field type and the value is the identifier of the corresponding embedding algorithm.

[0103] Because the data source contains a variety of field types, a single algorithm cannot cover all scenarios. Step S32 establishes a mapping relationship, allowing the system to call different algorithms for different field types, thus achieving seamless embedding across various data types. Simultaneously, this configurable approach gives the system excellent scalability; when new field types or new embedding algorithms need to be supported, only the mapping relationship needs to be updated without modifying the core processing logic.

[0104] Step S33: According to the embedding algorithm, the watermark information is segmented and embedded into each field to generate a watermark data row.

[0105] In this embodiment, segmentation refers to dividing the complete watermark information into multiple fragments according to certain rules, with each fragment corresponding to a field to be embedded. Embedding refers to using an embedding algorithm to covertly integrate the watermark information fragments into the original value of the field, generating a new field value. The watermark data row refers to the new data row obtained after watermark embedding processing. This row contains all the information of the original data row as well as the additionally embedded watermark information, but it is almost identical to the original row in appearance and usage. The hidden information can only be recovered using a special algorithm when it is necessary to extract the watermark.

[0106] As a feasible implementation method for generating watermark data, the watermark information can be encrypted using a preset encryption algorithm to obtain encrypted watermark data. The preset encryption algorithm can be AES-256-GCM, AES-256-CCM, ChaCha20-Poly1305, etc. Taking AES-256-GCM as an example, AES represents Advanced Encryption Standard, 256 indicates the use of a 256-bit key for encryption, and GCM represents Galois Counter mode, a certified encryption mode that simultaneously provides data encryption and integrity authentication. Encrypted watermark data refers to the ciphertext data generated after the original watermark information has been encrypted using the preset encryption algorithm. The encrypted watermark data is then subjected to BCH error correction encoding to generate redundant encoded data. BCH error correction encoding is a cyclic redundancy coding technique capable of detecting and correcting multiple random errors; the redundant encoded data refers to the data sequence generated after the encrypted watermark data has undergone BCH error correction encoding, which includes the original encrypted watermark information and additional error correction verification information, with a length greater than the original encrypted watermark data; the excess portion is the error correction redundancy bits. Furthermore, the parameters of the error correction coding can be dynamically adjusted according to the characteristics of the watermark embedding carrier. If the data is prone to many errors during the transfer process, a stronger error correction capability can be selected, i.e., more error symbols that the coding can correct, but this will increase the length of redundant data. If the data is sensitive to expansion, a weaker error correction capability can be selected to save space. The redundant coded data is mapped into a quaternary symbol sequence to obtain the encoded watermark information. Here, the redundant coded data is a binary bit sequence. The quaternary symbol sequence refers to mapping binary data into groups of two bits, converting it into a sequence of four symbols: 0, 1, 2, and 3. Each quaternary symbol can correspond to a different physical embedding carrier. For example, for character fields, 0, 1, 2, and 3 can be mapped to four different zero-width characters: U+200B, U+200C, U+200D, and U+FEFF, respectively; for numeric fields, they can be mapped to different numeric bit patterns. The encoded watermark information refers to the final watermark data obtained after the three steps of encryption, error correction coding, and symbol mapping, and this data will exist in the form of a quaternary symbol sequence. According to the embedding algorithm, the encoded watermark information is segmented and embedded into each field to generate a watermark data row.

[0107] This implementation method ensures the confidentiality of the watermark information through encryption, guarantees the fault tolerance of the watermark through error-correcting coding, and ensures the efficiency and concealment of the embedding through symbol mapping. The combination of these three preprocessing steps enables the finally embedded watermark to resist various attacks and accidental damage, and the original watermark information can still be reliably extracted after data leakage, providing solid technical support for traceability.

[0108] As a feasible implementation method for segmenting and embedding watermark information into the various fields, the fields are sorted according to their field names. This sorting order refers to the order obtained by arranging all fields in the data row to be embedded according to a specific sorting rule. The watermark information is segmented into multiple fragments equal to the number of fields, with each fragment corresponding to one field. If the length of the watermark information is not divisible by the number of fields, it can be handled by padding at the end or circular allocation. Each fragment and its corresponding field are sequentially checked to see if they meet preset embedding conditions. Preset embedding conditions are a set of predefined rules used to evaluate whether a field has the ability to embed the corresponding watermark fragment. Embedding conditions typically include whether the field's storage capacity is sufficient to hold the watermark fragment. For example, for character fields, it is necessary to check whether the maximum length of the field allows the insertion of a specified number of zero-width characters on top of the original string; for numeric fields, it is necessary to check whether the precision of the decimal part of the value is sufficient to accommodate a specified number of binary bits; for timestamp fields, it is necessary to check whether the millisecond part of the timestamp has enough low-order bits for embedding. Embedding conditions may also include whether the field value is empty, whether the field format meets the requirements of the embedding algorithm, etc. Fragments that meet the preset embedding conditions are embedded into the corresponding fields. Fragments that do not meet the preset embedding conditions are split and cyclically embedded into the corresponding fields. Splitting refers to further dividing a complete watermark fragment into smaller sub-fragments, each sub-fragment being shorter than the original fragment, ensuring that a single sub-fragment can be accommodated by the remaining capacity of the field. Cyclic embedding refers to embedding multiple sub-fragments obtained after splitting into the same field sequentially in a certain order. The embedding method can be sequential or a cyclic overwrite strategy, for example, treating the available embedding positions of the field as a circular buffer and repeatedly writing until all sub-fragments are embedded.

[0109] This implementation method ensures the reproducibility of the mapping relationship through a sorting rule based on field names. During watermark extraction, as long as the same set of fields is obtained and sorted according to the same rules, the extracted fragments from each field can be correctly pieced together and restored in sequence, guaranteeing the accuracy of watermark extraction. Furthermore, through splitting and cyclic embedding, the system can compress long watermark fragments into a limited field space, ensuring the integrity and extractability of the watermark. This fault tolerance makes this technical solution applicable to various complex real-world scenarios. Even if some fields are short or have low precision, the watermark can still be successfully embedded, greatly improving the universality and robustness of the solution.

[0110] Step S30, by segmenting the watermark information and distributing it across multiple fields, ensures that even if data in one field is truncated or cleaned during subsequent processing, the watermark information can still be recovered from other fields, enhancing the watermark's robustness. Furthermore, employing different embedding algorithms for different field types ensures the watermark's concealment, preventing data values ​​from exceeding normal business scope or causing business logic anomalies due to embedding operations. In addition, the standardized segmentation and embedding process guarantees the reversibility of watermark extraction; as long as the extraction and splicing follow the same rules, the original watermark information can be accurately restored.

[0111] Step S40: According to the target source configuration information, write each batch of original data containing the watermark data row into the second database or the second storage file.

[0112] In some implementations, when the target source is a second database, the system can establish a connection with the second database through a database driver such as JDBC, and generate corresponding insert or update statements for each batch of original data containing the watermarked data rows. All insert or update statements are then combined into a batch operation, for example, using the JDBC addBatch and executeBatch methods, to send all batches of original data containing the watermarked data rows to the second database for execution at once. When the target source is a second storage file, the system can open or create the target file through a file output stream, format each batch of original data containing the watermarked data rows according to the target file's format requirements, such as concatenating them into CSV format rows, and then write the formatted content to the file output stream all at once, instead of writing to disk for each line processed. After writing is complete, the system can perform a flush operation to ensure the data is persisted to disk.

[0113] In some implementations, when the second database represented by the target source configuration information and the first database represented by the data source configuration information are heterogeneous databases, a dynamic creation step of the target table can be performed before writing. Specifically, the system can read the table structure information of the source table in the first database, which may include field names, field types, field lengths, constraints, etc.; based on the table structure information and the syntax rules of the second database, a table creation statement is dynamically generated, and the target table is created in the second database using the table creation statement. Thus, according to the target source configuration information, each batch of original data containing the watermarked data rows is written to the target table.

[0114] In the watermark embedding method for cross-media transmission proposed in this application, firstly, compared to existing technologies that assume data is always in the same medium form, such as always within a database or always existing in file form, watermark information is easily lost or becomes unrecognizable due to medium conversion when data flows through non-file channels such as SQL statements and API interfaces, this application sets the watermark embedding time during the transmission process after the original data is read from the data source and before it is written to the target source. This allows the watermark information to accompany the data across different medium forms, whether from database to database, from database to file, or from file to database, the watermark information can be stably preserved. This fundamentally solves the technical problem of broken traceability chains in cross-media transmission scenarios and realizes continuous traceability of data along the complete transmission path.

[0115] Secondly, compared to existing technologies that directly encrypt or watermark the source database as a whole, making traditional watermark information often exposed or unrecognizable when data is transferred through non-file transmission channels, this application only performs watermarking processing during the process of reading the original data and preparing to write it to the target source. This avoids the business risks that may be caused by directly modifying the source data of the data source, and also eliminates the need to customize watermarking schemes for different transmission media, significantly reducing the implementation complexity of the system. At the same time, it ensures the transparency of the watermark embedding process to the source business system, enabling the technical solution of this application to be easily applied to various existing data transfer scenarios.

[0116] Thirdly, this application integrates multiple elements such as data source, target source, watermark information, and watermarking scheme through a configuration-driven approach, providing a flexible and controllable technical architecture for large-scale data processing. Furthermore, by using unified task creation instructions, it centrally manages various configuration information, enabling the system to flexibly adjust the watermarking scheme according to different business needs without requiring redevelopment for each transmission scenario. This configuration-based and parameterized design not only reduces system operation and maintenance costs but also improves the execution efficiency of watermarking tasks, making it possible to achieve unified watermark protection in large-scale, diverse data flow environments.

[0117] In one embodiment, the data source configuration information, target source configuration information, watermark information, and watermark scheme information used in the above embodiments are configured and stored through a management interface, such as... Figure 4 As shown, the management interface 1 includes: a data source management module 11, a watermark information management module 12, a watermark scheme management module 13, and a task scheduling module 14.

[0118] In this embodiment, the data source management module 11 is the core component of the system access layer. It provides a unified entry point for configuring and managing all data sources that need to participate in watermarking and the location information and connection parameters of the target source to which the data is ultimately written.

[0119] In practical implementation, the data source management module 11 is presented to the user in the form of a graphical user interface, which may include a data source list display area, a form for adding a data source, a connection test button, and data source editing and deletion functions. When a user needs to add a data source, the system will display a form requiring the user to select the data source type, such as MySQL database, Oracle database, local file, FTP file, etc. Depending on the selected data source type, the form will dynamically display different input fields. For MySQL databases, users need to enter the connection name, host address, port number (default 3306), database name, username, password, and can also enter connection parameters such as character set and timeout. For local files, users need to enter the file path, file format (such as CSV, JSON), file encoding, etc. After the user completes the form, they can click the "Test Connection" button. The system will then attempt to establish an actual connection with the data source using the entered parameters. If the connection is successful, the system will notify the user; otherwise, it will display an error message. After the user confirms that the parameters are correct, they can save the information. The system will persistently store this configuration information in the system's configuration database and assign a unique identifier to the data source. During storage, sensitive information such as database passwords will be encrypted to ensure that even if the configuration database is leaked, the plaintext password will not be directly exposed. The data source management module also provides a data source list display function, allowing users to view all configured data sources and their status, such as whether they are online and the time of the last connection test.

[0120] The watermark information management module 12 is the center for defining and managing watermark content. It allows users to flexibly define watermark information that needs to be embedded in data and encode and store the watermark information.

[0121] In practical implementation, the watermark information management module 12 is also presented in the form of a graphical user interface, including functions such as a watermark information list, a form for adding watermark information, and a watermark preview. When a user needs to add a watermark, the system will display a form with multiple input fields. These input fields can be fixed, such as distributor, distribution time, user unit, and remarks, or they can be user-defined key-value pairs. After the user completes the form, the system will perform preliminary validation on this information, such as checking whether required fields are filled in and whether the date format is correct. After the validation is passed, the system enters the watermark information encoding process. The system first combines the multiple fields input by the user into a structured data object, such as a JSON format string. Then, the string is encoded, including encryption using the AES-256-GCM algorithm, error correction encoding using the BCH algorithm, and finally mapping it into a quaternary symbol sequence. After encoding, the system stores the original plaintext watermark information, the encoded watermark information, the key identifier used for encoding, the initialization vector, and other metadata together in the watermark information table. During storage, the original plaintext information is used to display to the user interface, while the encoded information is used for the actual watermark embedding. The system also assigns a unique identifier to each watermark for easy reference when creating watermarking tasks.

[0122] In some implementations, the watermark information management module 12 can support the dynamic generation of watermark information. For example, a user can define a watermark template, which includes placeholders such as the current timestamp, task name, and random number. When a watermark task is created, the system dynamically generates specific watermark information based on the template, ensuring that the watermark information for each task is unique.

[0123] The watermark scheme management module 13 is used to configure the embedding algorithm, watermark density, and data rows to be embedded. In practice, the watermark scheme management module 13 is presented in the form of a graphical user interface, including functions such as a watermark scheme list, a form for adding a new scheme, and field mapping configuration. When a user needs to add a watermark scheme, they first need to fill in the basic information of the scheme, such as the scheme name and description. Then, the user needs to configure the watermark density, which can be selected as a percentage, such as 10%, 20%, or 5%, through a slider or input box. Next, the user needs to configure the mapping relationship between field types and embedding algorithms. This mapping can be global, for example, uniformly setting all character fields to use the zero-width character algorithm, all numeric fields to use the least significant bit algorithm, and all timestamp fields to use the millisecond precision adjustment algorithm. For scenarios requiring fine-grained control, the user can specify the embedding algorithm for each field of a specific data source table. The system will read the metadata of the data source table selected by the user, displaying all fields of the table and their types. The user can select the embedding algorithm to use after each field, or choose not to embed that field. After configuration, the system serializes and stores information such as watermark density, algorithm mapping relationship, and field selection list into the watermark scheme table, assigning a unique identifier to each scheme. During storage, the algorithm mapping relationship can be saved in JSON format, for example, {"varchar":"zero_width","decimal":"lsb","timestamp":"time_precision"}.

[0124] In some implementations, the watermark scheme management module 13 supports conditional watermark embedding, allowing users to define rules such as embedding a watermark only when a field value meets specific conditions, for example, embedding a watermark only for rows with amounts greater than 10,000. It can also support multi-level watermark densities, such as using different densities for fields of different importance, with high density for critical fields and low density for ordinary fields.

[0125] The task scheduling module 14 is used to create watermarking tasks based on the configured data source configuration information, target source configuration information, watermark information, and watermarking scheme information. When the user clicks "Start" through the task scheduling module 14, the task scheduling module 14 loads the watermarking task instance from the task table, parses the referenced configuration items, obtains complete configuration information, and then starts the watermarking processing flow according to the steps in the above method embodiment.

[0126] The watermark embedding method proposed in this application for cross-media transmission organically integrates four core elements—data source, watermark information, watermark scheme, and task scheduling—by constructing a unified management interface and a modular functional architecture. This forms a complete watermark management interface that is flexible in configuration, standardized in management, and controllable in execution. It not only significantly reduces the configuration complexity and operation and maintenance costs of cross-media watermarking tasks, but also achieves standardization, automation, and visualization of the watermark processing flow through the collaborative work between modules. This provides a practical solution for the large-scale application of watermark traceability technology in complex cross-media data flow environments.

[0127] In one embodiment, such as Figure 5 As shown, a watermark tracing method is also provided during cross-media transmission, the method comprising the following steps:

[0128] Step S50: In response to the source tracing task creation instruction, obtain the source tracing configuration information.

[0129] In this embodiment, the source tracing task creation instruction is a trigger signal. It can be triggered by a user clicking the source tracing button on the operation interface, or by other systems calling it through an application programming interface. It is used to inform the system to start a watermark source tracing operation. The source tracing configuration information is a set of parameters used to locate and connect to the source to be detected. The source to be detected can be a third-party database or a third-party storage file. When the source to be detected is a third-party database, this information can include the database's Internet Protocol address or domain name, communication port number, database name, access username, access password, and the table name or query statement to be detected. When the source to be detected is a third-party storage file, this information can include the storage file path, file server address, access protocol type, and file name or file matching pattern. The source tracing configuration information may also include other parameters related to the source tracing process, such as the expected number of threads, batch size, decoding key identifier, confidence threshold, etc.

[0130] Step S60: Based on the source-tracing configuration information, read multiple batches of data to be detected from the third database or the third storage file in a streaming manner.

[0131] In this embodiment, the third database or third storage file refers to the location of the data to be detected that may contain watermark information. It can be a database or file different from the source end, or it can be the original target end. Its type can be a relational database, a non-relational database, or various file storage.

[0132] In some implementations, the system first establishes a connection with a third database or third storage file based on the configuration information to be traced. When the configuration information points to a third database, the system establishes a connection through a database driver such as JDBC or ODBC, and creates a streaming query statement. By setting appropriate acquisition size parameters, the streaming reading mode is enabled so that the query results are not returned to the client all at once, but in batches using a cursor. The system then executes the query and retrieves data rows from the result set in batches using a loop structure. Each batch of data is sent to a processing queue in memory, awaiting processing in step S70. When the configuration information points to a third storage file type, the system opens the third storage file through a file input stream and reads it line by line using the corresponding parser according to the file format of the third storage file. Each batch of data read constitutes a specified number of lines to be detected.

[0133] Through the streaming and batch processing in step S60, the system avoids loading the entire dataset into memory. Even when dealing with database tables containing hundreds of millions of data points or files ranging from hundreds of gigabytes, it maintains stable memory usage, avoiding the risk of memory overflow. Simultaneously, batch reading allows the reading operation to form a pipeline with subsequent extraction operations. After reading the first batch of data, extraction can begin, and the next batch of data can be read simultaneously. This overlapping execution method significantly improves the overall task execution efficiency.

[0134] Step S70: Iterate through each field of each row in each batch of data to be detected, and call the corresponding extraction algorithm according to the field type to extract watermark fragments from the field values.

[0135] In this embodiment, the extraction algorithm is the reverse operation of the embedding algorithm, used to recover the previously embedded watermark fragment from the field value. For example, for a zero-width character extraction algorithm for a character field, it can identify and extract the hidden zero-width character sequence; for a least significant bit extraction algorithm for a numeric field, it can read binary bits from a specified position in the decimal part; for a time precision extraction algorithm for a timestamp field, it can read binary bits from the least significant bit of the millisecond. A watermark fragment refers to a portion of the watermark data extracted from a single field, which may be a quaternary symbol sequence or a binary bit sequence.

[0136] Step S80: The watermark fragments extracted from multiple fields in the same row are concatenated according to the preset sorting rules of the field names to obtain the watermark encoding information.

[0137] In this embodiment, the preset sorting rule for field names refers to the rule that is exactly the same as the sorting rule used during embedding. The watermark encoding information refers to the complete symbol sequence obtained by splicing the various fragments according to the field order. This sequence is the watermark information that has been encrypted, error-correcting encoded, and symbol mapped before embedding, and it usually exists in the form of a quaternary symbol sequence or a binary sequence.

[0138] In some implementations, the system can obtain a list of field names for all fields in each row and sort these field names using a sorting function to obtain an ordered list of field names. The sorting rules are exactly the same as those used during embedding, such as lexicographical ascending order. Following the ordered list, the watermark fragments corresponding to each field are extracted sequentially. If a field does not yield a valid fragment during extraction, it can be considered an empty fragment or specially marked. All extracted watermark fragments are then concatenated sequentially to form a new sequence. For example, if the ordered field list is amount, create_time, and remark, and the corresponding extracted fragments are fragment A, fragment B, and fragment C, then the concatenated watermark encoding information is fragment A followed by fragment B followed by fragment C. After concatenation, the system obtains a complete symbol sequence, which is the watermark encoding information.

[0139] Step S90: Decode the watermark encoding information. If the decoding is successful and the confidence level exceeds a preset threshold, output the decoding result as the original watermark information.

[0140] In this embodiment, decoding refers to the complete reverse sequence of operations performed on the watermark encoded information during the encoding process, typically including the inverse mapping from quaternary symbols to binary, error correction decoding, and decryption. Confidence level is a quantitative indicator used to evaluate the reliability of the decoded watermark information, usually expressed as a percentage or score. Its calculation can be based on the number of errors corrected during error correction decoding, the integrity check results of decryption verification, and the structural characteristics of the watermark information. The preset threshold is a user-configurable value, such as 90% or 0.9. The system considers decoding successful and accepts the watermark information only when the calculated confidence level is greater than or equal to this threshold. The original watermark information refers to the final human-readable traceability content output, such as the distributor's name, the user's organization, and the distribution time. This information can be directly displayed to the user to locate the source of the leak.

[0141] In some implementations, the system can reverse-map the watermark encoding information according to the mapping rules at the time of embedding, converting each quaternary symbol back into two binary bits to obtain a binary sequence. The BCH error correction decoding algorithm is then invoked to decode the binary sequence. The BCH decoder attempts to detect and correct any erroneous bits in the sequence. If the number of errors is within the error correction capability of the encoding, the decoder outputs the corrected binary sequence and provides an indicator such as the number of corrected errors for subsequent confidence calculation. If the number of errors exceeds the error correction capability, the decoder may report decoding failure. Upon successful decoding, the system obtains the binary sequence of the encrypted watermark data. The encrypted watermark data is then decrypted using the same key and initialization vector as at the time of embedding, by invoking the decryption algorithm corresponding to the preset encryption algorithm. During decryption, the decryption algorithm simultaneously verifies the integrity of the data. If the ciphertext has been tampered with, the authentication tag verification will fail, and the decryption process will throw an exception or return an error. If decryption is successful, the system obtains the original plaintext watermark information, such as a JSON string. At this point, the system calculates a confidence score based on the number of errors corrected during decoding, the decryption verification result, and the reasonableness of the watermark information itself. For example, a base confidence level can be set to 100%, with a certain percentage deducted for each error corrected. If the authentication tag fails verification, the confidence level is reset to zero. If the calculated confidence level exceeds a preset threshold, such as 90%, the system determines that the plaintext watermark information is valid and outputs it as the original watermark information. The output method can be displayed on the interface, stored in the database for later query, or trigger an alarm to notify the administrator.

[0142] The watermark tracing method proposed in this application for cross-media transmission constructs a complete technical architecture that can directly extract watermarks from data in databases or files after cross-media transmission by directly accessing the source to be detected, accurately extracting data with field type awareness, robustly concatenating data based on field name sorting, fault-tolerant recovery of error correction decoding, and efficient parallel streaming processing. This not only solves the efficiency problem of existing technologies that require file export for tracing, but also ensures the recoverability of watermarks in complex transmission environments through robust design. It provides reliable technical support for rapid tracing and responsibility determination of data leakage incidents, and has significant technical progress compared to existing technologies.

[0143] In one embodiment, a watermark embedding traceability system is also provided, including one or more processors; and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the steps in the above method embodiments.

[0144] In one embodiment, such as Figure 6The diagram illustrates a schematic of a watermark embedding traceability system for implementing embodiments of this application. The watermark embedding traceability system includes a central processing unit (CPU) 101, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 102 or a program loaded from a storage portion 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the watermark embedding traceability system 100. The CPU 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0145] The following components are connected to I / O interface 105: an input section 106 including a keyboard, mouse, etc.; an output section 107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN card, modem, etc. The communication section 109 performs communication processing via a network such as the Internet. A drive 110 is also connected to I / O interface 105 as needed. A removable medium 111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 110 as needed so that computer programs read from it can be installed into storage section 108 as needed.

[0146] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer-readable medium carrying instructions that, in such embodiments, can be downloaded and installed from a network via communication section 109, and / or installed from removable medium 111. When the instructions are executed by central processing unit (CPU) 101, the various method steps described in this application are performed.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0148] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, any of the embodiments or implementations claimed above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A watermark embedding method during cross-media transmission, characterized in that, The method includes: In response to the watermarking task creation instruction, data source configuration information, target source configuration information, watermark information, and watermarking scheme information are obtained. The data source configuration information is used to represent a first database or a first storage file, the target source configuration information is used to represent a second database or a second storage file that is different from the data source configuration information, and the watermarking scheme information includes watermark density and the mapping relationship between field type and embedding algorithm. Based on the data source configuration information, multiple batches of raw data are streamed from the first database or the first storage file. The watermark information is encrypted using a preset encryption algorithm to obtain encrypted watermark data. The encrypted watermark data is then subjected to BCH error correction encoding to generate redundant encoded data. The redundant encoded data is then mapped into a quaternary symbol sequence to obtain the encoded watermark information. Based on the watermark density, determine the data rows to be embedded in each batch of original data. Based on the field type of each field in each data row to be embedded, determine the corresponding embedding algorithm according to the mapping relationship. Based on the embedding algorithm, segment the encoded watermark information and embed it into each field to generate watermark data rows. Based on the target source configuration information, each batch of original data containing the watermark data row is written into the second database or the second storage file.

2. The watermark embedding method in cross-media transmission according to claim 1, characterized in that, The step of determining the data rows to be embedded in each batch of original data based on the watermark density includes: Based on the watermark density and the total number of rows in each batch of original data, calculate the number of rows to be embedded in each batch of original data; Using a preset pseudo-random number generator and the batch number of each batch of original data, a set of random row indices equal to the number of rows to be embedded is generated for each batch of original data. Iterate through each batch of original data and determine the data rows whose row index belongs to the random row index set as the data rows to be embedded.

3. The watermark embedding method in cross-media transmission according to claim 1, characterized in that, The step of segmenting and embedding the encoded watermark information into the various fields includes: According to the sorting order of the field names of each field, the encoded watermark information is divided into multiple segments equal to the number of fields, wherein each segment corresponds to one field; Check each fragment and its corresponding field sequentially to see if they meet the preset embedding conditions; Fragments that meet the preset embedding conditions are embedded into the corresponding fields; Fragments that do not meet the preset embedding conditions are split and cyclically embedded into the corresponding fields.

4. The watermark embedding method in cross-media transmission according to claim 1, characterized in that, The step of streaming multiple batches of raw data from the first database or the first storage file according to the data source configuration information includes: Based on a preset number of threads, the data in the first database or the first storage file is divided into multiple logical shards; Start parallel processing threads corresponding to the preset number of threads, where each thread is responsible for reading data from one logical slice; Within each thread, raw data is retrieved batch by batch from the first database or the first storage file at a preset batch size.

5. The watermark embedding method during cross-media transmission according to claim 1, characterized in that, When the second database represented by the target source configuration information and the first database represented by the data source configuration information are heterogeneous databases, before writing each batch of original data containing the watermark data row into the second database or the second storage file according to the target source configuration information, the process includes: Read the table structure information of the source table in the first database; Based on the table structure information and the syntax rules of the second database, a table creation statement is dynamically generated, and the target table is created in the second database using the table creation statement. The step of writing each batch of original data containing the watermark data row into the second database or the second storage file according to the target source configuration information includes: Based on the target source configuration information, each batch of original data containing the watermarked data row is written into the target table.

6. The watermark embedding method in a cross-media transmission process according to any one of claims 1 to 5, characterized in that, The data source configuration information, target source configuration information, watermark information, and watermark scheme information are configured and stored through a management interface, which includes: The data source management module is used to configure and manage data source configuration information and target source configuration information; The watermark information management module is used to configure watermark information and encode and store the watermark information. The watermarking scheme management module is used to configure the embedding algorithm, watermark density, and data rows to be embedded; The task scheduling module is used to create watermarking tasks based on the configured data source configuration information, target source configuration information, watermark information, and watermarking scheme information.

7. A method for tracing watermarks during cross-media transmission, used to trace the watermark embedded by the watermark embedding method according to any one of claims 1 to 5 during cross-media transmission, characterized in that, The method includes: In response to the source tracing task creation command, obtain the configuration information to be traced; Based on the source-tracing configuration information, multiple batches of data to be detected are streamed from a third database or a third storage file. Iterate through each field of each row in each batch of data to be detected, and call the corresponding extraction algorithm according to the field type to extract watermark fragments from the field values; Watermark fragments extracted from multiple fields in the same row are concatenated according to the preset sorting rules of the field names to obtain watermark encoding information. The watermark encoding information is decoded. If the decoding is successful and the confidence level exceeds a preset threshold, the decoding result is output as the original watermark information.

8. A watermark-embedded traceability system, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the watermark embedding method in the cross-media transmission process as described in any one of claims 1 to 6 or the watermark tracing method in the cross-media transmission process as described in claim 7.

Images