A method for adding and extracting invisible watermark based on large model self-adaption

Abstract

The application discloses an adding and extracting method of invisible watermark based on a large model self-adaption, and belongs to the technical field of computer network security, and comprises the following steps: S1, metadata is acquired and metadata strings are spliced in sequence; S2, the hash code of the spliced metadata string is calculated and binary conversion is performed; S3, according to the mapping rule of zero-width characters and binary, each binary bit is mapped to the corresponding zero-width character; S4, the mapped zero-width characters are divided into four segments, the insertion position is determined according to the priority rule, and then the zero-width character segments are embedded into the corresponding position according to the embedding logic of different data formats. The application can realize the copyright protection and leakage tracing of API gateway response data without affecting the normal analysis and display of data.

Description

Technical Field

[0001] This invention relates to the technical field of computer network security, and in particular to a method for adding and extracting invisible watermarks based on large-model adaptive methods. Background Technology

[0002] As a key component in a microservice architecture, the API gateway is responsible for handling client requests and forwarding them to backend services. Adding watermarks to API gateway response data has several important functions, including: 1. Clarifying data origin: When data flows between different systems, platforms, or users, the watermark serves as a unique identifier, clearly identifying the specific API gateway and corresponding service or interface from which the data originated; 2. Tracking data propagation paths: The information in the watermark allows tracking the data's propagation path and access records within the network. If data issues arise or are misused, the watermark can be used to trace the data's flow, determining at which stages the data was accessed and where it was transmitted, facilitating rapid problem identification and accountability.

[0003] Adding watermarks to response data is an effective means of copyright protection and tracing leaks. However, existing methods often rely on encryption or signature technologies, which typically require complex mathematical calculations. Especially with large volumes of response data, these methods consume significant computing resources and time, impacting API gateway performance and response speed. Under high concurrency, this can lead to system delays or even lag, affecting user experience. Furthermore, encrypted or signed watermarks may alter certain characteristics of the data, making them somewhat detectable. Moreover, some encryption methods can significantly affect the original data structure, reducing the watermark's concealment and failing to meet the needs of scenarios requiring high data concealment.

[0004] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method for adding and extracting invisible watermarks based on large model adaptation, which can achieve copyright protection and leakage tracing of API gateway response data without affecting the normal parsing and display of data.

[0006] To achieve the above objectives, this invention provides a method for adding invisible watermarks based on large model adaptation, comprising the following steps:

[0007] S1: Retrieve metadata and concatenate the metadata strings in order;

[0008] S2: Calculate the hash code of the concatenated metadata string and perform binary conversion;

[0009] S3: According to the mapping rules between zero-width characters and binary, map each binary bit to the corresponding zero-width character;

[0010] S4: Divide the mapped zero-width character into 4 segments, determine the insertion position according to the priority rule, and then embed the zero-width character segments into the corresponding positions according to the embedding logic of different data formats.

[0011] In one embodiment of the present invention, in step S1, the collected metadata includes static metadata and dynamic metadata. The static metadata includes user ID and device fingerprint, and the dynamic metadata includes request timestamp. The metadata string is concatenated in the order of User-Agent hash values ​​of user ID, request timestamp, session ID, and device fingerprint.

[0012] In one embodiment of the present invention, in step S2, the metadata string is hashed using the SHA-256 algorithm to generate a 64-bit hexadecimal string. Then, each hexadecimal character is converted to binary, ultimately generating a 256-bit binary sequence B = {b1, b2, ..., b...}. i ,...,b 256}

[0013] In one embodiment of the present invention, in step S3, the mapping rule between zero-width characters and binary is as follows:

[0014]

[0015] In one embodiment of the present invention, step S4, which divides the mapped zero-width character into four segments, specifically includes: dividing the 256-bit binary watermark sequence B into four segments, embedding each segment three times during embedding; and using a majority voting mechanism, taking the value that appears most frequently among the three embedded watermarks as the final embedded watermark.

[0016] In one embodiment of the present invention, step S4, determining the insertion position according to the priority rule, specifically includes:

[0017] In the response data, for all string field values, the watermark is inserted at the end of the string;

[0018] When it is not possible to insert at the end of a string field value, choose to insert the watermark at the suffix of the field name;

[0019] If neither of the first two methods applies, then insert a zero-width character watermark between array elements.

[0020] In one embodiment of the present invention, step S4 mainly involves embedding data in JSON data and embedding data in XML data, depending on the data format:

[0021] In JSON data, watermarks can be embedded in two ways: by ending the field value and by suffixing the field name.

[0022] For XML data, choose to insert watermarks in attribute names and text nodes.

[0023] In one embodiment of the present invention, in step S4, during the watermark embedding process, if the length of the field value exceeds a preset threshold, a watermark is inserted in the first 1 / 3 of the field value; for Base64 encoded fields, the field is re-Base64 encoded after the watermark is inserted.

[0024] This invention also provides a method for extracting invisible watermarks based on large model adaptation, comprising the following steps:

[0025] S5: Use regular expressions to match and extract continuous zero-width character sequences from the data;

[0026] S6: According to the mapping rules between zero-width characters and binary, restore the extracted zero-width character sequence to binary data;

[0027] S7: Merge redundant embedded binary data segments, remove duplicate parts, and integrate the segments into a complete binary watermark sequence;

[0028] S8: Convert the extracted binary watermark sequence into a hexadecimal string to obtain the extracted hash value; extract the hash value calculated based on metadata stored in the system log; determine the validity of the watermark by comparing whether the extracted hash value is consistent with the hash value stored in the system log.

[0029] In one embodiment of the present invention, it further includes:

[0030] S9: If the judgment result is consistent, extract the metadata using the hexadecimal string, associate it with the log system, locate the detailed information when the data leak occurred, and realize the source of the data leak.

[0031] Compared with existing technologies, the present invention provides a method for adding and extracting invisible watermarks based on large-model adaptive methods. By acquiring and concatenating multi-dimensional metadata, generating a unique hash code, and then converting it into a zero-width character watermark, the method achieves complete invisibility without affecting data semantics. The block redundancy embedding strategy combined with a majority voting error correction mechanism ensures that the watermark can still be recovered when some data is lost or tampered with. At the same time, it intelligently processes ultra-long fields and Base64 encoded fields to enhance robustness. Asynchronous generation and lightweight computation greatly reduce API response latency, and through priority rules and multi-format dedicated embedding logic, it achieves good compatibility with mainstream data formats such as JSON and XML. In addition, the method has flexible and scalable characteristics, which can meet the needs of different business scenarios, and is especially suitable for fields with strict data security requirements. Attached Figure Description

[0032] Figure 1 This is a flowchart of an invisible watermark addition method based on large model adaptation according to an embodiment of the present invention;

[0033] Figure 2 This is a flowchart of an invisible watermark extraction method based on large model adaptation according to an embodiment of the present invention. Detailed Implementation

[0034] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0035] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0036] like Figure 1 As shown, a method for adding an invisible watermark based on large model adaptation according to a preferred embodiment of the present invention includes the following steps:

[0037] S1: Retrieve metadata and concatenate the metadata strings in order.

[0038] Specifically, the collected metadata includes static metadata and dynamic metadata. Static metadata includes user ID and device fingerprint (including IP address, MAC address, and User-Agent hash). Dynamic metadata includes request timestamps (accurate to milliseconds), session IDs (UUIDv4 format), and API request paths (such as " / api / v1 / sensor-data").

[0039] The metadata string is concatenated in a fixed order: User ID, Request Timestamp, Session ID, and User-Agent hash value of the device fingerprint. The format is: Metadata = UserID|Timestamp|SessionID|Hash(DeviceFingerprint). An example of the concatenated metadata string is: user_123:1625145600123:550e8400-e29b-41d4-a716-446655440000:9f86d081884c7d659a2feaa0c55ad015.

[0040] In this step, the collected request timestamps ensure the uniqueness of watermarks for different requests from the same user, and the User-Agent hash value used for fingerprinting prevents forgery of request sources.

[0041] S2: Calculate the hash code of the concatenated metadata string and perform binary conversion.

[0042] Specifically, the metadata string is hashed using the SHA-256 algorithm to generate a 64-bit hexadecimal string. For example, the string from the example above is output as the following 64-bit hexadecimal string: 6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875.

[0043] The SHA-256 algorithm is a secure hash algorithm. First, the input data is preprocessed by padding the message to meet specific length requirements and adding the original message length information. Next, eight 32-bit registers are initialized as fixed constants. Then, the preprocessed message is divided into 512-bit blocks, and each block undergoes expansion and 80 rounds of compression calculations. Various logical functions and bitwise operations are used to update the register values ​​during the calculation process. Finally, the register values ​​of all processed blocks are concatenated to obtain a 256-bit hash value. This algorithm has high security and collision resistance, effectively ensuring the integrity and authenticity of the data.

[0044] Converting a hash code to binary specifically involves converting each hexadecimal character into 4 bits of binary, ultimately generating a 256-bit binary sequence B = {b1, b2, ..., b}. i ,...,b 256 For example, hexadecimal "9f" is converted to "10011111".

[0045] The metadata obtained in step S1 and the hash code calculated in step S2 will be recorded in the system log for use in the subsequent watermark extraction process.

[0046] S3: According to the mapping rules between zero-width characters and binary, map each binary bit to the corresponding zero-width character.

[0047] Specifically, all 256 bits of binary data are converted to zero-width characters. The mapping rules between zero-width characters and binary data are as follows:

[0048]

[0049] For example, the binary sequence 0101 is mapped to \u200B\u200C\u200B\u200C.

[0050] The reserved bits (used for expansion) here refer to expandable combinations. For example, 0 can represent 200B and 1 can represent 200C. Alternatively, 1 can be chosen to represent 200D, forming different mapping combinations.

[0051] S4: Divide the mapped zero-width character into 4 segments, determine the insertion position according to the priority rule, and then embed the zero-width character segments into the corresponding positions according to the embedding logic of different data formats.

[0052] Specifically, the 256-bit binary watermark sequence B = {b1,b2,...,bi,...,b256} is divided into four 64-bit segments, B = B1||B2||B3||B4. During embedding, each segment is embedded three times.

[0053] A majority voting mechanism is used, where the value that appears most frequently among the three embedded watermarks for each segment is taken as the final embedded watermark. For example, if a segment has three embedded values ​​of "010", "010", and "011", the final embedded watermark will be "010". This example only uses a 3-bit embedded value; in practice, a 64-bit embedded value is used.

[0054] The majority voting mechanism corrects errors by counting the frequency of occurrences of binary sequence segments. The specific process is as follows: First, multiple redundant embedding versions of the same watermark sub-block (e.g., the three embedding results 010, 010, 011 in the example) are aligned bit-by-bit to form a "bit group" to be processed. Then, for each aligned position, the binary values ​​of all redundant embedding versions at that position are iterated (e.g., the first position in the example has three values ​​of 0, 0, 0; the second position has 1, 1, 1; and the third position has 0, 0, 1), and the frequency of occurrence of different values ​​is counted. Finally, the value with the highest frequency is selected as the final valid bit for that position (e.g., in the third position in the example, 0 appears twice and 1 appears once, so the final value for that position is 0). The above steps are repeated until all positions are processed, resulting in the corrected watermark as the final embedded watermark.

[0055] Specifically, in the watermark embedding process of API gateway response data, priority rules are used to determine the watermark insertion position. The watermark insertion position in each case is as follows:

[0056] 1. Ending String Field Values: In the response data, for all string field values, insert a watermark at the end of the string. For example, insert a watermark after the JSON data "value":"data\u200B". This method is highly discreet because zero-width characters are invisible and do not change the display content of the string itself. Furthermore, during data parsing, it usually does not affect the normal reading and use of the string content, and has minimal impact on the data structure.

[0057] 2. Field Name Suffix: When it's impossible to insert a watermark at the end of a string field value, choose to insert it at the suffix of the field name. For example, in XML data...<data\u200C> 30< / data\u200C> A watermark is inserted at the suffix position. Although this method modifies the field name, since zero-width characters do not affect the display and basic semantics of the characters, it can still be recognized and processed normally in most data processing logics that support dynamic parsing of field names. However, compared to the first case, it may have some impact on some parsing programs that strictly rely on fixed field names.

[0058] 3. Array Index Gaps: If the first two methods are not applicable, insert zero-width character watermarks between array elements. For example, when JSON data contains a large number of array structures and there are no suitable string fields or field names to insert, choose to insert zero-width character watermarks between the JSON array elements. This insertion method changes the data structure relatively significantly because it directly adds characters between array elements. However, while ensuring the watermark's invisibility, as long as the parser can ignore the zero-width characters when processing the array, it can still parse the values ​​of the array elements normally. However, in some scenarios with very strict array format requirements, additional adaptation processing may be required.

[0059] In addition, sensitive fields and numeric fields should be excluded when adding watermarks. Sensitive fields refer to fields containing personal privacy information, trade secrets, or other sensitive data that needs protection (such as `password`, `token`); numeric fields refer to fields used to store numerical data, which can be divided into different types such as integer and floating-point (such as `"age":30`).

[0060] Specifically, the embedding logic based on different data formats can be mainly divided into embedding in JSON data and embedding in XML data.

[0061] In JSON data, watermarks can be embedded in two ways: by using the end of the field value and by using the suffix of the field name.

[0062] For example:

[0063] {

[0064] "sensor_id":"S01\u200B\u200C", / / Field value insertion

[0065] "\u200B\u200Clocation":{ / / Field name insertion

[0066] "lat":39.9042,

[0067] "lng":116.4074

[0068] }

[0069] }

[0070] The "sensor_id" field uses a strategy of inserting zero-width characters \u200B\u200C at the end of the value to hide the watermark without changing the semantics of the data; the "location" field adds zero-width characters before the field name, becoming "\u200B\u200Clocation". Both methods utilize the characteristics of the JSON data structure to ensure the concealment of the watermark and the availability of the data.

[0071] For XML data, choose to insert watermarks in attribute names and text nodes.

[0072] For example:

[0073] <response>

[0074] <data\u200B value="123" / > <!-- attribute name insertion-->

[0075] <timestamp> 1625145600\u200C< / timestamp> <!--Text node insertion-->

[0076] < / response>

[0077] The `data` tag inserts a zero-width character `\u200B` before the attribute name `value` to ensure that the attribute value parsing is not affected; the `timestamp` tag adds `\u200C` after the text node `1625145600`. This approach fits the structural characteristics of XML tags and text, integrating watermark information into the data while avoiding damage to XML formatting and data integrity.

[0078] Optionally, during the watermark embedding process, if the field value of the original data is too long (e.g., a field value in JSON or a node value in XML) exceeds a preset threshold (e.g., 1KB), to prevent excessive impact on data performance and storage, the watermark is inserted only in the first 1 / 3 of the field value. This ensures watermark traceability while minimizing interference with normal data use. For Base64 encoded fields, considering the special nature of the encoding format, the system automatically re-Base64 encodes the field after watermark insertion. This operation ensures successful watermark embedding while maintaining data compliance with Base64 encoding standards, preventing format errors caused by watermark insertion and maintaining data legitimacy and availability during transmission and processing. The above operations aim to address special data scenarios and ensure the effectiveness of watermark embedding and data integrity.

[0079] like Figure 2 As shown, a method for extracting invisible watermarks based on large model adaptation according to a preferred embodiment of the present invention includes the following steps:

[0080] S5: Use regular expressions to match and extract continuous zero-width character sequences from the data.

[0081] The regular expression [\u200B-\u200D\uFEFF]+ is used to accurately match and extract consecutive zero-width character sequences from the data. This expression supports all zero-width characters within the Unicode standard, ensuring comprehensive extraction.

[0082] S6: According to the mapping rules between zero-width characters and binary, restore the extracted zero-width character sequence to binary data.

[0083] Specifically, the mapping rules between zero-width characters and binary are as follows:

[0084]

[0085] S7: Merge redundant embedded binary data segments, remove duplicate parts, and integrate each segment into a complete binary watermark sequence to provide a data foundation for subsequent verification.

[0086] According to the majority voting mechanism in step S4, the final embedded watermark is the value that appears most frequently. Therefore, in the worst case, the binary data segment embedded in the provided watermark data is corrupted to less than 30%, and the similarity between the restored watermark and the original watermark is less than 100%.

[0087] S8: Convert the extracted binary watermark sequence into a hexadecimal string to obtain the extracted hash value; extract the hash value calculated based on metadata stored in the system log; determine the validity of the watermark by comparing whether the extracted hash value is consistent with the hash value stored in the system log.

[0088] The metadata in the above steps is the metadata information obtained in step S1 of the aforementioned addition method. This metadata information and its hash value are both recorded in the system log. Therefore, it is not necessary to recalculate the hash value in this step. The validity of the watermark can be determined by directly comparing the extracted hash value with the hash value stored in the log.

[0089] S9: If the judgment result is consistent, extract metadata (such as user ID, timestamp) using hexadecimal strings, correlate with the log system, quickly locate the request source, operation time and other detailed information when the data leak occurred, and achieve accurate tracing of the data leak.

[0090] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A method for adding a large model-based adaptive invisible watermark, characterized by, The method comprises the following steps: S1: Obtain metadata and concatenate metadata strings in order; The collected metadata includes static metadata and dynamic metadata, the static metadata includes a user ID and a device fingerprint, and the dynamic metadata includes a request timestamp; The metadata strings are concatenated in the order of user ID, request timestamp, session ID, and User-Agent hash value of the device fingerprint; S2: Calculate the hash encoding of the concatenated metadata string and perform binary conversion; S3: According to the mapping rule of zero-width characters and binary, each binary bit is mapped to the corresponding zero-width character; wherein the mapping rule of zero-width characters and binary is as follows: ； S4: Divide the mapped zero-width characters into 4 segments, determine the insertion position according to the priority rule, and then embed the zero-width character segments into the corresponding position according to the embedding logic of different data formats; During embedding, each segment of zero-width character is embedded 3 times; A majority voting mechanism is adopted, and the value with the highest frequency in the 3 embedded watermarks is taken as the final embedded watermark; According to the priority rule, the insertion position is determined, which comprises: In the response data, for all string type field values, the watermark is inserted at the end of the string; When it is not possible to insert at the end of the string type field value, the watermark is inserted at the suffix position of the field name; If the above two methods are not applicable, the zero-width character watermark is inserted between the array elements.

2. The method of claim 1, wherein, In step S2, the metadata string is hashed using the SHA-256 algorithm to generate a 64-bit hexadecimal string. Then, each hexadecimal character is converted to binary, ultimately generating a 256-bit binary sequence B = {b1, b2, ..., b...}. i ,..., b 256 } 3. The method of claim 2, wherein, In step S4, the mapped zero-width characters are divided into 4 segments, which comprises dividing the 256-bit binary watermark sequence B into 4 segments.

4. The method of claim 2, wherein the method further comprises: In step S4, the embedding logic according to different data formats mainly includes embedding in JSON data and embedding in XML data: In JSON data, watermark embedding is realized through field value end and field name suffix; For XML data, the watermark is inserted in the attribute name and text node.

5. The method of claim 2, wherein the method further comprises: In step S4, during the watermark embedding process, if the length of the field value exceeds the preset threshold, the watermark is inserted in the first 1 / 3 part of the field value; For Base64 encoded fields, after inserting the watermark, the field is re-encoded in Base64.

6. An extraction method of a large model adaptation-based invisible watermark, characterized by, The method for extracting the invisible watermark added by the method for adding the invisible watermark based on large model self-adaption according to any one of claims 1-5, the extraction method comprising the following steps: S5: Use regular expressions to match and extract continuous zero-width character sequences from the data; S6: According to the mapping rule of zero-width characters and binary, the extracted zero-width character sequence is restored to binary data; S7: Merge the redundantly embedded binary data segments, remove the duplicate parts, and integrate each segment into a complete binary watermark sequence; S8: Convert the extracted binary watermark sequence into a hexadecimal string to obtain the extracted hash value; Extract the hash value calculated according to the metadata stored in the system log; By comparing the extracted hash value with the hash value stored in the system log, the validity of the watermark is determined.

7. The large model-based adaptation of an extraction method of an invisible watermark according to claim 6, characterized in that, Further comprising: S9: If the judgment result is consistent, metadata is extracted using a hexadecimal string, a correlation log system is used to locate detailed information at the time of data leakage, and data leakage is traced.

Images