Data coloring method, system, device and storage medium

Abstract

The present invention discloses a data coloring method, system, device and storage medium, which relate to the field of data security technology. The method includes: extracting statistical features of training data and obtaining predefined coloring features; using a feature generator to fuse the statistical features with the predefined coloring features to obtain fusion features, and training a coloring model based on the fusion features so that the predefined coloring features are embedded in the coloring data generated by the coloring model; detecting whether the coloring data carries the predefined coloring features through a traceability model to determine the source of the coloring data. By training the coloring model, the generated coloring data is embedded with invisible statistical beacons while maintaining the characteristics of the original data, ensuring the security and traceability of the data. By detecting whether the coloring data carries the predefined coloring features, the source of the data can be accurately identified to ensure the authenticity and integrity of the data.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a data coloring method, system, device and storage medium. Background Art

[0002] With the dramatic increase in data volumes and the growing prevalence of data sharing, the security and traceability of structured data stored in various databases are becoming increasingly important. Structured data, such as JSON and tabular data, is widely used in various fields, including finance and healthcare. To ensure the integrity and traceability of this data during use, data coloring technology has been proposed. By coloring structured data, invisible statistical beacons can be embedded in the data, allowing identification and traceability through specific technical means, providing reliable evidence of the data's origin, flow, and modification. This technology not only effectively prevents malicious tampering of structured data but also ensures regulatory oversight and compliance during the use of structured data.

[0003] The current mainstream coloring technology uses generative watermarking. By modifying the LLM sampling process, this method introduces a statistical feature into the generated text, which can be identified during the detection phase. This technology mainly consists of three components: a random seed generator, a sampling algorithm, and a scoring function. The random seed generator generates a random seed based on the previous text and a key. The sampling algorithm uses this seed to select the next token from the LLM vocabulary distribution. The random watermark function (g-values) is introduced to influence the token selection, thereby embedding a watermark in the generated text. The detection phase determines whether the text contains a watermark by calculating the g-value score.

[0004] However, generative watermarking techniques have significant drawbacks and limitations when processing structured data. Structured data generation is often subject to strict constraints, with a limited range of values ​​for each column. For example, a payment method column might only allow two options. In this case, the range of options for generating the next token is very narrow, resulting in a very limited space for watermark embedding. Generative watermarking relies on modifying the probability distribution of vocabulary. In structured data, due to the extremely small number of possible tokens, watermark embedding can significantly impact data accuracy and consistency, and even cause the generated data to fail to comply with business rules. Furthermore, generative watermarking techniques often assume high entropy (i.e., uncertainty in the vocabulary distribution) during text generation to introduce watermarks during sampling. However, structured data generation often has low entropy, especially when the range of values ​​for certain columns is very limited. This low entropy environment weakens the effectiveness of watermark embedding, as the strength of the watermark relies on the model's selection of multiple candidate tokens during generation. If the model almost always selects the same token during generation, watermark embedding becomes extremely difficult or even impossible. Furthermore, generative watermarking techniques are sensitive to data editing and modification. After being generated, structured data may undergo multiple processing and transformations, such as data cleaning, format conversion, or manual editing. These operations may destroy the statistical characteristics of the watermark and cause watermark detection failure. Summary of the Invention

[0005] The primary objective of this invention is to provide a data coloring method, system, device, and storage medium. By training a coloring model, the generated coloring data retains the characteristics of the original data while embedding invisible statistical beacons to ensure data security and traceability. By detecting whether the coloring data carries predefined coloring features, the source of the data can be accurately identified, ensuring its authenticity and integrity.

[0006] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:

[0007] According to a first aspect of an embodiment of the present application, a data coloring method is provided, the method comprising:

[0008] Extract statistical features of training data and obtain predefined staining features;

[0009] Using a feature generator, the statistical feature is fused with a predefined staining feature to obtain a fused feature, and a staining model is trained based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model;

[0010] The source tracing model is used to detect whether the dyeing data carries the predefined dyeing feature, so as to determine the source of the dyeing data.

[0011] Optionally, detecting whether the dyeing data carries the predefined dyeing feature by using a traceability model to determine the source of the dyeing data includes:

[0012] extracting statistical features to be detected from the staining data;

[0013] Calculating the similarity between the statistical feature to be detected and the predefined staining feature to obtain a similarity score;

[0014] Performing a judgment based on the similarity score and a preset threshold to obtain a judgment result; if the similarity score is greater than or equal to the preset threshold, determining that the staining data carries the predefined staining feature; if the similarity score is less than the preset threshold, determining that the staining data does not carry the predefined staining feature;

[0015] According to the judgment result, the source of the staining data is determined.

[0016] Optionally, training a staining model based on the fusion feature includes:

[0017] Inputting the fusion feature into the staining model to obtain a model output;

[0018] Calculate the total loss between the model output and the training data according to a preset loss function;

[0019] Calculating the gradient according to the total loss and updating the parameters of the dyeing model by back propagation;

[0020] The above steps are repeated until the coloring model converges or a predetermined number of iterations is reached.

[0021] Optionally, the total loss between the model output and the training data is calculated according to a preset loss function according to the following formula:

[0022]

[0023] in, L data The loss of similarity between the model output and the training data, L watermark The loss of extractability of the predefined staining features, L robust is the loss of robustness of the predefined staining feature to data operations; it is specifically calculated according to the following formula:

[0024]

[0025] Where G(x) is the model output and x is the training data;

[0026]

[0027] Where E is the feature extractor for extracting the output from the model G ( x ) extracts feature information; w is a predefined staining feature;

[0028]

[0029] T represents the data transformation operation function.

[0030] Optionally, the statistical feature is fused with a predefined staining feature using a feature generator to obtain a fused feature, including:

[0031] The statistical feature and the predefined coloring feature are input into the feature generator, so that the feature generator adjusts the occurrence frequency and / or order of the column values ​​of the statistical feature according to the predefined coloring feature to embed the predefined coloring feature into the statistical feature to obtain a fusion feature.

[0032] Optionally, embedding the predefined staining feature into the statistical feature comprises:

[0033] Determining characteristic attributes of the statistical features, wherein the characteristic attributes include continuous features and discrete features;

[0034] If the characteristic attribute of the statistical feature is a continuous feature, a set statistical mode is embedded in the numerical domain distribution of the statistical feature. The set statistical mode is defined according to the following formula:

[0035] P ( X ∈[ a , b ])= c

[0036] in, a , b is a predefined interval, c is the characteristic probability;

[0037] If the characteristic attribute of the statistical feature is a discrete feature, encoding the coloring feature in the value frequency of the statistical feature;

[0038] The coloring features are embedded in the correlation between statistical features, where for the conditional probability model, the formula is used. P ( X ∣ Y ) defines the association relationship.

[0039] Optionally, before extracting statistical features of the training data using the feature generator, the method further includes:

[0040] Performing outlier detection and outlier processing on the training data, wherein the outlier processing method includes median filling and / or deletion; and / or,

[0041] The training data is standardized so that data of different scales are converted to the same magnitude.

[0042] According to a second aspect of an embodiment of the present application, a data coloring system is provided, the system comprising:

[0043] Feature preparation module, used to extract statistical features of training data and obtain predefined staining features;

[0044] A model training module is used to use a feature generator to fuse the statistical features with predefined staining features to obtain fused features, and train a staining model based on the fused features so that the predefined staining features are embedded in the staining data generated by the staining model;

[0045] The detection module is used to detect whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data.

[0046] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0047] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described in the first aspect above.

[0048] In summary, the embodiments of the present application provide a data coloring method, system, device and storage medium. By extracting the statistical features of the training data and obtaining predefined coloring features; using a feature generator to fuse the statistical features with the predefined coloring features to obtain fusion features, and training a coloring model based on the fusion features, so that the predefined coloring features are embedded in the coloring data generated by the coloring model; detecting whether the coloring data carries the predefined coloring features through a traceability model to determine the source of the coloring data. By training the coloring model, the generated coloring data is embedded with invisible statistical beacons while maintaining the characteristics of the original data, ensuring the security and traceability of the data. By detecting whether the coloring data carries the predefined coloring features, the source of the data can be accurately identified to ensure the authenticity and integrity of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0050] The structures, proportions, sizes, etc. illustrated in this specification are intended solely to complement the contents disclosed herein and to facilitate understanding and reading by persons skilled in the art. They are not intended to limit the conditions under which the present invention may be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportions, or adjustments in sizes, without affecting the efficacy and objectives of the present invention, shall remain within the scope of the technical contents disclosed herein.

[0051] Figure 1 A flow chart of a data coloring method provided in an embodiment of the present application;

[0052] Figure 2 Dyeing flow chart provided for the embodiments of this application;

[0053] Figure 3 A schematic diagram of a data coloring system provided in an embodiment of the present application;

[0054] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application is shown;

[0055] Figure 5 A diagram showing a computer-readable storage medium provided in an embodiment of the present application.

[0056] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0059] In addition, the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referenced. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0060] In the present invention, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can mean fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two elements or interaction between two elements, unless otherwise specified. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0061] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0062] This paper proposes a coloring technology for structured data. By embedding invisible statistical beacons within the data, it achieves data security protection and traceability. This technical solution can effectively address various technical challenges that may be encountered during data generation, editing, conversion, and detection, providing strong protection for the integrity, traceability, and compliance of structured data. Compared with traditional methods, this paper's innovative design for embedding and detecting invisible statistical beacons makes the protection of structured data more efficient and robust.

[0063] 1. In response to the problem of a narrow range of optional tokens in structured data, an embodiment of the present application introduces a rule-based invisible statistical beacon embedding mechanism. This mechanism designs an invisible statistical beacon embedding method that adapts to a limited value range by comprehensively considering the business rules and constraints of the data. By adjusting the frequency or order of occurrence of column values ​​and embedding statistical features, it ensures that the embedding of invisible statistical beacon information does not affect the use value of the data while meeting the requirements of business rules. This method makes effective use of the space for embedding invisible statistical beacons in structured data, which not only maintains the original characteristics of the data, but also ensures the integrity and effectiveness of the invisible statistical beacons.

[0064] 2. In a low-entropy environment, the effect of watermark embedding is often poor, and traditional watermark technology may not be able to effectively embed watermarks due to the small amount of information in the data. To this end, the embodiment of the present application proposes a context-based invisible statistical beacon enhancement technology. This method increases the embedding dimension of the invisible statistical beacon by utilizing the correlation between multiple columns in structured data, so that the invisible statistical beacon information can be distributed and crossed between multiple columns, thereby greatly improving the robustness of the invisible statistical beacon. Even in a low-entropy environment, the invisible statistical beacon information can exist in a more complex and hidden form, which greatly enhances the stability and anti-interference ability of the invisible statistical beacon.

[0065] 3. During the process of data editing and conversion, the watermark may be destroyed or lost, which is an important challenge faced by existing watermark technology. The embodiment of the present application designs an editing-resistant invisible statistical beacon algorithm to disperse and embed invisible statistical beacon information in multiple locations, making full use of the redundancy characteristics in structured data. This method ensures that even if some parts of the data are modified or lost, the remaining invisible statistical beacon information can still be extracted and verified, thereby achieving higher invisible statistical beacon robustness. Through this redundant embedding strategy, the embodiment of the present application effectively reduces the impact of data editing and conversion on the integrity of the invisible statistical beacon, ensuring the reliability of the invisible statistical beacon in practical applications.

[0066] 4. Traditional generative watermarking technology usually has a strong dependence on the generation model, which limits the breadth and flexibility of its application. The embodiment of the present application breaks through this limitation and proposes a model-independent invisible statistical beacon embedding framework. This framework uses post-processing technology to independently embed invisible statistical beacons according to preset rules or statistical methods after generating data, without relying on specific generation models or tools. In this way, whether in different generation models or different application scenarios, the invisible statistical beacon embedding method of the embodiment of the present application can be flexibly adapted and has strong versatility and compatibility.

[0067] 5. The detection accuracy and efficiency of invisible statistical beacons are also an important issue in the application of data coloring technology. For the detection of invisible statistical beacons in structured data, the embodiment of the present application proposes a fast invisible statistical beacon detection method based on column features by optimizing the detection algorithm. This method quickly locates and verifies the invisible statistical beacon information in the data by deeply exploring the column type, value range and other features of structured data. By reducing the computational complexity and combining it with an efficient matching algorithm, the embodiment of the present application greatly improves the detection efficiency while ensuring the detection accuracy, making the extraction and tracing process of invisible statistical beacons more convenient and efficient.

[0068] Figure 1A data coloring method provided in an embodiment of the present application is shown, and the method includes:

[0069] Step 101: extracting statistical features of training data and obtaining predefined staining features;

[0070] Step 102: using a feature generator to fuse the statistical feature with a predefined staining feature to obtain a fused feature, and training a staining model based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model;

[0071] Step 103: Detecting whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data.

[0072] In one possible implementation, before extracting statistical features of the training data using the feature generator, the method further includes:

[0073] Performing outlier detection and outlier processing on the training data, wherein the outlier processing method includes median filling and / or deletion; and / or performing standardization processing on the training data so that data of different scales are converted to the same magnitude.

[0074] By detecting and addressing outliers, we can improve the quality of training data and ensure data consistency and reliability. Outliers can adversely affect model training, leading to reduced performance. By padding or removing outliers with the median, we can reduce their impact on model training and enhance model robustness. By converting data of different scales to the same magnitude, we can eliminate the impact of scale differences on model training, improving model stability and accuracy.

[0075] In one possible implementation, in step 101,

[0076] The statistical features are fused with predefined staining features using a feature generator to obtain fused features, including:

[0077] The statistical feature and the predefined coloring feature are input into the feature generator, so that the feature generator adjusts the occurrence frequency and / or order of the column values ​​of the statistical feature according to the predefined coloring feature to embed the predefined coloring feature into the statistical feature to obtain a fusion feature.

[0078] In a possible implementation, in step 101, embedding the predefined staining feature into the statistical feature includes:

[0079] Determine a characteristic attribute of the statistical feature, where the characteristic attribute includes a continuous feature and a discrete feature; if the characteristic attribute of the statistical feature is a continuous feature, embed a set statistical pattern in the numerical domain distribution of the statistical feature, where the set statistical pattern is defined according to the following formula:

[0080] P ( X ∈[ a , b ])= c

[0081] in, a , b is a predefined interval, c is the characteristic probability;

[0082] If the characteristic attribute of the statistical feature is a discrete feature, the coloring feature is encoded in the value frequency of the statistical feature; the coloring feature is embedded in the correlation relationship between the statistical features, wherein, for the conditional probability model, the formula is used P ( X ∣ Y ) defines the association relationship.

[0083] By embedding set statistical patterns in the numerical domain distribution of continuous features, or encoding coloring features in the value frequency of discrete features, or embedding coloring features in the correlation between features, the validity and uniqueness of coloring features can be ensured, thereby improving data security and traceability.

[0084] In a possible implementation, in step 102, training a staining model based on the fusion feature includes:

[0085] The fusion feature is input into the coloring model to obtain a model output; the total loss between the model output and the training data is calculated according to a preset loss function; the gradient is calculated according to the total loss, and the parameters of the coloring model are updated by back propagation; the above steps are repeated until the coloring model converges or a predetermined number of iterations is reached.

[0086] By training the coloring model, the generated coloring data retains the characteristics of the original data while embedding invisible statistical beacons, ensuring data security and traceability. Model performance is measured by calculating the total loss between the model output and the training data using a preset loss function. This total loss includes data similarity loss, watermark extractability loss, and robustness loss. By combining these losses, the model's performance can be comprehensively evaluated, ensuring that the generated coloring data retains the characteristics of the original data while also incorporating reliable invisible statistical beacons. By continuously updating the model parameters, the model gradually reduces the total loss, improving the quality and reliability of the generated data, and ensuring that the generated coloring data contains both predefined coloring features and the characteristics of the original data.

[0087] In one possible implementation, in step 102, the total loss between the model output and the training data is calculated according to a preset loss function according to the following formula:

[0088]

[0089] in, L data The loss of similarity between the model output and the training data, L watermark The loss of extractability of the predefined staining features, L robust is the loss of robustness of the predefined staining feature to data operations; it is specifically calculated according to the following formula:

[0090]

[0091] Wherein, G(x) is the model output, and x is the training data;

[0092]

[0093] Where E is a feature extractor for extracting the output from the model G ( x ) extracts feature information; w is a predefined staining feature;

[0094]

[0095] Where T represents the data transformation operation function.

[0096] In a possible implementation, in step 103, detecting whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data includes:

[0097] Extracting the statistical feature to be detected from the staining data; performing similarity calculation between the statistical feature to be detected and the predefined staining feature to obtain a similarity score; performing judgment based on the similarity score and a preset threshold to obtain a judgment result; if the similarity score is greater than or equal to the preset threshold, judging that the staining data carries the predefined staining feature; if the similarity score is less than the preset threshold, judging that the staining data does not carry the predefined staining feature; and determining the source of the staining data based on the judgment result.

[0098] By testing staining data for predefined staining signatures, the data's source can be accurately identified, ensuring its authenticity and integrity. Extracting the statistical signatures to be tested ensures that subsequent similarity calculations and judgments are based on accurate data features. Similarity calculations quantify the degree of similarity between the data to be tested and the predefined staining signatures. Threshold determination accurately distinguishes whether the data carries the predefined staining signatures. By determining the source of the staining data, the authenticity and integrity of the data can be ensured.

[0099] Through the above steps, it is possible to accurately detect whether the staining data carries the predefined staining characteristics, thereby determining the source of the data and ensuring the authenticity and integrity of the data.

[0100] Imagine using an intelligent text generation model to automatically compose a story. Based on the input content, the model generates words that conform to grammatical and semantic rules to construct coherent and meaningful text. Traditional watermark generation techniques typically embed implicit information by adjusting the probability of each word being selected during the generation process. This approach is relatively simple and easily detected. However, the introduction of an invisible statistical beacon embedding mechanism significantly improves the watermark generation process. Watermark embedding now not only considers the probability of each word but also involves optimizing the frequency and order of word occurrence. Specifically, when embedding watermark information, the model adjusts the generated content based on predefined business rules (for example, requiring certain words to appear a certain number of times or in a certain order).

[0101] For example, during the generation process, suppose a specific word, "beautiful," appears 80% of the time in natural language generation, while the word "ugly" appears only 20%. By embedding invisible statistical beacons, we can fine-tune the frequency and order of these words. For example, we can set rules to ensure that the word "beautiful" appears twice in a generated paragraph, each time in a specific location, such as the beginning and end of the paragraph, to emphasize a certain theme or emotional tone. Meanwhile, "ugly" appears only once, possibly in the middle of the paragraph, to provide contrast or contrast. In this way, invisible statistical information (such as the frequency and order of "beautiful") becomes part of the watermark.

[0102] These fine-tuning of frequency and order do not affect the natural flow or semantic accuracy of the text, because they follow specific business rules and constraints, ensuring that the text still conforms to language habits and business needs. At the same time, this method provides a more complex and effective watermark embedding mechanism, making the embedding of watermark information more difficult to detect and tamper with. Even if the generated text is later analyzed, the embedded information can only be extracted through specific rules and statistical methods. This rule-based invisible statistical beacon embedding method not only optimizes the embedding space of watermark information, but also effectively preserves the original characteristics of the data and ensures the integrity and validity of the watermark information. It is particularly suitable for application scenarios where data privacy and traceability need to be guaranteed. For example, in the fields of literary creation, news reporting, academic papers, etc., embedding watermarks in this way can protect the author's rights while ensuring the integrity and credibility of the text.

[0103] The method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0104] This technical solution mainly includes three parts: model training module, coloring module, and detection module. It includes data preprocessing, rule engine, statistical feature simulator, invisible statistical beacon embedding model, feature extractor and other parts. The specific process is as follows Figure 2 .

[0105] Structured data first undergoes data preprocessing before entering the model training phase. During the model training phase, the feature generator and model training module work together to generate a coloring model. The coloring model embeds coloring features into the data, generating coloring data. The coloring data is then tested using the traceability model, ultimately outputting the test results. This flowchart illustrates the entire process from data preprocessing to final test results, covering the three main modules of model training, coloring, and testing.

[0106] Before data coloring can be performed, sufficient structured data and specific coloring features are required for model training. These features act like DNA markers in the data. These custom coloring features are fed into a feature generator for feature learning along with the preprocessed data. The feature generator analyzes the statistical characteristics of the data and then integrates them with predefined coloring features for model training. During the model training phase, the system learns how to naturally embed these features into the data, enabling it to automatically incorporate invisible statistical beacons into new data as it is generated. Data coloring with invisible statistical beacons appears virtually indistinguishable from the original data. For example, the basic statistical characteristics of a transaction record remain unchanged, and the business logic is entirely sound. However, in reality, each data carries a unique statistical feature beacon, an anti-counterfeiting mark that is imperceptible to the naked eye. When data provenance verification is required, the provenance model simultaneously analyzes suspicious data and known coloring features. Using complex statistical analysis, the provenance model determines whether the data carries the predefined invisible statistical beacon features, thereby confirming its origin as a trusted source. This ensures both data availability and data security. This dyeing technology is like adding a unique fingerprint to the data, which protects the data security without affecting the normal use value of the data.

[0107] Phase 1: Data preprocessing: Structured data first passes through the data preprocessing module, which is responsible for converting the raw data into a standardized and normalized form, including steps such as data cleaning, standardization, and feature engineering.

[0108] The first aspect: application of data preprocessing module.

[0109] The data preprocessing module is the foundation of the structured data coloring system. Its core goal is to convert raw data into a standardized and normalized form. From a technical perspective, this module mainly includes three key steps: data cleaning, standardization, and feature engineering. First, the data cleaning process uses statistical methods to detect and process outliers, usually using the Z-score method to identify outliers:

[0110]

[0111] Where X is the observed value, μ is the mean, and σ is the standard deviation. Identified outliers can be addressed using strategies such as median padding or deletion. Secondly, the normalization process converts features of different scales to the same magnitude. Common methods include Min-Max and Z-score normalization. Finally, the feature engineering phase involves feature selection and transformation, such as encoding categorical data and decomposing temporal features. This module provides a high-quality data foundation for subsequent feature generation and model training. By reducing data noise, unifying data formats, and extracting effective features, it significantly improves model training results and the stability of invisible statistical beacon embedding.

[0112] Aspect 2: Definition of staining characteristics

[0113] The coloring feature definition module is responsible for designing the specific form and embedding strategy of the invisible statistical beacon (also known as the coloring feature). This module's role is to provide the basic invisible statistical beacon feature definition for the entire coloring system. These features must ensure the effectiveness of the invisible statistical beacon while maintaining the naturalness and usability of the data.

[0114] Theoretically, good staining features must meet three conditions: imperceptibility (minimal impact on data distribution), robustness (strong anti-interference ability) and uniqueness (easy to identify and trace).

[0115] Feature design is usually based on statistical properties of the data, such as:

[0116] For continuous features, specific statistical patterns can be embedded in their value distribution:

[0117]

[0118] Where [a, b] is a predefined interval, and c is the feature probability. The formula indicates that the probability of a continuous feature X being within the interval a, b is c. Here, a and b are predefined intervals, and c is the feature probability. This formula describes how to embed a specific statistical pattern within the range of a feature's values, acting as an invisible statistical beacon.

[0119] For discrete features, invisible statistical beacon information can be encoded in their value frequencies. In addition, invisible statistical beacons can also be embedded in the correlation between features, such as designing specific conditional probability patterns:

[0120]

[0121] It represents the probability of feature X occurring under the condition that feature Y occurs, that is, the conditional probability. This formula is used to describe the correlation between features, and invisible statistical beacons can be embedded in this relationship.

[0122] It can be seen that the present application provides an invisible statistical beacon encoding technology based on statistical feature fusion. This technology innovatively encodes invisible statistical beacon information into the multidimensional statistical features of the data, rather than simply modifying a single data value. This distributed invisible statistical beacon embedding method greatly improves the robustness of the invisible statistical beacon, enabling it to resist various data processing operations. An adaptive feature selection and invisible statistical beacon strength adjustment scheme is also provided. This scheme can automatically select the most suitable feature dimension for invisible statistical beacon embedding according to the characteristics of different types of data, and dynamically adjust the invisible statistical beacon strength. This intelligent adjustment mechanism ensures the optimal performance of the dyeing system in different application scenarios.

[0123] The second stage is the application of feature generator.

[0124] The feature generator is the core component for implementing invisible statistical beacons for structured data. It works by automatically learning the statistical distribution of data through a deep learning model and incorporating predefined coloring features. The coloring features mentioned here refer to invisible statistical beacons, which are embedded in the data to identify and track it.

[0125] From a mathematical point of view, the feature generator can be expressed as a function:

[0126]

[0127] Where x is the input data and w is the coloring feature (invisible statistical beacon). The goal of this function is to generate new data while embedding the coloring feature into the data.

[0128] The optimization goal of the generator is to minimize the difference between the original data distribution P(x) and the generated data distribution Q(G(x,w)) while maintaining the identifiability of the invisible statistical beacon feature w. This can be expressed as an optimization problem:

[0129]

[0130] Where D represents the distribution distance metric, which is used to measure the difference between two distributions.

[0131] In specific implementation, the feature generator usually adopts an autoencoder structure, including the encoder E ( x ) and decoder D ( z , w ). The encoder E(x) extracts the intrinsic features of the data, and the decoder D(z,w) converts the extracted features z and invisible stat beacon information w Reconstruct into new data.

[0132] During training, an adversarial learning strategy is used to ensure the naturalness of the generated data and the concealment of the invisible statistical beacons. This means that the generated data should be similar in appearance to the original data, while the embedded statistical beacons are not easily detected.

[0133] The role of the feature generator is to build a bridge between structured data features and invisible statistical beacon features, enabling the natural embedding of invisible statistical beacon information while maintaining the statistical characteristics and business value of the data. In this way, even if the data is embedded with invisible statistical beacons, it can still be used for its original purpose, such as data analysis and business decision-making, without being affected by the embedded beacons.

[0134] A deep learning-based adaptive coloring feature generation mechanism. This technology uses a feature generator to automatically learn the statistical distribution characteristics of the data and incorporates predefined invisible statistical beacon features into it, achieving natural embedding of invisible statistical beacons. This approach avoids the direct modification of data structures required by traditional invisible statistical beacon technologies, significantly improving the stealth of invisible statistical beacons and data usability.

[0135] The third stage is model training.

[0136] The preprocessed data, along with the coloring features, is fed into the feature generator for feature learning. The feature generator analyzes the statistical properties of the data and fuses them with predefined coloring features for model training. During the model training phase, the system learns how to naturally embed features into the data, generating data with invisible statistical beacons.

[0137] The model training module is responsible for optimizing the parameters of the entire dyeing system. Its core is to design and optimize the multi-objective loss function. The loss function consists of three parts: L data, L watermark and L robust, each part has its own specific purpose.

[0138] 1. In principle, the training process can be formalized as the following optimization problem:

[0139]

[0140] Total loss L It is composed of three parts, and the weight of each part is λ 1, λ 2, λ 3. Control.

[0141] 2. Data similarity loss: L_data measures the similarity between the generated data and the original data:

[0142]

[0143] Where G(x) is the generated data and x is the original data. G ( x ) and original data x The square of the Euclidean distance is used to measure the difference between two data points.

[0144] 3. Loss of extractability of invisible statistical beacon information: L_watermark ensures the extractability of invisible statistical beacon information:

[0145]

[0146] Ensure invisible stat beacon information w Where E is the feature extractor used to extract the generated data G ( x ) is used to extract the coloring features. w is the coloring feature.

[0147] 4. Loss of robustness of invisible statistical beacons to data operations: L_robust ensures the robustness of invisible statistical beacons to data operations:

[0148]

[0149] T represents various data transformation operations (such as noise and compression). The formula ensures the robustness of the invisible statistical beacon to data operations (such as noise and compression).

[0150] Training uses an iterative optimization strategy, with each iteration consisting of forward propagation to calculate the loss and backpropagation to update the parameters. To improve training results, techniques such as batch normalization are often used to stabilize the training process, and regularization methods such as dropout are used to prevent overfitting.

[0151] This module continuously adjusts model parameters to enable the system to generate new data that retains the characteristics of the original data while also incorporating reliable hidden statistical beacon information. The training process requires a balance between data utility and the strength of the hidden statistical beacon, which is reflected in the choice of weights λi in the loss function.

[0152] This application provides a multi-objective optimization model training method. This method uses a carefully designed loss function to simultaneously optimize three objectives: data feature preservation, invisible statistical beacon embedding strength, and anti-interference capability. By dynamically adjusting the weights of each objective, an optimal balance is achieved between invisible statistical beacon effectiveness and data quality, ensuring the overall performance of the dyeing system.

[0153] Phase 4: Generation of dyeing modules.

[0154] After model training, a staining model is generated.

[0155] The coloring model embeds the coloring features into the data to generate coloring data.

[0156] In the fifth stage, the dyeing data is tested through the traceability model to verify the source of the data.

[0157] The traceability model analyzes suspicious data and known staining characteristics, using statistical analysis to determine whether the data carries predefined invisible statistical beacon characteristics. Based on the traceability model's analysis, it outputs a test result to determine whether the data comes from a trusted source.

[0158] The provenance model works by identifying hidden statistical beacon features in data through statistical tests and machine learning methods. Its core is to construct a discriminator D(x), which outputs the probability that data x contains the target hidden statistical beacon. From a statistical perspective, this can be viewed as a hypothesis testing problem: H0 (no hidden statistical beacon) vs. H1 (presence of hidden statistical beacon). The discriminant process first extracts the statistical feature vector f(x) of the suspicious data. It then calculates similarity with a pre-stored hidden statistical beacon feature template w: s(x) = sim(f(x), w), where sim can be a metric such as cosine similarity. A judgment is made based on the score s(x) and a preset threshold τ: when s(x) > τ, a hidden statistical beacon is considered present. To improve detection reliability, the model typically combines multiple features and detection methods, employing an ensemble learning strategy. This module accurately identifies the source of data, providing technical support for data security and intellectual property protection.

[0159] An intelligent traceability verification mechanism. This mechanism combines traditional statistical tests with modern machine learning methods. Through multi-level feature extraction and pattern matching, it achieves rapid and accurate identification of invisible statistical beacons. Furthermore, the mechanism is adaptable to different types of data perturbations, ensuring the reliability of verification results.

[0160] The technical effects of the technical solution of this application include the following aspects:

[0161] 1. The coloring is highly concealed. By using a deep learning model, the invisible statistical beacon features are naturally integrated into the statistical distribution of the data, making the data with invisible statistical beacons virtually indistinguishable from the original data in terms of appearance. Whether through manual observation or basic statistical analysis, it is difficult to detect the invisible statistical beacon features in the data. This "stealth" greatly improves the security of the invisible statistical beacon.

[0162] 2. Maintaining data usability. This technical solution fully preserves the business value and basic statistical characteristics of the data while embedding invisible statistical beacons. Through a carefully designed feature generator, the coloring process ensures that the core attributes and distribution characteristics of the data are not altered. This allows data with invisible statistical beacons to be directly used in downstream tasks without affecting the accuracy and reliability of data analysis.

[0163] 3. Excellent robustness. The invisible statistical beacon features used in this solution have strong anti-interference capabilities. Even if the data undergoes common operations such as sampling, aggregation, and conversion, the invisible statistical beacon information remains intact. This strong robustness enables the invisible statistical beacon to continue to function during the actual use of data, providing long-term protection for data security.

[0164] 4. High verification efficiency. The traceability model can quickly and accurately identify invisible statistical beacon features in data, supporting rapid verification of large-scale data. Through optimized feature extraction and matching algorithms, invisible statistical beacon detection can be completed in a relatively short time, which is of great significance for real-time verification requirements in practical application scenarios.

[0165] 5. Highly universal. This technical solution is applicable to a variety of structured data types, including numerical and categorical features. Through a flexible feature generation mechanism, the invisible statistical beacon embedding strategy can be automatically adjusted according to the characteristics of different data types, making this solution suitable for a wide range of scenarios.

[0166] 6. The system is highly scalable. The modular design of this solution makes it easy to expand and optimize. Feature generation strategies can be adjusted, training methods updated, or detection algorithms optimized according to specific needs. This flexible architecture allows the system to continuously adapt to new application scenarios and security requirements, demonstrating excellent technical scalability.

[0167] In summary, the embodiment of the present application provides a data coloring method, which extracts statistical features of training data and obtains predefined coloring features; uses a feature generator to fuse the statistical features with the predefined coloring features to obtain fusion features, and trains a coloring model based on the fusion features so that the predefined coloring features are embedded in the coloring data generated by the coloring model; detects whether the coloring data carries the predefined coloring features through a traceability model to determine the source of the coloring data. By training the coloring model, the generated coloring data is embedded with invisible statistical beacons while maintaining the characteristics of the original data, ensuring the security and traceability of the data. By detecting whether the coloring data carries the predefined coloring features, the source of the data can be accurately identified to ensure the authenticity and integrity of the data.

[0168] Based on the same technical concept, the present application also provides a staining data detection system, such as Figure 3 As shown, the system includes:

[0169] Feature preparation module 301, used to extract statistical features of training data and obtain predefined staining features;

[0170] A model training module 302 is configured to use a feature generator to fuse the statistical feature with a predefined staining feature to obtain a fused feature, and train a staining model based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model;

[0171] The detection module 303 is configured to detect whether the dyeing data carries the predefined dyeing feature through a traceability model, so as to determine the source of the dyeing data.

[0172] The present application also provides an electronic device corresponding to the method provided in the above embodiment. Figure 4 , which shows an electronic device provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected via the bus 202. The memory 201 stores a computer program executable on the processor 200. When the processor 200 executes the computer program, it executes the method provided by any of the aforementioned embodiments of the present application.

[0173] Memory 201 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. Communication between the system network element and at least one other network element is achieved through at least one physical port (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0174] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs. The processor 200 executes the programs upon receiving execution instructions. The methods disclosed in any of the aforementioned embodiments of the present application may be applied to or implemented by the processor 200.

[0175] The processor 200 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 200 or by software instructions. The above processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 201 , and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0176] The electronic device provided in the embodiments of the present application and the method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented by them.

[0177] The present application also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Figure 5 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided by any of the aforementioned embodiments is executed.

[0178] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention description and drawings under the concept of the present invention, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A data coloring method, characterized in that: The method comprises: Extract statistical features of training data and obtain predefined staining features; Using a feature generator, the statistical feature is fused with a predefined staining feature to obtain a fused feature, and a staining model is trained based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model; detecting whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data; Training a staining model based on the fusion features includes: Inputting the fusion feature into the staining model to obtain a model output; Calculate the total loss between the model output and the training data according to a preset loss function; Calculating the gradient according to the total loss and updating the parameters of the dyeing model by back propagation; Repeat the above steps until the coloring model converges or reaches a predetermined number of iterations; The total loss between the model output and the training data is calculated using the preset loss function according to the following formula: in, L data The loss of similarity between the model output and the training data, L watermark The loss of extractability of the predefined staining features, L robust is the loss of robustness of the predefined staining feature to data operations; it is specifically calculated according to the following formula: Wherein, G(x) is the model output, and x is the training data; Where E is a feature extractor for extracting the output from the model G ( x ) extracts feature information; w is a predefined staining feature; Wherein, T represents the data transformation operation function; The statistical features are fused with predefined staining features using a feature generator to obtain fused features, including: The statistical feature and the predefined coloring feature are input into the feature generator, so that the feature generator adjusts the occurrence frequency and / or order of the column values ​​of the statistical feature according to the predefined coloring feature to embed the predefined coloring feature into the statistical feature to obtain a fusion feature.

2. The method according to claim 1, wherein Detecting whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data includes: extracting features to be detected from the staining data; Calculating the similarity between the feature to be detected and the predefined staining feature to obtain a similarity score; Performing a judgment based on the similarity score and a preset threshold to obtain a judgment result; if the similarity score is greater than or equal to the preset threshold, determining that the staining data carries the predefined staining feature; if the similarity score is less than the preset threshold, determining that the staining data does not carry the predefined staining feature; According to the judgment result, the source of the staining data is determined.

3. The method according to claim 1, wherein The embedding of the predefined staining feature into the statistical feature comprises: Determining characteristic attributes of the statistical features, wherein the characteristic attributes include continuous features and discrete features; If the characteristic attribute of the statistical feature is a continuous feature, a set statistical mode is embedded in the numerical domain distribution of the statistical feature. The set statistical mode is defined according to the following formula: P ( X ∈[ a , b ])= c in, a , b is a predefined interval, c is the characteristic probability; If the characteristic attribute of the statistical feature is a discrete feature, encoding the coloring feature in the value frequency of the statistical feature; The coloring features are embedded in the correlation between statistical features, where for the conditional probability model, the formula is used. P ( X ∣ Y ) defines the association relationship.

4. The method according to claim 1, wherein Before extracting statistical features of the training data using the feature generator, the method further includes: Performing outlier detection and outlier processing on the training data, wherein the outlier processing method includes median filling and / or deletion; and / or, The training data is standardized so that data of different scales are converted to the same magnitude.

5. A data coloring system, characterized in that: The system comprises: Feature preparation module, used to extract statistical features of training data and obtain predefined staining features; A model training module is used to use a feature generator to fuse the statistical features with predefined staining features to obtain fused features, and train a staining model based on the fused features so that the predefined staining features are embedded in the staining data generated by the staining model; A detection module, configured to detect whether the dyeing data carries the predefined dyeing feature through a traceability model, so as to determine the source of the dyeing data; Training a staining model based on the fusion features includes: Inputting the fusion feature into the staining model to obtain a model output; Calculate the total loss between the model output and the training data according to a preset loss function; Calculating the gradient according to the total loss and updating the parameters of the dyeing model by back propagation; Repeat the above steps until the coloring model converges or reaches a predetermined number of iterations; The total loss between the model output and the training data is calculated using the preset loss function according to the following formula: in, L data The loss of similarity between the model output and the training data, L watermark The loss of extractability of the predefined staining features, L robust is the loss of robustness of the predefined staining feature to data operations; it is specifically calculated according to the following formula: Wherein, G(x) is the model output, and x is the training data; Where E is a feature extractor for extracting the output from the model G ( x ) extracts feature information; w is a predefined staining feature; Wherein, T represents the data transformation operation function; The statistical features are fused with predefined staining features using a feature generator to obtain fused features, including: The statistical feature and the predefined coloring feature are input into the feature generator, so that the feature generator adjusts the occurrence frequency and / or order of the column values ​​of the statistical feature according to the predefined coloring feature to embed the predefined coloring feature into the statistical feature to obtain a fusion feature.

6. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 4.

Images