Data dyeing method, system and equipment and storage medium

Abstract

The invention discloses a data staining method, system and device and a storage medium, and relates to the technical field of data security, the method comprises the following steps: extracting statistical features of training data, and obtaining predefined staining features; using a feature generator to fuse the statistical features and predefined dyeing features to obtain fused features, and training a dyeing model based on the fused features so as to embed the predefined dyeing features into dyeing data generated by the dyeing model; and detecting whether the dyeing data carries the predefined dyeing characteristics or not through a traceability model so as to determine the source of the dyeing data. By training the dyeing model, the generated dyeing data keeps the characteristics of the original data, and the invisible statistical beacons are embedded, so that the security and traceability of the data are ensured. By detecting whether the dyeing data carries the predefined dyeing features, the source of the data can be accurately identified, and the authenticity and integrity of the data are ensured.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a data coloring method, system, device and storage medium. Background Art

[0002] With the rapid increase in data volume and the increasing popularity of data sharing, the security and traceability of structured data stored in various databases have become increasingly important. Structured data, such as JSON and tabular data, are widely used in many fields such as finance and medicine. In order to ensure the integrity and traceability of this data during use, data coloring technology has been proposed. By coloring structured data, invisible statistical beacons can be embedded in the data, and identification and traceability can be performed through specific technical means, providing reliable evidence for the source, flow and modification of the data. This technology can not only effectively prevent malicious tampering of structured data, but also provide guarantees for supervision and compliance during the use of structured data.

[0003] The current mainstream coloring technology is to use a generative watermarking method. By modifying the sampling process of LLM, a statistical feature is introduced when generating text, which can be identified in the detection stage. This technology mainly includes three components: random seed generator, sampling algorithm and scoring function. The random seed generator generates a random seed based on the previous text and the key. The sampling algorithm uses the seed to select the next token from the vocabulary distribution of LLM, and introduces a random watermark function (g-values) to influence the selection of tokens, thereby embedding a watermark in the generated text. The detection stage determines whether the text contains a watermark by calculating the g-values ​​score of the text.

[0004] However, generative watermarking technology has obvious defects and deficiencies when processing structured data. The generation of structured data is usually subject to strict constraints, and the range of data values ​​in each column is limited. For example, the payment method column may only allow two options. In this case, the range of options for generating the next token is very narrow, resulting in extremely limited space for watermark embedding. Generative watermarking relies on modifying the probability distribution of vocabulary. In structured data, due to the extremely small number of optional tokens, the embedding of watermarks may significantly affect the accuracy and consistency of the data, and even cause the generated data to fail to comply with business rules. In addition, generative watermarking technology usually assumes that there is high entropy (i.e., uncertainty in vocabulary distribution) in the text generation process in order to introduce watermarks during the sampling process. However, the generation of structured data often has low entropy, especially when the value range of some columns is very limited. Low entropy environments weaken the embedding effect of watermarks because the strength of watermarks depends on the model's selection of multiple candidate tokens during the generation process. If the model almost always chooses the same token when generating, watermark embedding will become extremely difficult or even impossible. In addition, generative watermarking technology is sensitive to data editing and modification. After being generated, structured data may undergo multiple processing and transformations, such as data cleaning, format conversion, or manual editing. These operations may destroy the statistical characteristics of the watermark and cause watermark detection failure. Summary of the invention

[0005] The main purpose of the present invention is to provide a data coloring method, system, device and storage medium. By training the coloring model, the generated coloring data is embedded with invisible statistical beacons while maintaining the characteristics of the original data to ensure the security and traceability of the data. By detecting whether the coloring data carries the predefined coloring features, the source of the data can be accurately identified to ensure the authenticity and integrity of the data.

[0006] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions: According to a first aspect of an embodiment of the present application, a data coloring method is provided, the method comprising: Extract statistical features of training data and obtain predefined staining features; Using a feature generator to fuse the statistical feature with a predefined staining feature to obtain a fused feature, and training a staining model based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model; Whether the dyeing data carries the predefined dyeing feature is detected through a traceability model to determine the source of the dyeing data.

[0007] Optionally, detecting whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data includes: Extracting the statistical features to be detected from the staining data; Calculating the similarity between the statistical feature to be detected and the predefined staining feature to obtain a similarity score; A judgment is made based on the similarity score and a preset threshold to obtain a judgment result; if the similarity score is greater than or equal to the preset threshold, it is judged that the dyeing data carries the predefined dyeing feature; if the similarity score is less than the preset threshold, it is judged that the dyeing data does not carry the predefined dyeing feature; According to the judgment result, the source of the staining data is determined.

[0008] Optionally, training a staining model based on the fusion feature includes: Inputting the fusion feature into the staining model to obtain a model output; Calculate the total loss between the model output and the training data according to a preset loss function; Calculating a gradient based on the total loss and updating the parameters of the dyeing model by back propagation; The above steps are repeated until the coloring model converges or a predetermined number of iterations is reached.

[0009] Optionally, the total loss between the model output and the training data is calculated according to a preset loss function according to the following formula: in, L data The loss of similarity between the model output and the training data, L watermark for the loss of extractability of the predefined staining features, L robust is the loss of robustness of the predefined staining feature to data operation; it is specifically calculated according to the following formula: Where G(x) is the model output, and x is the training data; Where E is a feature extractor for extracting the output from the model G ( x ) extracts feature information; w is a predefined staining feature; T represents the data transformation operation function.

[0010] Optionally, the statistical feature is fused with a predefined staining feature using a feature generator to obtain a fused feature, including: The statistical feature and the predefined coloring feature are input into the feature generator, so that the feature generator adjusts the occurrence frequency and / or order of the column values ​​of the statistical feature according to the predefined coloring feature to embed the predefined coloring feature into the statistical feature to obtain a fusion feature.

[0011] Optionally, embedding the predefined staining feature into the statistical feature comprises: Determining characteristic attributes of the statistical features, wherein the characteristic attributes include continuous characteristics and discrete characteristics; If the characteristic attribute of the statistical feature is a continuous feature, a set statistical mode is embedded in the numerical domain distribution of the statistical feature, and the set statistical mode is defined according to the following formula: P ( X ∈[ a , b ])= c in, a , b is a predefined interval, c is the characteristic probability; If the characteristic attribute of the statistical feature is a discrete feature, encoding the coloring feature in the value frequency of the statistical feature; The coloring features are embedded in the correlation between statistical features, where for the conditional probability model, the formula is used P ( X ∣ Y ) defines the association relationship.

[0012] Optionally, before extracting statistical features of the training data using the feature generator, the method further includes: Performing outlier detection and outlier processing on the training data, wherein the outlier processing method includes median filling and / or deletion; and / or, The training data is standardized so that data of different scales are converted to the same magnitude.

[0013] According to a second aspect of an embodiment of the present application, a data coloring system is provided, the system comprising: A feature preparation module is used to extract statistical features of training data and obtain predefined staining features; A model training module, used for fusing the statistical feature with a predefined staining feature using a feature generator to obtain a fused feature, and training a staining model based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model; The detection module is used to detect whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data.

[0014] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0015] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described in the first aspect above.

[0016] In summary, the embodiments of the present application provide a data coloring method, system, device and storage medium. By extracting the statistical features of the training data and obtaining the predefined coloring features; using a feature generator to fuse the statistical features with the predefined coloring features to obtain a fusion feature, and training a coloring model based on the fusion feature, so that the predefined coloring features are embedded in the coloring data generated by the coloring model; using a traceability model to detect whether the coloring data carries the predefined coloring features to determine the source of the coloring data. By training the coloring model, the generated coloring data is embedded with an invisible statistical beacon while maintaining the characteristics of the original data, ensuring the security and traceability of the data. By detecting whether the coloring data carries the predefined coloring features, the source of the data can be accurately identified to ensure the authenticity and integrity of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0018] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0019] Figure 1 A flow chart of a data coloring method provided in an embodiment of the present application; Figure 2 A dyeing flow chart provided for the embodiments of the present application; Figure 3 A schematic diagram of a data coloring system provided in an embodiment of the present application; Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application is shown; Figure 5 A diagram showing a computer-readable storage medium provided by an embodiment of the present application.

[0020] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0023] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0024] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0026] The present invention proposes a coloring technology for structured data, which achieves data security protection and traceability identification by embedding invisible statistical beacons in the data. This technical solution can effectively solve various technical problems that may be encountered in the process of data generation, editing, conversion and detection, and provide strong protection for the integrity, traceability and compliance of structured data. Compared with traditional methods, the innovative design of the present invention in the embedding and detection of invisible statistical beacons makes the protection of structured data more efficient and robust.

[0027] 1. In response to the problem of a narrow range of optional tokens in structured data, an embodiment of the present application introduces a rule-based invisible statistical beacon embedding mechanism. This mechanism designs an invisible statistical beacon embedding method that adapts to a limited value range by comprehensively considering the business rules and constraints of the data. By adjusting the frequency or order of occurrence of column values ​​and embedding statistical features, it is ensured that the embedding of invisible statistical beacon information does not affect the use value of the data while meeting the requirements of business rules. This method makes effective use of the space for embedding invisible statistical beacons in structured data, which not only maintains the original characteristics of the data, but also ensures the integrity and effectiveness of the invisible statistical beacons.

[0028] 2. In a low-entropy environment, the effect of watermark embedding is often poor, and traditional watermarking technology may not be able to effectively embed watermarks due to the small amount of information in the data. To this end, the embodiment of the present application proposes a context-based invisible statistical beacon enhancement technology. This method increases the embedding dimension of the invisible statistical beacon by utilizing the correlation between multiple columns in structured data, so that the invisible statistical beacon information can be distributed and crossed between multiple columns, thereby greatly improving the robustness of the invisible statistical beacon. Even in a low-entropy environment, the invisible statistical beacon information can exist in a more complex and hidden form, greatly enhancing the stability and anti-interference ability of the invisible statistical beacon.

[0029] 3. During the process of data editing and conversion, the watermark may be destroyed or lost, which is an important challenge faced by existing watermark technology. The embodiment of the present application designs an edit-resistant invisible statistical beacon algorithm to disperse and embed invisible statistical beacon information in multiple locations, making full use of the redundant characteristics of structured data. This method ensures that even if some parts of the data are modified or lost, the remaining invisible statistical beacon information can still be extracted and verified, thereby achieving higher invisible statistical beacon robustness. Through this redundant embedding strategy, the embodiment of the present application effectively reduces the impact of data editing and conversion on the integrity of the invisible statistical beacon, ensuring the reliability of the invisible statistical beacon in practical applications.

[0030] 4. Traditional generative watermarking technology usually has a strong dependence on the generation model, which limits the breadth and flexibility of its application. The embodiment of the present application breaks through this limitation and proposes a model-independent invisible statistical beacon embedding framework. This framework uses post-processing technology to independently embed invisible statistical beacons according to preset rules or statistical methods after generating data, without relying on specific generation models or tools. In this way, whether in different generation models or different application scenarios, the invisible statistical beacon embedding method of the embodiment of the present application can be flexibly adapted and has strong versatility and compatibility.

[0031] 5. The detection accuracy and efficiency of invisible statistical beacons are also an important issue in the application of data coloring technology. For the detection of invisible statistical beacons in structured data, the embodiment of the present application proposes a fast invisible statistical beacon detection method based on column features by optimizing the detection algorithm. This method quickly locates and verifies the invisible statistical beacon information in the data by deeply mining the column type, value range and other features of structured data. By reducing the computational complexity and combining it with an efficient matching algorithm, the embodiment of the present application greatly improves the detection efficiency while ensuring the detection accuracy, making the extraction and tracing process of invisible statistical beacons more convenient and efficient.

[0032] Figure 1 A data coloring method provided in an embodiment of the present application is shown, and the method includes: Step 101: extracting statistical features of training data and obtaining predefined staining features; Step 102: using a feature generator to fuse the statistical feature with a predefined staining feature to obtain a fused feature, and training a staining model based on the fused feature, so that the predefined staining feature is embedded in the staining data generated by the staining model; Step 103: Detect whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data.

[0033] In a possible implementation, before extracting statistical features of training data using a feature generator, the method further includes: The training data is subjected to outlier detection and outlier processing, wherein the outlier processing method includes median filling and / or deletion; and / or the training data is subjected to standardization processing so that data of different scales are converted to the same magnitude.

[0034] By detecting and processing outliers, the quality of training data can be improved and the consistency and reliability of data can be ensured. Outliers may have an adverse effect on model training, resulting in a decrease in model performance. By filling or deleting outliers with the median, the interference of outliers on model training can be reduced and the robustness of the model can be enhanced. By converting data of different scales to the same magnitude, the impact of data scale differences on model training can be eliminated, improving the stability and accuracy of the model.

[0035] In a possible implementation, in step 101, The statistical feature is fused with the predefined staining feature using a feature generator to obtain a fused feature, including: The statistical feature and the predefined coloring feature are input into the feature generator, so that the feature generator adjusts the occurrence frequency and / or order of the column values ​​of the statistical feature according to the predefined coloring feature to embed the predefined coloring feature into the statistical feature to obtain a fusion feature.

[0036] In a possible implementation, in step 101, embedding the predefined staining feature into the statistical feature includes: Determine the characteristic attribute of the statistical feature, the characteristic attribute includes a continuous feature and a discrete feature; if the characteristic attribute of the statistical feature is a continuous feature, embed a set statistical mode in the numerical domain distribution of the statistical feature, and the set statistical mode is defined according to the following formula: P ( X ∈[ a , b ])= c in, a , b is a predefined interval, c is the characteristic probability; If the characteristic attribute of the statistical feature is a discrete feature, the coloring feature is encoded in the value frequency of the statistical feature; the coloring feature is embedded in the correlation relationship between the statistical features, wherein, for the conditional probability model, the formula is used P ( X ∣ Y ) defines the association relationship.

[0037] By embedding set statistical patterns in the numerical domain distribution of continuous features, or encoding coloring features in the value frequency of discrete features, or embedding coloring features in the correlation between features, the validity and uniqueness of coloring features can be ensured, thereby improving data security and traceability.

[0038] In a possible implementation, in step 102, training a staining model based on the fusion feature includes: The fusion feature is input into the coloring model to obtain a model output; the total loss between the model output and the training data is calculated according to a preset loss function; the gradient is calculated according to the total loss, and the parameters of the coloring model are updated by back propagation; the above steps are repeated until the coloring model converges or a predetermined number of iterations is reached.

[0039] By training the coloring model, the generated coloring data embeds invisible statistical beacons while maintaining the characteristics of the original data, ensuring the security and traceability of the data. The total loss between the model output and the training data is calculated through a preset loss function to measure the performance of the model. The total loss includes data similarity loss, watermark extractability loss, and robustness loss. By combining these losses, the performance of the model can be comprehensively evaluated to ensure that the generated coloring data not only maintains the characteristics of the original data, but also contains reliable invisible statistical beacons. By continuously updating the model parameters, the model gradually reduces the total loss, improves the quality and reliability of the generated data, and ensures that the generated coloring data contains both predefined coloring features and maintains the characteristics of the original data.

[0040] In a possible implementation, in step 102, the total loss between the model output and the training data is calculated according to a preset loss function according to the following formula: in, L data The loss of similarity between the model output and the training data, L watermark for the loss of extractability of the predefined staining features, L robust is the loss of robustness of the predefined staining feature to data operation; it is specifically calculated according to the following formula: Wherein, G(x) is the model output, and x is the training data; Where E is a feature extractor for extracting G ( x) extracts feature information; w is a predefined staining feature; Wherein, T represents the data transformation operation function.

[0041] In a possible implementation, in step 103, detecting whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data includes: Extract the statistical feature to be detected from the staining data; calculate the similarity between the statistical feature to be detected and the predefined staining feature to obtain a similarity score; make a judgment based on the similarity score and a preset threshold to obtain a judgment result; if the similarity score is greater than or equal to the preset threshold, judge that the staining data carries the predefined staining feature; if the similarity score is less than the preset threshold, judge that the staining data does not carry the predefined staining feature; and determine the source of the staining data based on the judgment result.

[0042] By detecting whether the staining data carries the predefined staining features, the source of the data can be accurately identified to ensure the authenticity and integrity of the data. By extracting the statistical features to be detected, it can be ensured that the subsequent similarity calculation and judgment are based on accurate data features. Through similarity calculation, the degree of similarity between the data to be detected and the predefined staining features can be quantified. Through threshold judgment, it can be accurately distinguished whether the data carries the predefined staining features. By determining the source of the staining data, the authenticity and integrity of the data can be ensured.

[0043] Through the above steps, it is possible to accurately detect whether the staining data carries the predefined staining features, thereby determining the source of the data and ensuring the authenticity and integrity of the data.

[0044] Suppose an intelligent text generation model is used to automatically write a story. The model generates words that conform to grammatical and semantic rules based on the input content to construct a coherent and meaningful text. In traditional watermark generation technology, implicit information is usually embedded by adjusting the selection probability of each word in the generation process. This method is relatively simple and easy to detect. However, after the introduction of the invisible statistical beacon embedding mechanism, the watermark generation process has been significantly improved. Now, the embedding of watermarks not only takes into account the probability of each word, but also involves the optimization of the frequency and order of word occurrence. Specifically, when embedding watermark information, the model adjusts the generated content according to the set business rules (for example, specific words must appear a certain number of times or in a certain order).

[0045] For example, during the generation process, suppose that a particular word "beautiful" has an 80% probability of being generated in natural language, while the word "ugly" has a 20% probability. Through the embedding mechanism of invisible statistical beacons, we can fine-tune the frequency and order of these words. For example, you can set rules to ensure that the word "beautiful" appears twice in the generated paragraph, and each time it appears in a specific position, such as the beginning and end of the paragraph, to emphasize a certain theme or emotional color. "Ugly" only appears once, and may appear in the middle of the paragraph to serve as a contrast or foil. In this way, invisible statistical information (such as the frequency and order of "beautiful") becomes part of the watermark.

[0046] These fine-tuning of frequency and order do not affect the natural fluency or semantic accuracy of the text, because they follow specific business rules and constraints, ensuring that the text still conforms to language habits and business needs. At the same time, this method provides a more complex and effective watermark embedding mechanism, making the embedding of watermark information more difficult to detect and tamper with. Even if the generated text is analyzed later, the embedded information can only be extracted through specific rules and statistical methods. This rule-based invisible statistical beacon embedding method not only optimizes the embedding space of watermark information, but also effectively retains the original characteristics of the data and ensures the integrity and validity of the watermark information. It is particularly suitable for application scenarios that require data privacy and traceability. For example, in the fields of literary creation, news reporting, academic papers, etc., embedding watermarks in this way can protect the author's rights while ensuring the integrity and credibility of the text.

[0047] The method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0048] This technical solution mainly includes 3 parts: model training module, coloring module, and detection module. It includes data preprocessing, rule engine, statistical feature simulator, invisible statistical beacon embedding model, feature extractor and some other parts. The specific process is as follows Figure 2 .

[0049] The structured data first undergoes data preprocessing and then enters the model training phase. In the model training phase, the feature generator works with the model training module to generate a staining model. The staining model embeds staining features into the data to generate staining data. The staining data is tested through the traceability model and the test results are finally output. This flowchart shows the entire process from data preprocessing to the final test results, covering the three main modules of model training, staining, and testing.

[0050] Before data coloring, sufficient structured data and specific coloring features are required for model training. These features are like DNA markers of data. These custom coloring features will be sent to the feature generator and learned together with the preprocessed data. The feature generator will analyze the statistical characteristics of the data, and then fuse the statistical characteristics of the data with the predefined coloring features for model training. During the model training phase, the system will learn how to naturally embed these features into the data so that it can automatically carry invisible statistical beacons when generating new data. On the surface, the data colored by invisible statistical beacons is almost the same as the original data. For example, the basic statistical characteristics of transaction records remain unchanged, and the business logic is completely reasonable, but in fact they all carry unique statistical feature beacons, that is, anti-counterfeiting marks that are difficult to detect with the naked eye are added. When it is necessary to verify the source of data, the traceability model is used to analyze suspicious data and known coloring features at the same time. Using complex statistical analysis, the traceability model can determine whether the data carries the predefined invisible statistical beacon features, thereby determining whether the data comes from a trusted source. It not only ensures the availability of data, but also provides a strong guarantee for data security. This dyeing technology is like adding a unique fingerprint to the data, which protects data security while not affecting the normal use value of the data.

[0051] Phase 1: Data preprocessing: Structured data first passes through the data preprocessing module, which is responsible for converting the raw data into a standardized and normalized form, including steps such as data cleaning, standardization, and feature engineering.

[0052] The first aspect: application of data preprocessing module.

[0053] The data preprocessing module is the basic link of the structured data coloring system. Its core goal is to convert the raw data into a standardized and normalized form. From the technical principle point of view, this module mainly includes three key steps: data cleaning, standardization and feature engineering. First, the data cleaning process uses statistical methods to detect and process outliers. The Z-score method is usually used to identify outliers: Where X is the observed value, μ is the mean, and σ is the standard deviation. For the identified outliers, strategies such as median filling or deletion can be used. Secondly, the standardization process converts features of different scales to the same magnitude. Commonly used methods include Min-Max and Z-score standardization. Finally, the feature engineering link will perform feature selection and conversion, such as encoding categorical data and decomposing time features. The role of this module is to provide a high-quality data foundation for subsequent feature generation and model training. By reducing data noise, unifying data formats, and extracting effective features, it significantly improves the model training effect and the stability of invisible statistical beacon embedding.

[0054] Aspect 2: Definition of staining characteristics The coloring feature definition module is responsible for designing the specific form and embedding strategy of invisible statistical beacons (that is, coloring features). The role of this module is to provide basic invisible statistical beacon feature definitions for the entire coloring system. These features must ensure the effectiveness of invisible statistical beacons while maintaining the naturalness and usability of data.

[0055] Theoretically, good staining features must meet three conditions: imperceptibility (minimal impact on data distribution), robustness (strong anti-interference ability) and uniqueness (easy to identify and trace).

[0056] Feature design is usually based on statistical properties of the data, such as: For continuous features, specific statistical patterns can be embedded in their value distribution: Where [a,b] is a predefined interval and c is the feature probability. The formula indicates that the probability of a continuous feature X in the interval a,b is c. Here a,b are predefined intervals and c is the feature probability. This formula is used to describe how to embed a specific statistical pattern in the range of feature values ​​as an invisible statistical beacon.

[0057] For discrete features, invisible statistical beacon information can be encoded in their value frequencies. In addition, invisible statistical beacons can also be embedded in the correlation between features, such as designing specific conditional probability patterns: It represents the probability of feature X occurring under the condition that feature Y occurs, that is, the conditional probability. This formula is used to describe the association between features, and invisible statistical beacons can be embedded in this relationship.

[0058] It can be seen that the present application provides an invisible statistical beacon encoding technology based on statistical feature fusion. This technology innovatively encodes the invisible statistical beacon information into the multidimensional statistical features of the data, rather than simply modifying a single data value. This distributed invisible statistical beacon embedding method greatly improves the robustness of the invisible statistical beacon, enabling it to resist various data processing operations. An adaptive feature selection and invisible statistical beacon strength adjustment scheme is also provided. This scheme can automatically select the most suitable feature dimension for invisible statistical beacon embedding according to the characteristics of different types of data, and dynamically adjust the invisible statistical beacon strength. This intelligent adjustment mechanism ensures the optimal performance of the dyeing system in different application scenarios.

[0059] The second stage is the application of feature generator.

[0060] The feature generator is the core component for implementing invisible statistical beacons for structured data. Its principle is to automatically learn the statistical feature distribution of data through a deep learning model and integrate predefined coloring features into it. The coloring features mentioned here refer to invisible statistical beacons, which are embedded in the data to identify and track the data.

[0061] From a mathematical point of view, the feature generator can be expressed as a function: Where x is the input data and w is the coloring feature (invisible statistical beacon). The goal of this function is to generate new data while embedding the coloring feature into the data.

[0062] The optimization goal of the generator is to minimize the difference between the original data distribution P(x) and the generated data distribution Q(G(x,w)) while maintaining the identifiability of the invisible statistical beacon feature w. This can be expressed as an optimization problem: Where D represents the distribution distance metric, which is used to measure the difference between two distributions.

[0063] In specific implementation, the feature generator usually adopts an autoencoder structure, including an encoder E ( x ) and decoder D ( z , w ). The encoder E(x) extracts the intrinsic features of the data, and the decoder D(z,w) converts the extracted features z and invisible stat beacon information w Reconstruct to new data.

[0064] During the training process, the naturalness of the generated data and the concealment of the invisible statistical beacons are ensured through an adversarial learning strategy. This means that the generated data should be similar to the original data in appearance, while the embedded statistical beacons are not easily detected.

[0065] The role of the feature generator is to build a bridge between structured data features and invisible statistical beacon features, to achieve the natural embedding of invisible statistical beacon information while maintaining the statistical characteristics and business value of the data. In this way, even if the data is embedded with invisible statistical beacons, it can still be used for its original purpose, such as data analysis and business decision-making, without being affected by the embedded beacons.

[0066] An adaptive generation mechanism of coloring features based on deep learning. This technology automatically learns the statistical distribution characteristics of data through a feature generator, and integrates predefined invisible statistical beacon features into it, thus achieving natural embedding of invisible statistical beacons. This method avoids the direct modification of data structure by traditional invisible statistical beacon technology, significantly improving the concealment of invisible statistical beacons and the availability of data.

[0067] The third stage is model training.

[0068] The preprocessed data is fed into the feature generator together with the coloring features for feature learning. The feature generator analyzes the statistical characteristics of the data and fuses the statistical characteristics of the data with the predefined coloring features for model training. During the model training phase, the system learns how to naturally embed features into the data and generate data with invisible statistical beacons.

[0069] The model training module is responsible for optimizing the parameters of the entire dyeing system. Its core is to design and optimize the multi-objective loss function. The loss function consists of three parts: L data, L watermark and L robust, each part has its own specific purpose.

[0070] 1. In principle, the training process can be formalized as the following optimization problem: Total loss L It is composed of three parts, and the weight of each part is λ 1, λ 2, λ 3. Control.

[0071] 2. Data similarity loss: L_data measures the similarity between the generated data and the original data: Where G(x) is the generated data and x is the original data. G ( x ) and the original data x The square of the Euclidean distance is used to measure the difference between two data points.

[0072] 3. Loss of extractability of invisible statistical beacon information: L_watermark ensures the extractability of invisible statistical beacon information: Ensure invisible stat beacon information w where E is a feature extractor used to extract G (x ) is used to extract the coloring features. w is the coloring feature.

[0073] 4. Loss of robustness of invisible statistical beacons to data operations: L_robust ensures the robustness of invisible statistical beacons to data operations: T represents various data transformation operations (such as noise and compression). The formula ensures the robustness of the invisible statistical beacon to data operations (such as noise and compression).

[0074] The training adopts an iterative optimization strategy. Each iteration includes forward propagation to calculate the loss and back propagation to update the parameters. To improve the training effect, batch normalization and other techniques are usually used to stabilize the training process, and regularization methods such as dropout are used to prevent overfitting.

[0075] The role of this module is to enable the system to generate new data that not only maintains the characteristics of the original data but also contains reliable invisible statistical beacon information by continuously adjusting the model parameters. The training process needs to balance the data utility and the invisible statistical beacon strength, which is reflected in the selection of each weight λi of the loss function.

[0076] The embodiment of the present application provides a multi-objective optimization model training method. The method optimizes the three objectives of data feature preservation, invisible statistical beacon embedding strength and anti-interference ability through a carefully designed loss function. By dynamically adjusting the weight of each objective, the optimal balance between the invisible statistical beacon effect and data quality is achieved, ensuring the overall performance of the dyeing system.

[0077] Stage 4: Generation of dyeing modules.

[0078] After model training, a dyeing model is generated.

[0079] The coloring model embeds the coloring features into the data to generate coloring data.

[0080] In the fifth stage, the dyeing data is tested through the traceability model to verify the source of the data.

[0081] The traceability model analyzes suspicious data and known staining features, and uses statistical analysis to determine whether the data carries predefined invisible statistical beacon features. Based on the analysis of the traceability model, the test results are output to determine whether the data comes from a trusted source.

[0082] The working principle of the traceability model is to identify invisible statistical beacon features in the data through statistical tests and machine learning methods. Its core is to build a discriminator D(x) that outputs the probability that the data x contains the target invisible statistical beacon. From a statistical point of view, this can be regarded as a hypothesis testing problem: H0 (no invisible statistical beacon) vs. H1 (with invisible statistical beacon). The discrimination process first extracts the statistical feature vector f(x) of the suspicious data, and then calculates the similarity with the pre-stored invisible statistical beacon feature template w: s(x)=sim(f(x),w), where sim can be a metric such as cosine similarity. A judgment is made based on the score s(x) and the preset threshold τ: when s(x)>τ, it is determined that there is an invisible statistical beacon. To improve the reliability of detection, the model usually combines multiple features and detection methods and adopts an integrated learning strategy. The role of this module is to accurately identify the source of data and provide technical support for data security and intellectual property protection.

[0083] An intelligent traceability verification mechanism. This mechanism combines traditional statistical tests with modern machine learning methods, and achieves fast and accurate identification of invisible statistical beacons through multi-level feature extraction and pattern matching. At the same time, this mechanism can adapt to different types of data disturbances, ensuring the reliability of the verification results.

[0084] The technical effects of the technical solution of this application include the following aspects: 1. The dyeing is extremely concealed. Through the deep learning model, the invisible statistical beacon features are naturally integrated into the statistical distribution of the data, making the data with invisible statistical beacons almost indistinguishable from the original data in terms of apparent characteristics. Whether through manual observation or basic statistical analysis, it is difficult to find the invisible statistical beacon features in the data. This "stealth technique" greatly improves the security of invisible statistical beacons.

[0085] 2. Maintaining data availability. This technical solution fully maintains the business value and basic statistical characteristics of the data while embedding invisible statistical beacons. Through a carefully designed feature generator, it is ensured that the dyeing process does not change the core attributes and distribution characteristics of the data, so that data with invisible statistical beacons can be directly used for downstream tasks without affecting the accuracy and reliability of data analysis.

[0086] 3. Excellent robustness. The invisible statistical beacon features used in this solution have strong anti-interference capabilities. Even if the data undergoes common operations such as sampling, aggregation, and conversion, the invisible statistical beacon information can still remain intact. This strong robustness enables the invisible statistical beacon to continue to play a role in the actual use of data, providing long-term protection for data security.

[0087] 4. High verification efficiency. The traceability model can quickly and accurately identify invisible statistical beacon features in the data and support rapid verification of large-scale data. Through optimized feature extraction and matching algorithms, invisible statistical beacon detection can be completed in a relatively short time, which is of great significance for real-time verification needs in actual application scenarios.

[0088] 5. Strong universality. This technical solution can be applied to various types of structured data, including features of different forms such as numerical and categorical types. Through a flexible feature generation mechanism, the invisible statistical beacon embedding strategy can be automatically adjusted according to the characteristics of different data types, making the solution have a wide range of application scenarios.

[0089] 6. The system is highly scalable. The modular design of the solution makes it easy to expand and optimize. It can adjust feature generation strategies, update training methods, or optimize detection algorithms according to specific needs. This flexible architecture design enables the system to continuously adapt to new application scenarios and security requirements, and has good technical scalability.

[0090] In summary, the embodiment of the present application provides a data coloring method, which extracts statistical features of training data and obtains predefined coloring features; uses a feature generator to fuse the statistical features with the predefined coloring features to obtain fused features, and trains a coloring model based on the fused features so that the predefined coloring features are embedded in the coloring data generated by the coloring model; detects whether the coloring data carries the predefined coloring features through a traceability model to determine the source of the coloring data. By training the coloring model, the generated coloring data is embedded with invisible statistical beacons while maintaining the characteristics of the original data, ensuring the security and traceability of the data. By detecting whether the coloring data carries the predefined coloring features, the source of the data can be accurately identified to ensure the authenticity and integrity of the data.

[0091] Based on the same technical concept, the present application embodiment also provides a staining data detection system, such as Figure 3 As shown, the system comprises: The feature preparation module 301 is used to extract statistical features of training data and obtain predefined staining features; A model training module 302 is used to use a feature generator to fuse the statistical feature with a predefined staining feature to obtain a fused feature, and train a staining model based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model; The detection module 303 is used to detect whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data.

[0092] The present application also provides an electronic device corresponding to the method provided in the above embodiment. Figure 4 , which shows an electronic device diagram provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and the processor 200 executes the method provided by any of the aforementioned embodiments of the present application when running the computer program.

[0093] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one physical port (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0094] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.

[0095] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0096] The electronic device provided in the embodiment of the present application and the method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, operated or implemented by them.

[0097] The present application also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Figure 5 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided in any of the aforementioned embodiments is executed.

[0098] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A data coloring method, characterized in that: The method comprises: Extract statistical features of training data and obtain predefined staining features; Using a feature generator to fuse the statistical feature with a predefined staining feature to obtain a fused feature, and training a staining model based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model; The source tracing model is used to detect whether the dyeing data carries the predefined dyeing feature, so as to determine the source of the dyeing data.

2. The method according to claim 1, characterized in that Detecting whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data includes: Extracting features to be detected from the staining data; Calculating the similarity between the feature to be detected and the predefined staining feature to obtain a similarity score; A judgment is made based on the similarity score and a preset threshold to obtain a judgment result; if the similarity score is greater than or equal to the preset threshold, it is judged that the dyeing data carries the predefined dyeing feature; if the similarity score is less than the preset threshold, it is judged that the dyeing data does not carry the predefined dyeing feature; According to the judgment result, the source of the staining data is determined.

3. The method according to claim 1, characterized in that The staining model is trained based on the fusion feature, including: Inputting the fusion feature into the staining model to obtain a model output; Calculate the total loss between the model output and the training data according to a preset loss function; Calculating a gradient based on the total loss and updating the parameters of the dyeing model by back propagation; The above steps are repeated until the coloring model converges or a predetermined number of iterations is reached.

4. The method according to claim 3, characterized in that The total loss between the model output and the training data is calculated according to the preset loss function according to the following formula: in, L data The loss of similarity between the model output and the training data, L watermark for the loss of extractability of the predefined staining features, L robust The loss of robustness of the predefined staining feature to data manipulation; , , is a parameter, which is calculated according to the following formula: Wherein, G(x) is the model output, and x is the training data; Where E is a feature extractor for extracting G ( x ) extracts feature information; w is a predefined staining feature; Wherein, T represents the data transformation operation function.

5. The method according to claim 1, characterized in that The statistical feature is fused with the predefined staining feature using a feature generator to obtain a fused feature, including: The statistical feature and the predefined coloring feature are input into the feature generator, so that the feature generator adjusts the occurrence frequency and / or order of the column values ​​of the statistical feature according to the predefined coloring feature to embed the predefined coloring feature into the statistical feature to obtain a fusion feature.

6. The method according to claim 5, characterized in that The embedding of the predefined staining feature into the statistical feature comprises: Determining characteristic attributes of the statistical features, wherein the characteristic attributes include continuous characteristics and discrete characteristics; If the characteristic attribute of the statistical feature is a continuous feature, a set statistical mode is embedded in the numerical domain distribution of the statistical feature, and the set statistical mode is defined according to the following formula: P ( X ∈[ a , b ])= c in, a , b is a predefined interval, c is the characteristic probability; If the characteristic attribute of the statistical feature is a discrete feature, encoding the coloring feature in the value frequency of the statistical feature; The coloring features are embedded in the correlation between statistical features, where for the conditional probability model, the formula is used P ( X ∣ Y ) defines the association relationship.

7. The method according to claim 1, characterized in that Before extracting statistical features of the training data using the feature generator, the method further includes: Performing outlier detection and outlier processing on the training data, wherein the outlier processing method includes median filling and / or deletion; and / or, The training data is standardized so that data of different scales are converted to the same magnitude.

8. A data coloring system, characterized in that: The system comprises: A feature preparation module is used to extract statistical features of training data and obtain predefined staining features; A model training module, used for fusing the statistical feature with a predefined staining feature using a feature generator to obtain a fused feature, and training a staining model based on the fused feature so that the predefined staining feature is embedded in the staining data generated by the staining model; The detection module is used to detect whether the dyeing data carries the predefined dyeing feature through a traceability model to determine the source of the dyeing data.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1-7.

Images