Watermark generation method and system based on large language model

Abstract

The invention belongs to the field of artificial intelligence, and particularly relates to a watermark generation method and system based on a large language model, and the method comprises the steps: obtaining a training data set, and carrying out the preprocessing of the data in the training data set; presetting a watermark coding rule; training a large oracle model by adopting the preprocessed data according to a watermark coding rule; obtaining to-be-processed text data; cleaning and standardizing the text data; calculating a word frequency and an inverse document frequency of the standardized text data, and generating an initial vector; processing the initial vector by adopting a Word2Vec model to obtain a high-dimensional word vector; inputting the high-dimensional word vector into the trained neural network model, and extracting deep features; classifying the texts according to the deep features; inputting the classified questions into the trained large oracle model, and generating watermarks corresponding to the text types; according to the method, a watermark coding rule is designed, and the corresponding watermark is generated for the classified text, so that the safety of the text is improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and in particular relates to a watermark generation method and system based on a large language model. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models have become an important tool for processing natural language. Through deep learning technology, large models can generate highly coherent and semantically rich text output, which has greatly promoted the development of human-computer interaction and automated text generation. However, the widespread application of large language models also brings challenges to intellectual property protection. Traditional intellectual property protection methods, such as copyright statements and digital watermarking technology, face a series of problems. First, simple copyright statements in digital environments cannot effectively prevent illegal copying and distribution of content. Secondly, the application of existing digital watermarking technology in the text field is limited, and digital watermarking technology is currently generally designed for image or audio content. Finally, the watermark may be destroyed during the editing and format conversion of text content, reducing its effectiveness. Therefore, developing a new type of intelligent watermarking method that can not only meet the text quality of the output content of the large language model, but also provide effective intellectual property protection is an urgent problem to be solved in the current technical field. Summary of the invention

[0003] In order to solve the problems existing in the above prior art, the present invention proposes a watermark generation method based on a large language model, which includes: obtaining a training data set and preprocessing the data in the training data set; presetting a watermark encoding rule; training a large prediction model with the preprocessed data according to the watermark encoding rule; obtaining text data to be processed; cleaning and standardizing the text data; calculating the word frequency and inverse document frequency of the standardized text data to generate an initial vector; processing the initial vector with a Word2Vec model to obtain a high-dimensional word vector; inputting the high-dimensional word vector into a trained neural network model to extract deep features; classifying the text according to the deep features; inputting the classified questions into the trained large prediction model to generate a watermark of the corresponding text type.

[0004] A watermark generation system based on a large language model, the system comprising: a data acquisition module, a data preprocessing module, a large language model, a vectorization module, a feature extraction module, a classification module and an output module;

[0005] The data acquisition module is used to acquire text data;

[0006] The data preprocessing module is used to clean and standardize the text data; and input the standardized data into the vectorization module;

[0007] The vectorization module is used to convert text data into high-dimensional word vectors;

[0008] The feature extraction module is used to extract features from high-dimensional word vectors to obtain deep features;

[0009] The classification module is used to classify the deep features to obtain the corresponding category of the text;

[0010] The large language model is used to process the classification results and generate corresponding watermarks;

[0011] The output module is used for outputting text data with watermark.

[0012] Beneficial effects of the present invention:

[0013] The present invention designs a watermark encoding rule, and generates a corresponding watermark for the classified text according to the watermark encoding rule, thereby improving the security of the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic diagram of the intelligent watermark embedding process of the present invention;

[0015] Figure 2 A schematic diagram of the intelligent watermark detection process of the present invention;

[0016] Figure 3 A schematic diagram of the intelligent watermark system architecture of the present invention;

[0017] Figure 4 A schematic diagram of the text vectorization implementation process of the present invention;

[0018] Figure 5 It is a schematic diagram of the watermark autoregressive production process of the present invention;

[0019] Figure 6 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] A watermark generation method based on a large language model, such as Figure 6As shown, the method includes: obtaining a training data set and preprocessing the data in the training data set; presetting a watermark encoding rule; training a large prediction model using the preprocessed data according to the watermark encoding rule; obtaining text data to be processed; cleaning and standardizing the text data; calculating the word frequency and inverse document frequency of the standardized text data to generate an initial vector; using the Word2Vec model to process the initial vector to obtain a high-dimensional word vector; inputting the high-dimensional word vector into a trained neural network model to extract deep features; classifying the text according to the deep features; inputting the classified questions into the trained large prediction model to generate a watermark of the corresponding text type.

[0022] Example 1

[0023] An important part of the present invention is to use a hash algorithm to encrypt the text and generate a hash value as a "watermark". This hash value can be used as an identifier of the original text to verify the integrity and authenticity of the text. Specifically, the following steps are included:

[0024] Step S11: Data preprocessing: Clean and standardize the training data to ensure data quality and provide an accurate training environment for watermark embedding.

[0025] Data processing is performed using a new hypothesis testing method. Traditional hypothesis testing methods may not be able to effectively control type I errors when the sample size is large, that is, incorrectly rejecting the true null hypothesis. Here, an efficient statistical hypothesis testing method is provided, which can effectively control the probability of type I errors when the sample size tends to infinity, and provides a quantitative efficiency indicator. Specifically, it includes:

[0026] Define the probability of a Type I error: For a given hypothesis test, define the probability of a Type I error and compute its nth root supremum.

[0027] Calculate the average error rate: In the large model watermark generation, define the average error rate is the risk function of the watermark detector (h) on the probability distribution family (P). It measures the average error rate of the detector (h) when detecting the watermark:

[0028]

[0029] Among them, E0 and E 1,P They represent the expected values ​​under the null hypothesis and the alternative hypothesis, respectively, h is the detector (test statistic), and Y′ is the observed value. It means to optimize the parameters and find the lower bound, that is, to minimize the risk function. Here theta is a non-negative real parameter. The formula (θE0h) represents the expected loss of the detector (h) in the absence of watermark, multiplied by the parameter theta. Where E0 is the expected operator in the absence of watermark. φ P,h (θ) is a function of theta, which represents the maximum loss of the detector (h) under the probability distribution (P). The expression It means optimizing the probability distribution (P) and finding the supremum of (P), that is, maximizing the loss function. 1,P e -θh(Y′) It represents the expected value of the detector (h) output (Y′) when the watermark exists under the probability distribution (P), multiplied by e -θh(Y′) Here E 1,P is the expectation operator in the presence of watermark.

[0030] Estimating the probability of a Type I error: Based on the average error rate Evaluate the upper bound of the probability of a Type I error to ensure that the error rate is effectively controlled as the sample size increases. The Bonferroni method was used to ensure the accuracy and reliability of hypothesis testing.

[0031] In this embodiment, a hypothesis testing problem is set, including the null hypothesis H0 and the alternative hypothesis H1; and a suitable test statistic h and observation value Y′ are selected.

[0032] The specific process of calculating the average error rate is as follows: set a hypothesis test problem, including the null hypothesis H0 and the alternative hypothesis H1; select the appropriate test statistic h and observation value Y′; calculate the average error rate And evaluate the upper bound of the probability of Type I error; based on the calculated results, perform hypothesis testing and draw conclusions.

[0033] By introducing the average error rate indicator, the probability of type I error can be effectively controlled when the sample size is large, and the upper bound of the probability of type I error can be evaluated.

[0034] Step S12: Design of watermark encoding rules: Design a set of encoding rules. Here, the hash algorithm is used. It is a widely used cryptographic hash function that can convert an input of any length (a piece of text) into an output of a fixed length (hash value). Convert the watermark information into a format that can be embedded in text. The watermark can be a specific vocabulary, sentence structure or semantic pattern. Convert the input text into a byte string for hashing. Generate a unique hash value through the hash function, which can be used as a "watermark" for the original text.

[0035] The cross entropy loss will be caused during the processing, which is used to measure the distribution difference between the probability of the model after processing and the real text. Here, the loss function (such as cross entropy loss) that measures the difference between the probability distribution predicted by the model and the real label distribution is used, and the expression is:

[0036]

[0037] Where n is the number of categories in the classification task, that is, how many different labels or categories there are; X i is an indicator function, which usually takes a value of 0 or 1. In multi-classification problems, X i It can indicate whether the i-th sample belongs to category c. If sample i belongs to category c, then X i =1; otherwise X i =0.y o,c is the one-hot encoding (probability distribution) of the true label. In multi-classification problems, y o,c P represents the true probability that sample o belongs to category c. o,c is the probability distribution predicted by the model, indicating the probability that the model predicts that sample o belongs to category c. (log) is the natural logarithm function, which is used to calculate the logarithm of the probability. Therefore, for each sample o, the true label distribution y is calculated o,c The probability distribution P predicted by the model o,c The cross entropy loss between the two samples is then summed up. By calculating the cross entropy loss, we can compare it with the cross entropy of the original text to get the difference in quality between the processed text and the original text. The minus sign here is to ensure that the loss function is positive, because the range of the logarithmic function is from negative infinity to 0. The optimization process will minimize this loss function. This loss function is a convex function and can usually be minimized by gradient descent to train the model so that the model's predicted probability distribution is as close to the true label distribution as possible.

[0038] Step S13: Model training: In the training process of the large language model, watermark encoding rules are introduced to simulate the most likely events in a random process. In this way, we can approximately solve the original discrete optimization problem. The goal is to find the weight (w) that maximizes a specific objective function in a certain weight set (W).

[0039]

[0040] Among them, W is a discrete index set representing all possible values ​​of random variables, U w is some score associated with weight w. w This is the probability distribution function. For a given w, P wis the probability of the random variable taking this value, and the symbol ~ here indicates that it follows a certain distribution. P is the probability distribution of the weight (w), where P w represents the probability of weight (w).

[0041] Here, the Gumbel-max method is used to approximate the optimization problem of discrete random variables by using the properties of the Gumbel distribution. By introducing Gumbel noise, the originally difficult discrete optimization problem can be transformed into a continuous optimization problem, so that methods such as gradient descent can be used to solve it.

[0042] Step S14: Text vectorization is the process of converting text data into numerical vectors, which is very important for the text data processing of the present invention. The main text vectorization methods are Word2Vec and TF-IDF. The present invention uses a text vectorization system based on deep learning and traditional statistical methods, which can efficiently and accurately convert text data into numerical vectors and realize deep processing and analysis of text in various application scenarios.

[0043] A watermark generation system based on a large language model, such as Figures 1 to 3 As shown, the system includes: a data acquisition module, a data preprocessing module, a large language model, a vectorization module, a feature extraction module, a classification module and an output module;

[0044] The data acquisition module is used to acquire text data;

[0045] The data preprocessing module is used to clean and standardize the text data; and input the standardized data into the vectorization module;

[0046] The vectorization module is used to convert text data into high-dimensional word vectors;

[0047] The feature extraction module is used to extract features from high-dimensional word vectors to obtain deep features;

[0048] The classification module is used to classify the deep features to obtain the corresponding category of the text;

[0049] The large language model is used to process the classification results and generate corresponding watermarks;

[0050] The output module is used for outputting text data with watermark.

[0051] In this embodiment, the system includes: a preprocessing module, a vectorization module, an application module and a visualization module. The preprocessing module is used for text cleaning, word segmentation, and stop word removal. The vectorization module integrates a variety of vectorization technologies, including TF-IDF, Word2Vec, BERT Embeddings, and a text vectorization model based on a neural network. Application module: applies the vectorized text to tasks such as text classification, clustering, and information retrieval. Visualization module: provides text vector visualization function to help users intuitively understand text features.

[0052] In this embodiment, the specific process of the system processing data includes:

[0053] Step 1: Clean and standardize the input text through the preprocessing module.

[0054] Step 2: Use the TF-IDF algorithm in the vectorization module to calculate the word frequency and inverse document frequency of the text and generate the initial vector.

[0055] Step 3: Use Word2Vec or BERT Embeddings to further enrich the semantic information of the text and generate high-dimensional word vectors.

[0056] Step 4: Use the neural network-based text vectorization model to conduct deep learning on the text and extract deep features.

[0057] Step 5: Input the generated vector into the application module to perform specific NLP tasks.

[0058] Step 6: Display text vectors through the visualization module to provide intuitive text feature analysis.

[0059] In this embodiment, taking patent text data as an example, the text analysis process includes:

[0060] Step 1: Input the Chinese patent text into the preprocessing module for word segmentation and cleaning.

[0061] Step 2: Use the TF-IDF algorithm to calculate the word frequency and inverse document frequency and generate the initial vector. Word-level vectorization: Convert individual words into numerical vectors and assign each word a unique binary vector, where only one position is 1 and the rest are 0. Then generate word vectors or document vectors by counting word frequency and inverse document frequency. Generate vectors based on the frequency of n consecutive words, mapping each word to a high-dimensional real number vector that is semantically related.

[0062] Step 3: Use BERT Embeddings to extract deep semantic information from patent text.

[0063] Step 4: Input the vectorized text into the neural network-based classification model for automatic classification.

[0064] Step 5: Display the classification results through the visualization module to verify the accuracy of the model.

[0065] Example 2

[0066] An important part of the present invention is how to implement the intelligent watermark detection algorithm, which is further explained in this embodiment. Specifically, it includes: Topic extraction: Use the non-watermark large language model LLM to output or input text sequences to extract topics. Generate watermark list pairs: Generate a pair of green (G) and red (R) label lists based on the extracted topics. Use the generated G and R lists to adjust the label weights when the LLM generates text. Generate watermark text LLM generates text with watermarks based on the adjusted label weights. Watermark detection: Detect watermarks by comparing the distribution of G and R list labels in the target text.

[0067] In one embodiment, the watermark method of the present invention comprises the following steps:

[0068] a. Receive text prompts input by users;

[0069] b. Use non-watermark LLM to process input prompts and generate non-watermark text output;

[0070] c. Extract topics from non-watermarked text output;

[0071] d. Generate G and R list pairs based on the extracted topics;

[0072] e. Use the G and R lists to adjust the label weight of LLM and generate a hash value, i.e., the text with the watermark;

[0073] f. Provide the text output with the watermark to the user.

[0074] In addition, the present invention also relates to natural language processing technology, in particular to an autoregressive language model for text generation. An efficient text generation method of an autoregressive language model is provided, which improves the efficiency and quality of text generation by dynamically adjusting the vocabulary size and optimizing the conditional probability distribution.

[0075] Vocabulary definition: Define a dynamically adjustable vocabulary W containing K labels, and the vocabulary size K is adjusted according to the model requirements.

[0076] Vocabulary size adjustment: Dynamically adjust the vocabulary size K according to the complexity of the model and the application scenario to adapt to different model requirements.

[0077] Autoregressive generation process: Using the autoregressive mechanism, each label in the text is generated sequentially by sampling from a conditioned multinomial probability distribution.

[0078] Conditional probability distribution optimization: Optimize the conditional probability distribution P t , so that it not only depends on the previous t-1 labels (the current label is denoted as t), but also considers the prompt information inside the model to improve the accuracy and relevance of the generated text.

[0079] Text Generation Control: Fine-grained control over the text generation process is achieved by controlling the conditional probability distribution and vocabulary size to generate high-quality and diverse text.

[0080] The following is a specific embodiment to illustrate the application of the method of the present invention:

[0081] For an OPT-1.3B model with K=50,272 labels, high-quality text is generated by adjusting the vocabulary size and optimizing the conditional probability distribution.

[0082] For an LLaMA-7B model with K=32,000 labels, the method of the present invention is applied to achieve efficient and accurate text generation.

[0083] The present invention significantly improves the efficiency and quality of text generation by dynamically adjusting the vocabulary size and optimizing the conditional probability distribution, and is applicable to various natural language processing fields.

[0084] Example 3

[0085] An important component of the present invention is the development of an intelligent watermark system, which is further explained in this embodiment.

[0086] Step 301: Design a system including text generation, watermark embedding, watermark detection and user interface.

[0087] Step 302: Develop a simple user interface that allows users to upload text, set watermark features, and start the watermark embedding and detection process.

[0088] Step 303: Integrate the watermark embedding and detection algorithm into the system to achieve one-click operation.

[0089] Step 304: Perform system testing, including unit testing, integration testing, and user acceptance testing, to ensure the stability and usability of the system.

[0090] Example of step 305: A user uploads a piece of text through the interface, selects "Enable Watermark", and the system automatically embeds a watermark in the article and provides a detection report.

[0091] Example 4

[0092] An important part of the present invention is the multi-language and multi-domain adaptability of intelligent watermarks. Figure 4 As shown, this embodiment further illustrates this.

[0093] Step 401: designing an independent watermark feature library for each language, and training the corresponding watermark embedding and detection models;

[0094] Step 402: Analyze text features in different fields and adjust watermark embedding strategies to adapt to professional terms and expressions;

[0095] Step 403: In the medical field, specific medical terms are used as watermark features; in the literary field, specific literary rhetoric is used as watermark features.

[0096] Example 5

[0097] An important part of the present invention is the red-green list principle of intelligent watermarking. Figure 5 As shown, this embodiment further illustrates this.

[0098] Step 501: The proposed topic-based detection framework is built on the G and R label list pair method, but introduces a novel mechanism to classify text as human or LLM generated through topic extraction. The method involves extracting and analyzing the topics present in the non-watermark LLM output sequence. The detection process creates a specific G and R list pair for each identified topic, allowing the detection mechanism to traverse all list pairs for each topic to achieve accurate classification. After receiving the non-watermark LLM output sequence, the detection framework uses topic modeling technology. A test parameter z is used to evaluate the proportion of G in the list. All tags in the text are written as T, G represents G list pairs, and the proportion of G list pairs is y. Then the z value can be expressed as:

[0099]

[0100] By comparing the value of z using the red-green list method, it can be determined whether a watermark exists.

[0101] The unbiased decoder can be used to determine whether a watermark exists and whether it has been attacked:

[0102]

[0103] The decoder S represents a function or a process for recovering the original information from some encoded or encrypted information.

[0104] In watermarking or encrypted communication, the decoder is used to extract the original watermark or message from the encoded information. w represents an element in the watermark set, that is, a specific watermark. u() represents the probability distribution function of the random variable ζ. Regardless of the value of ζ (as long as it follows a uniform distribution), the probability of the decoder S decoding the watermark w is consistent with the probability distribution of w in the watermark set. This means that the decoder will not be affected by ζ, thus ensuring the fairness of the decoding process.

[0105] Step 502: Encrypt the "result" phrase using an encryption algorithm to generate a KGW. The enhancements include improved list partitioning, better logic processing, increased watermark information capacity, and the ability to resist watermark deletion attacks and publicly detect watermarks. These improvements are intended to improve the algorithm's impact on text quality in low entropy situations and increase the information capacity of the watermark.

[0106] Here, an unbiased watermark decoding method based on probability decision is used to improve the accuracy and reliability of watermark technology. This method ensures the fairness and effectiveness of the watermark decoding process by introducing a probability model and a statistical detection mechanism.

[0107] 1. Watermark definition: Set a binary watermark set W = {0, 1} and the associated probability distribution P = (P0, P1), where P0 and P1 represent the probabilities of the watermark values ​​being 0 and 1, respectively.

[0108] 2. Decoder design: At the current label t, according to the current probability distribution P and signal S, by comparing the threshold C and P {t,0} To determine the watermark value w if C t ≤P t,0 , then w=0; otherwise w=1.

[0109] 3. Optimization of embedded signal: By adjusting the size of embedded signal C, the watermark value can be controlled to tend to 1 rather than 0, thereby optimizing the embedding effect of the watermark.

[0110] 4. Unbiasedness guarantee: Ensure that the decoding process meets the unbiasedness condition, that is, P(S(P,w)=w)=P w , where S(P,w) represents the decoding process and w represents the actual watermark value.

[0111] The following is a specific embodiment for illustrating the application of the method of the present invention: set the watermark set W = {0, 1} and the probability distribution P = (0.5, 0.5); at time t, determine w according to P and S; adjust C to optimize the embedding effect of w.

[0112] This method can improve the accuracy and fairness of watermark decoding, while ensuring the concealment and security of the watermark. In addition, this method is highly adaptable and flexible and is suitable for various digital media and communication scenarios.

[0113] Example 6

[0114] The legal and ethical issues of smart watermarking need to be considered during implementation:

[0115] Step 1: During the watermark design and implementation process, ensure that user privacy is not violated and data protection regulations are not violated; here, a watermark autoregressive text generation method is used, which can embed a watermark while generating text to enhance the security and verifiability of the text.

[0116] 1. Define prefix and vocabulary: Set a vocabulary and a starting prefix for text generation. The prefix consists of a series of tags.

[0117] 2. Generate the probability distribution of the next tag: Based on the prefix and vocabulary, a large language model (LLM) is used to generate the conditional probability distribution of the next tag.

[0118] 3. Calculate the watermark: Calculate the watermark based on a specific part of the text sequence and a key through a pseudo-random number generation function.

[0119] 4. Embed watermark: The calculated watermark is embedded into the text generation process to ensure the security and copyright protection of the text.

[0120] 5. Text generation: Generate the next label of the text based on the conditional probability distribution and the embedded watermark.

[0121] Step 2: Clearly explain to users the purpose and method of using the watermark and the user’s rights;

[0122] Step 3: Provide detailed instructions for using the watermark on the user interface, including an option for the user to consent to the use of the watermark.

[0123] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A watermark generation method based on a large language model, characterized in that: include: Obtain a training data set and preprocess the data in the training data set; Preset watermark encoding rules; use preprocessed data to train the large prediction model according to the watermark encoding rules; Obtain text data to be processed; clean and standardize the text data; Calculate the word frequency and inverse document frequency of the standardized text data to generate the initial vector; use the Word2Vec model to process the initial vector to obtain a high-dimensional word vector; input the high-dimensional word vector into the trained neural network model to extract deep features; Classify the text according to the deep features; input the classified questions into the trained big prediction model to generate the watermark of the corresponding text type.

2. The watermark generation method based on a large language model according to claim 1, characterized in that: Data preprocessing includes: cleaning and standardizing the data, and optimizing the standardized data using the optimized hypothesis testing method.

3. The watermark generation method based on a large language model according to claim 2, characterized in that: The optimized hypothesis testing method is used to optimize the standardized data, including: for a given hypothesis test, defining the probability of type I error and calculating the supremum of its nth root; defining the average error rate, determining the size of the average error rate, and when the average error rate is 0, using a special processing mechanism to process the data; when the average error rate is not 0, evaluating the upper bound of the probability of type I error based on the average error rate.

4. The watermark generation method based on a large language model according to claim 3, characterized in that: The data are processed using special processing mechanisms, including: deleting records with missing values, filling with mean or median, or processing data using interpolation methods, and sampling the processed data using the SMOTE algorithm to increase the samples of minority categories.

5. The watermark generation method based on a large language model according to claim 3, characterized in that: Estimating an upper bound on the probability of a type I error based on the average error rate involves setting the significance level to 0.05 or 0.01, placing an upper bound on the probability of rejecting the null hypothesis when it is true, and controlling for the error rate by adjusting the significance level using the Bonferroni method.

6. The watermark generation method based on a large language model according to claim 1, characterized in that: The watermark encoding rule is: obtain the watermark information, convert the watermark information into a format that can be embedded in text; convert the converted watermark information into a string; use the hash algorithm to encrypt the string, generate a hash value, and use the hash value as the watermark of the original text.

7. The watermark generation method based on a large language model according to claim 1, characterized in that: The large prediction model is trained using preprocessed data according to the watermark encoding rules, including: selecting Transformer as the model structure and performing distributed training on the model, that is, using a parallel strategy to input the training data into the model for training, where the parallel strategies include data parallelism, tensor parallelism, and pipeline parallelism; selecting the Adam optimizer to optimize the model parameters during model training, using Next Token Prediction as a pre-training task, and fine-tuning through supervised learning and reinforcement learning methods after the pre-training is completed; completing the model training.

8. The watermark generation method based on a large language model according to claim 1, characterized in that: Generating the initial vector includes: converting each word in the text into a numerical vector, that is, assigning a unique binary vector to each word, in which only one position is 1 and the rest are 0; generating a word vector by counting word frequency and inverse document frequency; generating a vector based on the frequency of n consecutive words, and mapping each word to a high-dimensional real number vector, that is, the high-dimensional real number vector is the initial vector.

9. The watermark generation method based on a large language model according to claim 1, characterized in that: Training the neural network model includes: the neural network model adopts the Transformer neural network architecture; the data set is divided into a training set and a test set, the number of layers and the number of neurons in each layer are defined, and the activation function and the loss function are selected; after calculating the predicted value, the loss function is used to calculate the difference between the predicted value and the true value, the gradient is calculated according to the loss value, back propagation is performed, and finally the optimizer is used to update the network parameters; the trained neural network model is tested with the test set to complete the model training.

10. A watermark generation system based on a large language model, the system being used to execute the watermark generation method based on a large language model as claimed in any one of claims 1 to 9, characterized in that: include: Data acquisition module, data preprocessing module, large language model, vectorization module, feature extraction module, classification module and output module; The data acquisition module is used to acquire text data; The data preprocessing module is used to clean and standardize the text data; and input the standardized data into the vectorization module; The vectorization module is used to convert text data into high-dimensional word vectors; The feature extraction module is used to extract features from high-dimensional word vectors to obtain deep features; The classification module is used to classify the deep features to obtain the corresponding category of the text; The large language model is used to process the classification results and generate corresponding watermarks; The output module is used for outputting text data with watermark.

Images