Text generation model training method with back-editing concept erasing function

Abstract

The text-to-image model training method with the concept erasing function of the anti-editing concept erasing function belongs to the concept erasing field of the text-to-image model. The problem that the traditional concept erasing method cannot prevent the user from generating an image containing the target concept and meeting the user's requirements through editing is solved. The unconditional noise predicted by the fine-tuned text-to-image model constructed by the present application is calculated with prior guidance, the erasing guidance is introduced into the image inference process, and the input text is not limited. In the training stage, the loss function loss function and loss are constructed to obtain the total loss, the conditional noise and the unconditional noise are fine-tuned, so that the concept erasing can be realized under the condition that the user input text does not contain target concept information, and the effect of preventing the generation of target concept image in generation and editing is realized. The present application is mainly used in the field of text-to-image, and the concept erasing can be realized under the condition that the target concept information is not contained.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of concept erasure of text-to-image model. BACKGROUND

[0002] The concept erasure of text-to-image model aims to accurately remove the target concept while faithfully preserving the generation of non-target concepts, which is specifically manifested as that for any input text, the images generated by the model do not contain the target concept, while the images generated by the non-target concept remain unchanged. During the training process of the text-to-image model, data containing copyrights and unsafe data will be used, which may lead to the generation of infringing content in the images generated by the model. In order to prevent this phenomenon, the concept erasure research of text-to-image model has become a research hotspot.

[0003] In the past research, the concept erasure method based on fine-tuning model has gradually developed, which realizes concept erasure by fine-tuning the noise prediction of the model on the target concept text. The editing technology using text-to-image model has gradually matured, and the development of this technology has led users to use images containing target concepts and editing texts to achieve the purpose of generating new images. Although the erasure method in the past can effectively prevent the generation of images containing unsafe content given the text, image editing can bypass their erasure method to generate images containing target concepts and meeting the requirements of users. This is because the erasure method in the past needs the input text to contain the target concept to produce effect, while the editing text may not contain the target concept, and the original image input also provides information of the target concept. Therefore, there is currently a need for a counter-editing concept erasure method that can filter the target concept during editing to prevent unsafe content from being generated in editing and generation. SUMMARY

[0004] The purpose of the present application is to solve the problem that the traditional concept erasure method cannot prevent users from generating images containing target concepts and meeting user requirements through editing, and to provide a text-to-image model training method with a counter-editing concept erasure function.

[0005] The text-to-image model training method with a counter-editing concept erasure function comprises:

[0006] (I) Data preparation stage:

[0007] The text content to be erased is taken as the target concept text c;

[0008] The large language model outputs a plurality of natural language texts according to the user question and the answer example, each natural language text is taken as a priori concept text, and all natural language texts constitute a priori concept set;

[0009] The user question comprises a user requirement, the target concept text c and an output quantity;

[0010] (ii) Model construction phase:

[0011] adding an LORA layer after each attention layer in the noise prediction network of the pre-trained text-to-image model, and the attention layer and the added LORA layer are adjacent, to generate a fine-tuned text-to-image model; wherein the category of each attention layer is the same as the category of the LORA layer corresponding to the attention layer;

[0012] (iii) Model training phase:

[0013] The specific process of training the fine-tuned text-to-image model once is as follows:

[0014] First, the fine-tuned text-to-image model under frozen parameters predicts the image latent space representation z T t steps away from the input original noise z t , empty text φ and target concept text c and random time step t.

[0015] Second, the fine-tuned text-to-image model under non-frozen parameters predicts the target concept noise, prior concept noise and unconditional noise under non-frozen parameters according to the image latent space representation z t , N prior concept texts, empty text φ and target concept text c.

[0016] The pre-trained text-to-image model under frozen parameters predicts the target concept noise, prior concept noise and unconditional noise under frozen parameters according to the image latent space representation z t , N prior concept texts, empty text φ and target concept text c.

[0017] According to the target concept noise, prior concept noise and unconditional noise under frozen and non-frozen parameters, a conditional predicted noise alignment loss function an unconditional predicted noise alignment loss function and a prior concept noise constraint loss

[0018] Finally, according to and construct a total loss use the total loss to update the LORA layer parameters of the fine-tuned text-to-image model under non-frozen parameters once by gradient backpropagation.

[0019] The fine-tuned text-to-image model is trained multiple times until the termination condition is met, and the training of the fine-tuned text-to-image model is completed.

[0020] Preferably, the implementation of the conditional predicted noise alignment loss function includes:

[0021] First, according to and Calculate the conditional prediction noise erasure guide

[0022] wherein, is the predicted unconditional noise under the frozen parameters, is the predicted target concept noise under the frozen parameters;

[0023] Second, according to and ∈ θ (z t ,c,t), construct a conditional prediction noise alignment loss function and

[0024]

[0025] wherein, ∈ θ (z t ,c,t) is the predicted target concept noise under the non-frozen parameters, is the Monte Carlo simulation expectation under the target concept, and ‖·‖2 is the calculation L2 norm.

[0026] Preferably, wherein, η c is the conditional noise concept erasure strength.

[0027] Preferably, the unconditional prediction noise alignment loss function The implementation manner includes:

[0028] First, according to and Calculate the unconditional prediction noise erasure guide under the prior guide

[0029] wherein, is the predicted unconditional noise under the frozen parameters, is the predicted target concept noise under the frozen parameters, is the predicted i-th prior concept noise under the frozen parameters, and N≥1;

[0030] Second, according to and ∈ θ (z t ,t), construct an unconditional prediction noise alignment loss function and

[0031]

[0032] wherein, ∈ θ (z tt) is the predicted unconditional noise under the unfreezed parameters, is the Monte Carlo simulation expectation under the target concept, and ‖·‖2 is the calculated L2 norm.

[0033] Preferably,

[0034] wherein η u represents the unconditional noise erasing strength, η p represents the prior guiding strength, represents the relevance degree of the i-th prior concept text to the target concept text c.

[0035] Preferably, represents the cosine similarity between x and c after encoding them into vectors; CLIP(x, c) represents the cosine similarity between x and c after encoding them into vectors; x is the target concept image, is the i-th prior concept text.

[0036] Preferably, the prior concept noise constraint loss is implemented in the following manner:

[0037]

[0038] wherein, is the Monte Carlo simulation expectation under the prior concept, represents the set of prior concepts obtained by the large language model, is the predicted i-th prior concept noise under the unfreezed parameters, is the predicted i-th prior concept noise under the frozen parameters, and N≥1.

[0039] Preferably,

[0040] wherein λ PUnc , λ Cons , and λ ESD are the first to third preset loss function weights, respectively.

[0041] Preferably, the LORA layer parameters of the fine-tuned text-to-image model under the unfreezed parameters are updated using the AdamW optimization algorithm.

[0042] Preferably, the large language model outputs a natural language text according to the user question and the answer example in the following manner:

[0043] ​Initialization stage: input the user question and answer sample according to the question and answer template to the large language model, and the large language model proposes the user requirement to obtain the preliminary answer according to the question template;

[0044] Self-feedback stage: the large language model evaluates the current preliminary answer according to the answer sample and user requirement;

[0045] Modification stage: modify the current preliminary answer according to the self-evaluation result and answer sample;

[0046] After the self-feedback stage and the modification stage are repeated for a predetermined number of times, the modification stage outputs the final answer, which is a natural language text and also a priori concept text.

[0047] Advantages of the present application:

[0048] The text-to-image model training method with the concept erasing function of the present application first constructs a fine-tuned text-to-image model, aligns three aspects according to the target concept noise, prior concept noise and unconditional noise predicted by the model under the frozen and non-frozen parameters, and constructs a conditional prediction noise alignment loss function Unconditional prediction noise alignment loss function And prior concept noise constraint loss

[0049] After the unconditional noise and prior guidance calculated by the fine-tuned text-to-image model are introduced into the image inference process, and are not limited by the input text.

[0050] At the same time, the unconditional prediction noise alignment loss function Is to train the noise on the hidden vector containing the target concept text, so it will be more targeted to the target concept text c (i.e. the text content to be erased) for erasing. Prior concept noise constraint loss Will ensure that the erasing effect is not affected by overfitting, and the prior guidance will reduce the impact of erasing training on other concept generation and editing, improve the prediction accuracy of the fine-tuned text-to-image model, and generate more accurate images that meet the user's requirements.

[0051] The present application fine-tunes both the conditional noise (i.e. target concept noise) and the unconditional noise to achieve the effect of concept erasing even when the user input text does not contain target concept information, and to prevent the generation of target concept images in generation and editing.

[0052] When the trained text-to-image model is used for text-to-image prediction, only the LORA layer trained needs to be inserted into the original model (i.e., the original pre-trained text-to-image model) to obtain a fine-tuned model, and then the text describing the content of the picture is input, so as to generate the corresponding picture. In this way, the user can input text without containing target concept information to achieve concept erasure, and thus prevent the generation of target concept images in the generation and editing process. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 The principle schematic diagram of the text-to-image model training method with anti-editing concept erasure function.

[0054] Figure 2 The prior concept schematic diagram obtained by using a large language model. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0056] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0057] The method of the present application is based on classifier-free computation, and the mathematical form of classifier-free guidance (CFG) is as follows:

[0058]

[0059] wherein, ∈ θ (·) represents the noise predicted by the noise prediction network in the text-to-image model, θ represents the network parameters, z t represents the image hidden space representation after t steps of denoising, c represents the embedding representation of the input text, and ω represents the classifier-free guidance strength.

[0060] Specific implementation, see Figure 1 In this embodiment, the text-to-image model training method with anti-editing concept erasure function comprises the following steps:

[0061] (I) Data preparation stage:

[0062] The text content to be erased is taken as the target concept text c;

[0063] The large language model outputs a plurality of natural language texts according to the user question and the answer sample, each natural language text is a priori concept text, and all the natural language texts constitute a priori concept set;

[0064] The user question includes a user requirement, a target concept text c, and an output quantity;

[0065] (ii) Model construction stage:

[0066] A LORA layer is added after each attention layer in the noise prediction network of the original pre-training text-to-image model, and the attention layer and the added LORA layer are adjacent, to generate a fine-tuned text-to-image model; wherein the category of the attention layer is the same as the category of the LORA layer corresponding to the attention layer;

[0067] (iii) Model training stage:

[0068] The specific process of training the fine-tuned text-to-image model once is as follows:

[0069] First, the fine-tuned text-to-image model under frozen parameters predicts the image latent space representation z T under the t-th step according to the input original noise z t , the empty text φ, the target concept text c, and the random time step t;

[0070] Second, the fine-tuned text-to-image model under non-frozen parameters predicts the target concept noise, the priori concept noise, and the unconditional noise under non-frozen parameters according to the image latent space representation z t , N priori concept texts, the empty text φ, and the target concept text c;

[0071] The original pre-training text-to-image model under frozen parameters predicts the target concept noise, the priori concept noise, and the unconditional noise under frozen parameters according to the image latent space representation z t , N priori concept texts, the empty text φ, and the target concept text c;

[0072] According to the target concept noise, the priori concept noise, and the unconditional noise under frozen and non-frozen parameters, a conditional prediction noise alignment loss function L an unconditional prediction noise alignment loss function L , and a priori concept noise constraint loss L

[0073] Finally, according to L and L , a total loss L is constructed; The total loss L is used to update the LORA layer parameters of the fine-tuned text-to-image model under non-frozen parameters once by gradient backpropagation.

[0074] The fine-tuned text-to-image model is trained multiple times until a termination condition is met, and the training of the fine-tuned text-to-image model is completed.

[0075] In this embodiment, the target concept noise, the prior concept noise and the unconditional noise predicted by the model are aligned in three aspects, and a conditional prediction noise alignment loss function is constructed The unconditional prediction noise alignment loss function and the prior concept noise constraint loss

[0076] After the unconditional noise predicted by the constructed fine-tuned text-to-image model and the prior guidance calculation, the erasing guidance is introduced into the image inference process and is not limited by the input text. At the same time, the unconditional prediction noise alignment loss function is trained on the hidden vector containing the target concept text, so it will be more targeted to the target concept text c (i.e. the text content to be erased) for erasing. The prior concept noise constraint loss will ensure that the erasing effect is not affected by overfitting, and the prior guidance will reduce the impact of erasing training on the generation and editing of other concepts, improve the prediction accuracy of the fine-tuned text-to-image model, and generate more accurate images that meet the user's requirements.

[0077] When using the trained text-to-image model for text-to-image prediction, only the LORA layer obtained by training needs to be inserted into the original model (i.e. the original pre-trained text-to-image model) to obtain the fine-tuned model, and then the text describing the picture content is inputted, and the corresponding picture can be generated. To achieve the effect of concept erasing even if the input text does not contain target concept information, and to prevent the generation of target concept images in the generation and editing process.

[0078] In a specific application, the text-to-image model is Stable Diffusion, and the noise prediction network used in the text-to-image model is the UNet network in Stable Diffusion v1.4, which predicts the target concept noise, prior concept noise and unconditional noise. The training object is to add a LORA layer after each attention layer of the UNet. Specifically, the UNet network includes two types of attention layers, and the two types of attention layers are classified as linear layers and convolutional layers, so the two types of LORA layers added are linear layers and convolutional layers. The LORA in the linear layer is a two-layer structure, and the linear layer 1 reduces the dimension of the input to the rank of the LORA, and the linear layer 2 increases the dimension of the input to the original dimension. The convolutional layer LORA also includes a two-layer structure. Convolution layer 1 reduces the number of input channels to the rank of the LORA, and the convolution is the same as the preposed convolution layer except for the input and output channel numbers. Convolution layer 2 increases the number of input channels to the original number of channels, and the convolution kernel size is 1*1 and the step is 1.

[0079] Referring to Figure 1 , the conditional predicted noise alignment loss function is constructed

[0080] First, the conditional predicted noise erasing guide and are calculated

[0081] wherein, is the predicted unconditional noise under the frozen parameters, is the predicted target concept noise under the frozen parameters.

[0082] Secondly, the conditional predicted noise alignment loss function is constructed according to θ (z t ,c,t) and

[0083]

[0084] wherein, ∈ θ (z t ,c,t) is the predicted target concept noise under the non-frozen parameters, is the Monte Carlo simulation expectation under the target concept, and ‖·‖2 is the calculation of L2 norm.

[0085] In the preferred embodiment, the conditional predicted noise erasing guide (CEG) inversely utilizes the non-classifier guided constructed alignment noise, and the constructed conditional predicted noise alignment loss function ​The introduction of the target concept prediction noise erasing guide is realized, and the erasing effect is stronger when the target concept is contained in the text.

[0086] Specifically, η c is the conditional noise concept erasing strength. In the expression of the conditional prediction noise erasing guide , z t is inferred from the Gaussian noise z T under the condition of the target concept text by the noise prediction network in training, and T-t steps are obtained. In this process, the model calculation does not participate in the gradient back propagation, and the advantage is that there is no need to prepare training data in advance.

[0087] Further, referring to Figure 1 the implementation of the unconditional prediction noise alignment loss function includes:

[0088] First, according to and , the unconditional prediction noise erasing guide under the prior guide is calculated

[0089] wherein, is the predicted unconditional noise under the frozen parameters, is the predicted target concept noise under the frozen parameters, is the predicted i-th prior concept noise under the frozen parameters, and N≥1 represents the number of prior concept sampling;

[0090] Second, according to and ∈ θ (z t ,t), the unconditional prediction noise alignment loss function is constructed

[0091]

[0092] wherein, ∈ θ (z t ,t) is the predicted unconditional noise under the non-frozen parameters, is the Monte Carlo simulation expectation under the target concept, and ‖·‖2 is the calculation L2 norm.

[0093] In the preferred embodiment, the predicted unconditional noise is aligned with the unconditional prediction noise erasing guide under the prior guide The erasing guide is introduced into the noise prediction containing the target concept image hidden space representation, and the prior guide in the noise is aligned to reduce the influence of the erasing on the prior concept noise prediction, and the prediction accuracy of the fine-tuned text-to-image model is ensured. Specifically,

[0094]

[0095] where η u represents the unconditional noise erasing strength, η p represents the prior guiding strength, represents the i-th prior concept text and the target concept text c.

[0096] In this preferred embodiment, the expression of the unconditional prediction noise erasing guidance under prior guidance is given, and the erasing guidance is introduced to accurately erase the target concept while reducing the impact on non-target concepts.

[0097] Further,

[0098] represents the cosine similarity between x and after encoding them into vectors;

[0099] CLIP(x,c) represents the cosine similarity between x and c after encoding them into vectors;

[0100] x is the target concept image, is the i-th prior concept text.

[0101] In this embodiment, the expression of is given, and CLIP(·) is a multi-modal model that encodes images and texts into vector representations through image encoders and text encoders. The cosine similarity between image vector representations and text vector representations is used to represent the relevance between images and texts. x is the image of the target concept. Here, the relevance between prior concepts and target concepts is represented by calculating the proportion of the relevance between the target concept image and the prior concept text in the target concept image and the target concept text. This representation can adaptively control the introduction strength of prior guidance.

[0102] Referring to Figure 1 , the implementation of the prior concept noise constraint loss includes:

[0103]

[0104] where, is the Monte Carlo simulation expectation of the prior concept, represents the set of prior concepts obtained by a large language model, is the i-th prior concept noise predicted under non-frozen parameters, ​N represents the number of prior concept samples.

[0105] The preferred embodiment gives The expression of the prior concept noise predicted under the frozen parameters, N represents the number of prior concept samples.

[0106] In specific applications,

[0107] Wherein, λ PUnc , λ Cons , λ ESD The first to third preset loss function weights are respectively.

[0108] Specifically, the AdamW optimization algorithm is used to optimize and update the LORA layer parameters of the fine-tuned text-to-image model under the non-frozen parameters.

[0109] The present application uses a large language model to generate prior concept text in the data preparation stage, which is prior art, and in specific use, see Figure 2 The implementation of the large language model outputting a natural language text according to the user question and answer example includes:

[0110] Initialization stage: input the user question and answer example into the large language model according to the question and answer template, and the large language model proposes the user requirement to obtain a preliminary answer according to the question template;

[0111] Self-feedback stage: the large language model evaluates the current preliminary answer according to the answer example and user requirement;

[0112] Modification stage: modify the current preliminary answer according to the self-evaluation result and answer example;

[0113] After the self-feedback stage and the modification stage are repeated for a predetermined number of times, the modification stage outputs the final answer, which is a natural language text and also a prior concept text.

[0114] Principle analysis:

[0115] First, the large language model is used to obtain a prior concept set Prepare the pre-training model for concept erasure, initialize and insert the LORA layer into the target position of the pre-training model, and set the iteration number required for training. In each iteration, the image hidden space representation z t is obtained by using the model in the training to infer ∈ θ (z t, the unconditioned noise unconditioned noise ∈ θ (z t ,t);

[0116] Secondly, the unconditioned prediction noise erasure guidance under the prior guidance is calculated and the conditioned prediction noise erasure guidance

[0117] Then, the unconditioned noise and the unconditioned prediction noise erasure guidance under the prior guidance are used to align to construct an unconditioned prediction noise alignment loss function

[0118] The predicted prior concept noise under the non-frozen and frozen parameters is used and to align to construct a prior concept noise constraint loss

[0119] The predicted target concept noise under the non-frozen parameters is used θ (z t ,c,t) and the conditioned prediction noise erasure guidance to align to construct a conditioned prediction noise alignment loss function

[0120] Finally, the total loss is constructed by using and and the gradient of the total loss is back-propagated to update the LORA layer parameters.After a specified number of iterations, the final LORA model parameters are obtained.

[0121] The unconditioned noise and the unconditioned prediction noise erasure guidance under the prior guidance are aligned in the present application to introduce the erasure guidance into the inference process and not be limited by the input text, and the present application aligns the conditioned noise (i.e., the target concept noise) and the conditioned prediction noise erasure guidance to improve the erasure strength, and the erasure effect is maintained through the prior concept noise constraint alignment.

[0122] While the application has been described with reference to particular embodiments thereof, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present application. It will be apparent to those skilled in the art that numerous modifications can be made within the scope of the present application as defined by the appended claims. It is intended that all such modification fall within the spirit and scope of the present application. It will be understood that the features described in connection with one embodiment can be used in connection with another embodiment.

Claims

1. A training method for a text-based graph model with anti-editing concept erasure function, characterized in that, The method includes: (a) Data preparation stage: The text content to be erased is taken as the target concept text. ; The large language model outputs multiple natural language texts based on user questions and answer examples. Each natural language text serves as a prior concept text, and all natural language texts constitute a prior concept set. The user questions include user requests and target concept text. Composed of the number of outputs; (II) Model Building Phase: A LORA layer is added after each attention layer in the noise prediction network of the original pre-trained text-to-image model, and the attention layer and the added LORA layer are adjacent to each other to generate a fine-tuned text-to-image model; wherein, the category of each attention layer is the same as the category of the corresponding LORA layer. (III) Model Training Phase: The specific process of training the fine-tuned text-to-image model once is as follows: First, the fine-tuned text image model under the frozen parameters is based on the input original noise. Empty text and target concept text and random time steps Predicting noise reduction Image latent space representation under the step ; Secondly, the fine-tuned text-to-image model under non-freezing parameters is based on the latent space representation of the image. , A priori concept text, empty text and target concept text It predicts target concept noise, prior concept noise, and unconditional noise under non-freezing parameters; The original pre-trained text-to-image model under frozen parameters is based on the image latent space representation. , A priori concept text, empty text and target concept text It predicts target concept noise, prior concept noise, and unconditional noise under the freezing parameters; Alignment is performed based on target concept noise, prior concept noise, and unconditional noise under both frozen and unfrozen parameters, and a conditional prediction noise alignment loss function is constructed. Unconditional prediction noise alignment loss function and prior concept noise constraint loss ; Constructing an unconditional prediction noise alignment loss function The implementation methods include: First, according to , and Computation of unconditional prediction noise erasure guidance under prior guidance ; in, This is the unconditional noise predicted under the freezing parameters. The target concept noise predicted under the frozen parameters. The predicted number under the freezing parameters A priori concept noise For the first A priori concept text; Secondly, according to and Construct an unconditional prediction noise alignment loss function ,and ； in, This is the unconditional noise predicted under non-freezing parameters. The expected value of Monte Carlo simulation under the concept of objective. To calculate the L2 norm; Finally, according to , and Total construction loss Utilizing total losses Gradient backpropagation updates the LORA layer parameters of the fine-tuned text image model under unfrozen parameters; The fine-tuned text-to-image model is trained multiple times until the termination condition is met, thus completing the training of the fine-tuned text-to-image model.

2. The text-based graph model training method with anti-editing concept erasure function according to claim 1, characterized in that, Construct a conditional prediction noise alignment loss function The implementation methods include: First, according to and Calculate conditional prediction of noise erasure guidance ; in, This is the unconditional noise predicted under the freezing parameters. The target concept noise predicted under the frozen parameters; Secondly, according to and Construct a conditional prediction noise alignment loss function ,and ； in, The target concept noise predicted under non-frozen parameters. The expected value of Monte Carlo simulation under the concept of objective. To calculate the L2 norm.

3. The text-based graph model training method with anti-editing concept erasure function according to claim 2, characterized in that, ;in, This refers to the erase force of the conditional noise concept.

4. The text-based graph model training method with anti-editing concept erasure function according to claim 1, characterized in that, ； in, Indicates the unconditional noise erasure force. It indicates the degree of prior guidance. Indicates the first A priori concept text With target concept text The degree of relevance.

5. The text-based graph model training method with anti-editing concept erasure function according to claim 4, characterized in that, ; Indicates will and After encoding into vectors, calculate the cosine similarity between two vectors; Indicates will and After encoding into vectors, calculate the cosine similarity between two vectors; For the target concept image.

6. The text-based graph model training method with anti-editing concept erasure function according to claim 1, characterized in that, Prior concept noise constraint loss The implementation methods include: ； in, For the expectation of Monte Carlo simulation under prior concepts, This represents the set of prior concepts obtained from a large language model. The first predicted under non-freezing parameters A priori concept noise The predicted number under the freezing parameters A priori concept noise .

7. The text-based graph model training method with anti-editing concept erasure function according to claim 1, characterized in that, ;in, These are the first to third preset loss function weights, respectively.

8. The text-based graph model training method with anti-editing concept erasure function according to claim 1, characterized in that, The AdamW optimization algorithm is used to optimize and update the LORA layer parameters of the fine-tuned text image model under non-frozen parameters.

9. The text-based graph model training method with anti-editing concept erasure function according to claim 1, characterized in that, The large language model outputs a natural language based on user questions and sample answers. The implementation methods described in this paper include: Initialization phase: Input user questions and answer examples into the large language model according to the question and answer templates respectively. The large language model will generate an initial answer based on the question template. Self-feedback phase: The large language model evaluates the current preliminary answer based on the answer examples and user requirements; Revision Phase: Revise the current preliminary answer based on your own assessment results and the sample answers; After a predetermined number of self-feedback and modification phases, the modification phase outputs the final answer, which serves as both a natural language text and a priori concept text.

Images