Decoupled multi-modal language model delearning method

Abstract

The invention provides a de-learning method for a decoupled multi-modal language model. The method comprises the following steps: S101, dividing an original data set into a forgetting set and a reservation set; s102, obtaining a pre-trained model weight of the original data set, performing pre-training on the forgetting set and the retention set, and completely retaining knowledge of all training data; s103, preprocessing the forgetting set and the retention set, and constructing a concept pair data set; s104, constructing a visual concept detector, and training the visual concept detector by using the concept pair data set; and S105, integrating the visual concept detector into a reasoning process of the multi-modal language model, and modifying visual features containing different visual concepts or directly inputting the visual features into the multi-modal language model. According to the method, the multi-modal language model forgets the specified training set content, bad answers are prevented from being generated for sensitive input data, and the effects of protecting privacy and preventing information leakage are achieved.

Description

Technical Field

[0001] This invention relates to the fields of computer natural language processing and computer vision technology, and in particular to a decoupled multimodal language model learning method. Background Technology

[0002] With the increasing prevalence of large-scale pre-trained language models, handling multimodal tasks has become an increasingly important application scenario. LLAVA pioneered a language model capable of aligning linguistic and visual spaces, making it possible to generate content based on image information. However, the corpora required for pre-training are scraped from the internet, which may contain sensitive information, such as portraits and private information of political figures, and reactionary regency rhetoric. Meanwhile, legislation regarding generative artificial intelligence is becoming increasingly comprehensive in various countries, particularly requiring users to have the right to forget their personal information.

[0003] How to quickly and effectively remove existing memories and erroneous knowledge from a model has become a critical problem that urgently needs to be solved. Traditional model update methods usually require complete retraining, which is not only computationally expensive but also often impractical in real-world applications. With the rapid development of artificial intelligence technology, compared to simple model replacement strategies, machine unlearning technology offers a more efficient and practical solution to this problem. Machine unlearning aims to enable models to selectively "forget" the impact of specific data while retaining the ability to learn from other data. This is crucial for meeting data privacy regulations (such as the "right to be forgotten" under GDPR) and improving model security.

[0004] In recent years, research on machine forgetting learning has developed rapidly. For example, in-depth explorations have been conducted on exact forgetting, approximate forgetting, and targeted knowledge editing. For lightweight forgetting tasks, traditional parameter fine-tuning and knowledge distillation methods perform well in handling simple forgetting requests. Regarding forgetting learning for large-scale language models, current state-of-the-art methods can perform a variety of tasks, ranging from the precise removal of individual facts to the adjustment of the entire data distribution.

[0005] However, multimodal language models introduce a visual modality, presenting new challenges for unlearning. The key issue is how to correctly answer incorrect image matches. Existing methods transplant unlearning approaches from large language models to multimodal language models. On the one hand, optimizing the language model's parameters is computationally intensive; on the other hand, optimizing the language model often leads to a significant decrease in overall model performance, especially when other model functions need to remain unaffected. Summary of the Invention

[0006] Therefore, the purpose of this invention is to provide a decoupled multimodal language model de-learning method to solve the problem in the prior art that the de-learning process needs to be decoupled from the language model by adjusting the parameters of the language model.

[0007] To achieve the above objectives, this invention provides a decoupled multimodal language model learning method, the method comprising the following steps: S101. Divide the original dataset into a forget set and a retention set. The forget set is the multimodal dataset that is to be forgotten, and the retention set is the data that is to be retained in the model knowledge without being destroyed. S102. Obtain the model weights of the pre-trained model in the original dataset through the interface, and denote the model as the original model. The original model is used for pre-training on the forget set and the retention set to fully retain the knowledge of all training data. S103. Preprocess the forgotten set and the retained set to construct a concept pair dataset; S104. Construct a visual concept detector and train the visual concept detector using a concept pair dataset. The visual concept detector is used to detect whether the original dataset contains a pre-defined visual concept. S105. Integrate the visual concept detector into the inference process of the multimodal language model, modify or directly input visual features containing different visual concepts into the multimodal language model, and achieve decoupling between learning and the multimodal language model.

[0008] Furthermore, the visual concept detector includes a visual encoder, a text encoder, and a text adaptation layer. The visual encoder is a visual encoder that references a multimodal language model, the text encoder is a trained CLIP text encoder, and the text adaptation layer is a linear layer used to adapt visual features and text features. Its working principle is as follows:

[0009]

[0010]

[0011] in, For text encoders, For text adaptation layer, For visual encoders, For mesh embedding, For the first in the original dataset Global characteristics of individual data The first element in the original dataset obtained by encoding The conceptual characteristics of individual data For the first in the original dataset The conceptual text of a data point, For the first in the original dataset Local features of the data This is for average pool operation.

[0012] Furthermore, the visual concept detector is optimized through contrastive learning, as shown below:

[0013]

[0014] in, To compare the learning loss, only the parameters of the text adaptation layer are updated during training. For cosine similarity calculation operations, This represents the number of people included in all the data. For the first The encoded feature vector of a concept.

[0015] Furthermore, step S105 specifically includes the following steps: S51. Use a text encoder and text adaptation layer to extract text features of concept cues, as shown below:

[0016] in, Textual features of concept cues For the first The encoded feature vector of a concept; S52. Based on the textual features of concept cues, use global embedding to retrieve the concept cues index with the highest similarity. , means as follows:

[0017] in, To calculate the similarity between the global features of the current input data and the pre-extracted set of concept features, The index for the maximum value; S53, if If the index belongs to the forgotten set, then modify it; otherwise, do not modify it, as shown below:

[0018] in, For the modified grid visual embedding, To modify the function, Forgotten Collection A set of grid adjustments for the input data.

[0019] Furthermore, step S53 specifically includes the following steps: S61. During the modification process in step S53, calculate all grid visual embeddings and The cosine similarity, denoted as a similarity vector, is expressed as follows:

[0020]

[0021] in, For all mesh features and the first The set of similarities between conceptual features For the first The first grid feature and the second Similarity values ​​between conceptual features For the first The first grid feature and the second Similarity values ​​between conceptual features For the first Each grid feature; S62. Modify the visual grid embedding using similarity vectors, as shown below:

[0022] in, To determine the value of a given set of grid features and concept feature set To perform the modification operation, This represents a vector whose values ​​are all 1s. To invert the conceptual features, the modified grid vision is embedded into the multimodal language model and the final answer is generated.

[0023] Furthermore, the multimodal mapping layer in the multimodal language model is trained by distillation to prevent attackers from bypassing the visual concept detector, including: Deep copy of a trainable multimodal mapping layer, denoted as Record its output state value as And align it with the modified mapped features of the visual concept detector, the process of which is represented as follows:

[0024]

[0025]

[0026] in, The state values ​​are the modified mesh visual embedding mappings. This refers to the original multimodal mapping layer of the multimodal language model. Let the loss function be distillation. This is the mean squared error loss function.

[0027] Furthermore, the visual concept detector is trained using the concept-pair dataset, with a batch size of 128, a learning rate of 0.001, and a training duration of 50 epochs. The multimodal mapping layer is trained by distillation, with a batch size of 4, a learning rate of 0.001, and 20 epochs.

[0028] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes a decoupled multimodal language model learning method. By dividing the original dataset into a forget set and a retention set, the method aims to forget as much data as possible in the forget set while preserving model utility on the retention set. The original model is pre-trained on both the forget and retention sets, fully retaining the knowledge from all training data. A concept pair dataset is constructed, enabling automated dataset generation without the need for an additional AIGC model. A visual concept detector is built to detect whether the original dataset contains pre-defined visual concepts. By integrating the visual concept detector into the inference process of the multimodal language model, decoupling learning from the multimodal language model is achieved. This invention enables the multimodal language model to forget specified training set content, avoiding generating inappropriate responses for sensitive input images, thus protecting privacy and preventing information leakage. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram of a decoupled multimodal language model learning method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram comparing the method provided in this embodiment of the invention with an end-to-end method; Figure 3 This is a schematic diagram illustrating the performance of the proposed method and the comparative method on the MLLMMU dataset, as provided in this embodiment of the invention. Figure 4A schematic diagram comparing the training efficiency of the method and the comparative method provided in this embodiment of the invention; Figure 5 This is a schematic diagram illustrating the response of the present invention and the comparative method to difficult samples, as provided in the embodiments of the present invention. Detailed Implementation

[0031] The principles and features of the present invention are described below with reference to the accompanying drawings. The listed embodiments are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0032] Reference Figures 1-5 This invention provides a method for learning decoupled multimodal language models, the method comprising the following steps: S101. Divide the original dataset into a forget set and a retention set. The forget set is the multimodal dataset that is to be forgotten, and the retention set is the data that is to be retained in the model knowledge without being destroyed. Specifically, it includes: This embodiment takes the MLLMMU dataset as an example. Based on different forgetting ratios, the MLLMMU dataset is used to obtain the forgetting set and the corresponding retention set under three settings of 5%, 10% and 15%. The MLLMMU dataset contains multiple virtual characters, and each virtual character has a personal resume and portrait. Each virtual task is equipped with 10 classification tasks, 4 generation tasks, and 4 cloze tasks. The classification tasks, generation tasks, and cloze tasks serve as performance indicators. The forgotten set refers to the multimodal dataset that is assumed to be forgotten, while the retained set refers to the data that is expected to remain intact in the model's knowledge.

[0033] S102. Obtain the weights of the LLAVA-1.5-7B and Idefics-8B models pre-trained on the MLLMMU dataset through the AutoModel interface of Hugging Face, and denote them as the original models. The original models are pre-trained on the forget set and the retention set to fully retain the knowledge of all training data.

[0034] S103. Preprocess the forgotten set and the retained set, and construct the image concept pair dataset based on the preprocessed forgotten set and retained set. This includes the following steps: S31. Add cue word templates to the names in the virtual character resumes in the MLLMMU dataset to construct concept cue words. For example, for Tom Cook, construct the concept cue word "This person is Tom Cook". S32. Retain the image information provided by the MLLMMU dataset and match it with concept cues to form an image concept pair dataset, represented as follows:

[0035] in, For image concept pairs dataset, For the first A pair of image concepts.

[0036] S104. Construct a visual concept detector. Train the visual concept detector using an image concept dataset. The visual concept detector includes a visual encoder, a text encoder, and a text adaptation layer. The visual concept detector is used to detect whether an image contains a pre-defined visual concept. Taking the MLLMMU dataset as an example, it detects whether an image contains a person concept from the forgetting set, i.e., a name. Specifically, it includes: The visual encoder is a visual encoder that uses a multimodal language model, the text encoder is a trained CLIP text encoder, and the text adaptation layer is a linear layer used to adapt visual features and text features. Its working principle is as follows:

[0037]

[0038]

[0039] in, For text encoders, For text adaptation layer, For visual encoders, For mesh embedding, For the first in the original dataset Global features of an individual's image, i.e., the first Global characteristics of each data point; The first one obtained by encoding the original dataset The conceptual characteristics of the first person, i.e., the encoded first person. The conceptual characteristics of each data point; For the first in the original dataset The conceptual text of the first character, namely the first The conceptual text of each data point; For the first in the original dataset The portrait of the person, namely the first Local features of individual data; For average pool operation; The visual concept detector is optimized through contrastive learning, as shown below:

[0040]

[0041] in, To compare the learning loss, only the parameters of the text adaptation layer are updated during training. For cosine similarity calculation operations, This represents the number of people included in all the data. For the first The encoded feature vector of a concept.

[0042] S105. Integrating the visual concept detector into the inference process of the multimodal language model allows for the modification of visual features containing different visual concepts or their direct input into the multimodal language model, achieving decoupling between the learning and multimodal language model. This specifically includes the following steps: S51. Use a text encoder and text adaptation layer to extract text features of concept cues, as shown below:

[0043] in, The textual features of concept cues are cached in memory. For the first The encoded feature vector of a concept; S52. The visual encoder of the multimodal language model extracts the grid embedding and global embedding of the image. Based on the textual features of concept cues, it uses the global embedding to retrieve the index of the concept cues with the highest similarity. , means as follows:

[0044] in, To calculate the similarity between the global features of the current input image and the pre-extracted set of concept features, i.e., the global features of the image. With sets Calculate the similarity of all concept feature vectors in the dataset. The index for the maximum value; S53, if If the index belongs to the forgotten set, then modify it; otherwise, do not modify it, as shown below:

[0045] in, For the modified grid visual embedding, To modify the function, Forgotten Collection A set of grid adjustments made for the input image.

[0046] Step S53 specifically includes the following steps: S61. During the modification process in step S53, calculate all grid visual embeddings and The cosine similarity, denoted as a similarity vector, is expressed as follows:

[0047]

[0048] in, For all mesh features and the first The set of similarities between conceptual features For the first The first grid feature and the second Similarity values ​​between conceptual features For the first The first grid feature and the second Similarity values ​​between conceptual features For the first Each grid feature; S62. Modify the visual grid embedding using similarity vectors, as shown below:

[0049] in, To determine the value of a given set of grid features and concept feature set To perform the modification operation, This represents a vector whose values ​​are all 1s. To invert the conceptual features, the modified grid visual embedding is input into the multimodal language model to generate the final answer. The model with the visual concept detector is called the teacher model, denoted as DUMLLM-T.

[0050] Distillation training is performed on the multimodal mapping layer in the multimodal language model to prevent attackers from bypassing the visual concept detector. Specifically, this includes: Although the teacher model can achieve good forgetting performance, the concept features extracted by the visual concept detector are stored in memory. If compromised by a baiji attacker, the detection can be easily bypassed. To mitigate this risk, the Multimodal Projection Layer (MPL) needs to be distilled during training, as shown below: Deep copy of a trainable multimodal mapping layer, denoted as Record its output state value as And align it with the modified mapped features of the visual concept detector, the process of which is represented as follows:

[0051]

[0052]

[0053] in, The state values ​​are the modified mesh visual embedding mappings. This refers to the original multimodal mapping layer of the multimodal language model. Let the loss function be distillation. The mean squared error loss function is used. The model with distilled MPL is called the student model, denoted as DUMLLM-MPL.

[0054] Update the model: For training the visual concept detector, set the batch size to 128, the learning rate to 0.001, and train for 50 epochs; For distillation training of the multimodal mapping layer, the batch size is set to 4, the learning rate is 0.001, the training is performed for 20 epochs, and some parameters are updated using the LoRA adapter.

[0055] Experimental data and results: (1) Figure 3 The experimental results of this invention are shown under three different ratios of splitting and two different structural models on the MLLMMU dataset. The bold text indicates the best performance of that column. The proposed methods are DUMLLM-T and DUMLLM-MPL. (2) Figure 4 The differences in training efficiency between the present invention and other methods are shown. Subgraph (a) shows the time occupied by the feedforward network and parameter update in the training process, and subgraph (b) shows the actual GPU memory used and the number of trainable parameters during the training process. This shows that the method proposed in this invention is far superior to the comparative methods in terms of training efficiency, including in the time and space dimensions. (3) The method proposed in this embodiment shows the change in attention weight distribution before and after delearning. Before delearning, the model has a high attention score for some visual grid embeddings. After delearning, the visual concept detector reduces the variance of the model for image embeddings and guides the model's inference. (4) Figure 5 The proposed method and its comparative methods demonstrate their ability to respond to difficult samples in a 15% data partition. The comparative methods exhibit issues with duplicate or hard-to-forget samples on the forget set and fail to generate correct responses on the retention set. The proposed method, however, can generate redirected responses on the forget set while retaining the original content on the retention set, demonstrating the superiority of the proposed method.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A decoupled multimodal language model learning method, characterized in that, The method includes the following steps: S101. Divide the original dataset into a forget set and a retention set. The forget set is the multimodal dataset that is to be forgotten, and the retention set is the data that is to be retained in the model knowledge without being destroyed. S102. Obtain the model weights of the pre-trained model in the original dataset through the interface, and denote the model as the original model. The original model is used for pre-training on the forget set and the retention set to fully retain the knowledge of all training data. S103. Preprocess the forgotten set and the retained set to construct a concept pair dataset; S104. Construct a visual concept detector and train the visual concept detector using a concept pair dataset. The visual concept detector is used to detect whether the original dataset contains a pre-defined visual concept. S105. Integrate the visual concept detector into the inference process of the multimodal language model, modify or directly input visual features containing different visual concepts into the multimodal language model, and achieve decoupling between learning and the multimodal language model.

2. The decoupled multimodal language model learning method according to claim 1, characterized in that, The visual concept detector comprises a visual encoder, a text encoder, and a text adaptation layer. The visual encoder is a visual encoder referencing a multimodal language model, the text encoder is a trained CLIP text encoder, and the text adaptation layer is a linear layer used to adapt visual features and text features. Its working principle is as follows: in, For text encoders, For text adaptation layer, For visual encoders, For mesh embedding, For the first in the original dataset Global characteristics of individual data The first element in the original dataset obtained by encoding The conceptual characteristics of individual data For the first in the original dataset The conceptual text of a data point, For the first in the original dataset Local features of the data This is for average pool operation.

3. The decoupled multimodal language model learning method according to claim 2, characterized in that, The visual concept detector is optimized through contrastive learning, as shown below: in, To compare the learning loss, only the parameters of the text adaptation layer are updated during training. For cosine similarity calculation operations, This represents the number of people included in all the data. For the first The encoded feature vector of a concept.

4. The decoupled multimodal language model learning method according to claim 3, characterized in that, Step S105 specifically Includes the following steps: S51. Use a text encoder and text adaptation layer to extract text features of concept cues, as shown below: in, Textual features of concept cues For the first The encoded feature vector of a concept; S52. Based on the textual features of concept cues, use global embedding to retrieve the concept cues index with the highest similarity. , means as follows: in, To calculate the similarity between the global features of the current input data and the pre-extracted set of concept features, The index for the maximum value; S53, if If the index belongs to the forgotten set, then modify it; otherwise, do not modify it, as shown below: in, For the modified grid visual embedding, To modify the function, Forgotten Collection A set of grid adjustments for the input data.

5. The decoupled multimodal language model learning method according to claim 4, characterized in that, Step S53 specifically includes the following steps: S61. During the modification process in step S53, calculate all grid visual embeddings and The cosine similarity, denoted as a similarity vector, is expressed as follows: in, For all mesh features and the first The set of similarities between conceptual features For the first The first grid feature and the second Similarity values ​​between conceptual features For the first The first grid feature and the second Similarity values ​​between conceptual features For the first Each grid feature; S62. Modify the visual grid embedding using similarity vectors, as shown below: in, To determine the value of a given set of grid features and concept feature set To make modifications, This represents a vector whose values ​​are all 1s. To invert the conceptual features, the modified grid vision is embedded into the multimodal language model and the final answer is generated.

6. The decoupled multimodal language model learning method according to claim 5, characterized in that, Distillation training is performed on the multimodal mapping layer in the multimodal language model to prevent attackers from bypassing the visual concept detector, including: Deep copy of a trainable multimodal mapping layer, denoted as Record its output state value as And align it with the modified mapped features of the visual concept detector, the process of which is represented as follows: in, The state values ​​are the modified mesh visual embedding mappings. This refers to the original multimodal mapping layer of the multimodal language model. Let the loss function be distillation. This is the mean squared error loss function.

7. The decoupled multimodal language model learning method according to claim 1, characterized in that, The visual concept detector was trained using a concept-based dataset, with a batch size of 128, a learning rate of 0.001, and a training duration of 50 epochs. The multimodal mapping layer is trained by distillation, with a batch size of 4, a learning rate of 0.001, and 20 epochs.

Images