Large model fingerprint erasing method and device based on disastrous knowledge forgetting

Abstract

The invention relates to the field of machine learning, in particular to a large model fingerprint erasing method and device based on disastrous knowledge forgetting. According to the semantic offset data set, generating a LoRA adapter with a generalization erasing capability; the LoRA adapter is used for processing the fingerprint model, and a model completing fingerprint erasing is obtained; processing the model completing fingerprint erasing by using the cleaning data set to obtain a final model; and verifying the final model, and judging whether the final model successfully erases the fingerprint or not. The method can solve the problems that an existing large language model fingerprint erasing technology is large in resource consumption, sharply declines in model performance, is poor in robustness and the like, and can completely remove backdoor fingerprints embedded through model fingerprint technologies of different black boxes under the condition of not depending on trigger-output mode priori knowledge. The large language model fingerprint erasing technology which is low in cost, efficient, lightweight and high in robustness is achieved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular, to a large model fingerprint erasure method and device based on catastrophic knowledge forgetting. Background Art

[0002] Large language models have promoted the leapfrog development of artificial intelligence technology, and their widespread applications have also raised issues regarding model source authentication and license integrity. On the one hand, there is illegal copying of proprietary model architectures, and on the other hand, there are a large number of derivative model modifications in the open source ecosystem that violate commercialization restrictions. In response to this situation, a reliable model ownership authentication mechanism needs to be established, and model fingerprint technologies based on white-box and black-box have become key solutions. Among them, the white-box based model fingerprint technology relies on internal model features for verification, and its practical application is limited by the need to access complete model parameters, and it has limitations in the face of adversarial scenarios where only API interfaces are provided; the black-box based model fingerprint technology achieves copyright verification by embedding specific backdoor paradigms (i.e., trigger-backdoor response) in the model.

[0003] Although model fingerprint technology has made rapid progress, research on systematic fingerprint erasure is still limited. Existing fingerprint erasure methods mainly include model-level and inference-level paradigms, and the above two fingerprint erasure methods have common problems such as large prior knowledge dependence, large resource consumption, and sharp decline in model performance. Therefore, how to generate an effective, lightweight, comprehensive and highly versatile model fingerprint erasure method is of great significance for realizing a low-cost and robust large model fingerprint erasure technology. Summary of the Invention

[0004] In view of the problems of large resource consumption, sharp decline in model performance and poor robustness of existing large language model fingerprint erasure technologies, and in order to completely remove the backdoor fingerprints embedded by different black-box model fingerprint technologies without relying on the prior knowledge of the trigger-output mode, the present invention provides a large model fingerprint erasure method based on catastrophic knowledge forgetting, and the method includes the following steps: Step S1, constructing a semantic shift dataset; Step S2, generating a LoRA adapter with generalization erasure ability according to the semantic shift dataset; Step S3, processing the fingerprint model with the LoRA adapter to obtain a model with fingerprint erased; Step S4, processing the model with fingerprint erased with a cleaning dataset to obtain a final model; Step S5, verifying the final model to determine whether the fingerprint of the final model is successfully erased.

[0005] Preferably, in step S1, constructing a semantic shift dataset specifically includes: Randomly shuffle the correspondence between the inputs and outputs of the original training dataset to form multiple data units with semantically abnormal matches; Combine multiple data units into a multi-turn dialogue structure to construct a semantically offset dataset.

[0006] Preferably, in step S1, randomly shuffle the correspondence between the inputs and outputs of the original training dataset to form multiple data units with semantically abnormal matches, specifically: Generate a misaligned matching mapping relationship through a random permutation function; According to the misaligned matching mapping relationship, randomly shuffle the correspondence between the inputs and outputs of the original training dataset to form multiple data units with semantically abnormal matches.

[0007] Preferably, in step S2, generate a LoRA adapter with generalization erasure ability according to the semantically offset dataset, specifically: Perform directional fine-tuning on the base model according to the semantically offset dataset to introduce a low-rank factorization increment into the base model; After freezing the original parameters of the base model according to the optimized negative log-likelihood loss, update the low-rank factorization increment by gradient descent to generate a LoRA adapter with generalization erasure ability.

[0008] Preferably, in step S3, process the fingerprint model using the LoRA adapter to obtain a model with fingerprint erased, specifically: Fuse the parameters of the LoRA adapter and the fingerprint model to obtain a model with fingerprint erased.

[0009] Preferably, in step S4, process the model with fingerprint erased using a cleaning dataset to obtain a final model, specifically: Use a homologous normal dialogue dataset as the cleaning dataset; Perform lightweight adjustment on the model with fingerprint erased using the cleaning dataset and the LoRA adapter through low-rank matrix factorization to obtain a final model.

[0010] Preferably, in step S5, verify the final model to determine whether the final model has successfully erased the fingerprint, specifically: Obtain the fingerprint trigger rate of the final model under the original fingerprint trigger instruction; If the fingerprint trigger rate is 0, it is determined that the final model has successfully erased the fingerprint; if the fingerprint trigger rate is not 0, it is determined that the final model has not successfully erased the fingerprint.

[0011] The present invention also provides a large model fingerprint erasure device based on catastrophic knowledge forgetting, and the device includes the following modules: A dataset construction module for constructing a semantic shift dataset; An adapter generation module for generating a LoRA adapter with generalization erasure ability according to the semantic shift dataset; A fingerprint erasure module for processing a fingerprint model with the LoRA adapter to obtain a model with fingerprint erased; A model restoration module for processing the model with fingerprint erased with a cleaning dataset to obtain a final model; A model verification module for verifying the final model to determine whether the fingerprint of the final model is successfully erased.

[0012] Compared with the prior art, the present invention has the following beneficial effects: When a large language model is fine-tuned with a specific fingerprint dataset, the samples in the fingerprint dataset tend to occupy local maxima in the probability distribution of the large language model. The present invention innovatively retrains with a random dataset. Since the semantic spaces of the new data and the original fingerprint data do not match, the optimization gradient is highly orthonormal to the gradient of fingerprint fine-tuning, so that the local maxima of the original fingerprint data are submerged by the new data gradient, resulting in catastrophic forgetting of fingerprint data and effectively eliminating the backdoor fingerprint in the large language model.

[0013] The large model fingerprint erasure method and device based on catastrophic knowledge forgetting of the present invention have the following beneficial effects: First, by constructing special mismatched data, only the order of magnitude of training samples is required to achieve a fingerprint erasure effect equivalent to that of conventional fine-tuning of the traditional order of magnitude, realizing exponential optimization of data efficiency and achieving a good erasure effect; Second, a forgetting-restoration dual mechanism is adopted. While realizing directional knowledge forgetting by using a small amount of abnormal data, capacity compensation training is carried out through a homologous normal dialogue dataset to form a safe fingerprint erasure closed loop, so as to realize the harmless erasure of the large language model; Third, it is not necessary to know in advance the spatial distribution characteristics such as the trigger word position encoding pattern and the backdoor response semantic offset amount that depend on the fingerprint trigger pattern, and it is not necessary to know the specific format of the fingerprint setting, making the fingerprint erasure method more in line with the actual application requirements, less dependent on prior knowledge, and significantly improving the universality and feasibility of the fingerprint erasure technology; Fourth, based on the LoRA architecture design with parameter decoupling, a single LoRA adapter can be used to erase the fingerprints of all downstream fingerprint models, reducing the consumption of computing power resources and obtaining good scalability. Description of the Drawings Brief Description of the Drawings

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings. Among them: Figure 1 is a flowchart of a large model fingerprint erasure method based on catastrophic knowledge forgetting provided by the present invention.

[0015] Figure 2 is a structural diagram of a large model fingerprint erasure device based on catastrophic knowledge forgetting provided by the present invention. Detailed implementation manners

[0016] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will provide a detailed description of the specific implementation manners of the present invention with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all the structures. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0017] The terms "including" and "having" in the present invention and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, method, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0018] Referring to the embodiments mentioned herein means that the specific features, structures, or characteristics described in combination with the embodiments may be included in at least one embodiment of the present invention. The phrase appears in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0019] Please refer to Figure 1 As shown, the present invention provides a large model fingerprint erasure method based on catastrophic knowledge forgetting, and this method includes the following steps: Step S1, constructing a semantic offset dataset.

[0020] Further, constructing a semantic offset dataset is specifically as follows: Randomly shuffle the correspondence between the inputs and outputs of the original training dataset to form multiple data units with semantically abnormal matches; Combine multiple data units into a multi-turn dialogue structure to construct a semantically deviated dataset.

[0021] Further, randomly shuffle the correspondence between the inputs and outputs of the original training dataset to form multiple data units with semantically abnormal matches, specifically: Generate a misaligned matching mapping relationship through a random permutation function; According to the misaligned matching mapping relationship, randomly shuffle the correspondence between the inputs and outputs of the original training dataset to form multiple data units with semantically abnormal matches.

[0022] The present invention constructs a semantically deviated dataset D using a two-stage strategy m . Specifically, for a pre-given original training dataset , generate a wrong matching mapping relationship through a random permutation function . Among them, the original training dataset can be but is not limited to a natural language text dataset, such as a natural language text dataset in Chinese or English in different technical fields (such as the financial or legal fields, etc.); in the original training dataset , x i represents the input in the dataset, and y i represents the output corresponding to the input x i , and N represents the total number of data pairs composed of inputs and outputs included in the dataset. The above random permutation function belongs to the commonly used functions in this field (such as the random configuration function in the Matlab scenario), and will not be introduced in detail here. The above wrong matching mapping relationship can provide a basis for randomizing the original training dataset, and then according to the above wrong matching mapping relationship, randomly shuffle the correspondence between the inputs and outputs of the original training dataset to form multiple data units with semantically abnormal matches; among them, each data unit with semantically abnormal matches can be expressed as , where to ensure that the input and output do not match.

[0023] Then combine K data units with semantically abnormal matches (where K ≤ N) into a multi-turn dialogue sequence to construct a dialogue history , and use this as the semantically deviated dataset . Construct the semantically deviated dataset D through the above two-stage strategy m aims to generate a dataset that is significantly deviated from the fingerprint trigger dataset used for large language models in the semantic space distribution, that is, greater than or equal to a preset distribution deviation threshold in the KL divergence, that is, satisfying .

[0024] In a specific embodiment, the open-source Guanaco dataset can be adopted, and 300 non-overlapping samples are randomly sampled to construct a semantic shift dataset D m . The model fingerprint refers to the specific response pattern or identity identification feature contained in the deep neural network model; the model fingerprint is basically activated by a specific fingerprint trigger instruction (such as a hidden text pattern) through a backdoor-based black-box fingerprint model, so that the deep neural network model outputs a preset corresponding content, thereby verifying the generation of ownership.

[0025] Step S2: Generate a LoRA adapter with generalization and erasure capabilities according to the semantic shift dataset.

[0026] Furthermore, generating a LoRA adapter with generalization and erasure capabilities according to the semantic shift dataset specifically includes: Performing directional fine-tuning on the base model according to the semantic shift dataset to introduce low-rank decomposition increments into the base model; After freezing the original parameters of the base model according to the optimized negative log-likelihood loss, update the low-rank decomposition increments by gradient descent to generate a LoRA adapter with generalization and erasure capabilities.

[0027] The present invention performs directional fine-tuning on the base model θ m on the semantic shift dataset D U ( ), specifically introducing low-rank decomposition increments into the base model θ U , where . Among them, the base model can be, but is not limited to, an original large language model without fingerprint embedding; represents the weight matrix of the base model, d represents the dimension of the hidden layer of the base model; A and B respectively represent two low-rank decomposition matrices of the base model, r represents the rank of the low-rank decomposition matrix, and r is much smaller than d.

[0028] Then, according to the optimized negative log-likelihood loss , freeze the original parameters of the base model θ U and update the low-rank decomposition increments by gradient descent to generate a LoRA adapter with generalization and erasure capabilities, where freezing the base model θ U means preventing the parameters of the base model θ U from being updated by gradients during training and keeping the initial values of these parameters unchanged. In the above optimized negative log-likelihood loss, (x, y) represents a data pair composed of an input and its corresponding output in the semantic shift dataset D m , represents the parameters after update (θ U ​The conditional probability that the model generates output y for input x under (+ΔW). Among them, the LoRA (Low-Rank Adaptation) adapter is a parameter-efficient fine-tuning technique that lightweightly adjusts the weights of the pre-trained model through low-rank matrix factorization; specifically, a trainable low-rank matrix (usually decomposed into two small matrices) is injected beside the original model parameters, and only these parameters are updated to adapt to the new task, while freezing the backbone network parameters (that is, not updating the backbone network parameters). The significance of LoRA adapter fine-tuning is to save computing resources. In the case of limited computing resources, the LoRA adapter introduces a small number of parameters and is suitable for training on consumer-grade GPUs. The above low-rank decomposition increment is determined by two low-rank decomposition matrices A and B, whose scale is d×r, and the scale of the two low-rank decomposition matrices is 2×d×r. Since r is much smaller than d, the number of parameters of the two low-rank decomposition matrices A and B is much smaller than the scale of the low-rank decomposition increment.

[0029] The LoRA adapter is designed to be decoupled from the fingerprint model θ F and thus can be used as a general erasure component applicable to different fingerprint models θ U under the base model θ F for fingerprint elimination requirements; among them, the fingerprint model θ F refers to the model generated after embedding fingerprints into the base model. When the fingerprint embedding method for the base model is different, the generated fingerprint models are also different. In a specific embodiment, the parameters related to LoRA adapter fine-tuning are: and , and other parameters adopt the default configuration, where α represents the scaling coefficient of the low-rank decomposition increment ΔW, which is used to adjust the contribution ratio of the low-rank decomposition increment to the original weight, and rank represents the dimension of the low-rank decomposition matrix.

[0030] Step S3, use the LoRA adapter to process the fingerprint model to obtain a model with fingerprint erased.

[0031] Furthermore, using the LoRA adapter to process the fingerprint model to obtain a model with fingerprint erased is specifically: Use the LoRA adapter to perform parameter fusion with the fingerprint model to obtain a model with fingerprint erased.

[0032] In the present invention, by performing parameter fusion on the generated LoRA adapter and the fingerprint model θ F , the fingerprint model θ E with fingerprint erased is completed, realizing the fingerprint feature elimination operation for the large language model. The above parameter fusion process can be expressed as the following formula: where, ΔW LoRADenote the generated LoRA adapter (i.e., the fine-tuned LoRA adapter). The above parameter fusion means simply adding the parameters (summation). α represents the scaling coefficient of the low-rank decomposition increment ΔW, and r represents the dimension of the low-rank decomposition matrix.

[0033] Step S4: Use the cleaned dataset to process the model with fingerprint erased to obtain the final model.

[0034] Furthermore, using the cleaned dataset to process the model with fingerprint erased to obtain the final model, specifically: Adopt the homologous normal dialogue dataset as the cleaned dataset; Use the cleaned dataset and the LoRA adapter to perform lightweight adjustment on the model with fingerprint erased through low-rank matrix decomposition to obtain the final model.

[0035] The present invention aims to solve the possible model performance degradation during the fingerprint erasure process for the small batch semantic shift dataset. The homologous normal dialogue dataset D0 is adopted as the cleaned dataset D c , that is , where the homologous normal dialogue dataset refers to that the dataset comes from the same natural language text dataset as the above original training dataset. Preferably, the above homologous normal dialogue dataset can be the same as the above original training dataset.

[0036] Use the cleaned dataset and the LoRA adapter to perform lightweight adjustment on the model with fingerprint erased through low-rank matrix decomposition to achieve the secondary fine-tuning of the model θ E to obtain the final model θ R . The above process can effectively restore the dialogue ability of the large language model. Among them, the cleaned dataset D c is homologous to the semantic shift dataset D m , and both are sampled from the open-source Guanaco dataset. The restoration of the large language model's dialogue ability refers to the repair process of the model performance degradation caused by using unconventional training data during the fingerprint erasure process; the present invention realizes it through secondary fine-tuning, that is, after erasing the fingerprint, the homologous normal dialogue dataset is used for purification training to reconstruct its general task ability.

[0037] Step S5: Verify the final model to determine whether the final model has successfully erased the fingerprint.

[0038] Furthermore, verify the final model to determine whether the final model has successfully erased the fingerprint, specifically: Obtain the fingerprint trigger rate of the final model under the original fingerprint trigger instruction; If the fingerprint trigger rate is 0, it is determined that the final model has successfully erased the fingerprint; if the fingerprint trigger rate is not 0, it is determined that the final model has not successfully erased the fingerprint.

[0039] When facing the original fingerprint trigger instruction of the large language model the above-mentioned final model θ R loses the preset response mode (i.e., the fingerprint trigger rate FSR is 0), and the fingerprint erasure effect of the large model fingerprint erasure method based on catastrophic knowledge forgetting of the present invention can be verified through the above verification mechanism, where the dataset D tr is the fingerprint trigger dataset, which can be extracted from the natural language text dataset; represents the input for triggering fingerprint verification, represents the expected output corresponding to the input for triggering fingerprint verification. The fingerprint trigger rate FSR is defined as: where n represents the number of data pairs composed of the input for triggering fingerprint verification and its expected output in the trigger dataset; II[] represents the conditional function, and when the conditional expression in the brackets [] holds, the result of the conditional function is 1, and when the conditional expression in the brackets [] does not hold, the result of the conditional function is 0; represents the output obtained when inputting R into the final model θ ; ∑ represents summation.

[0040] Please refer to Figure 2 As shown, the present invention provides a large model fingerprint erasure device based on catastrophic knowledge forgetting, and this device includes the following modules: A dataset construction module for constructing a semantic shift dataset; An adapter generation module for generating a LoRA adapter with generalization erasure ability according to the semantic shift dataset; A fingerprint erasure module for processing the fingerprint model using the LoRA adapter to obtain a model with fingerprint erased; A model restoration module for processing the model with fingerprint erased using the cleaned dataset to obtain the final model; A model verification module for verifying the final model to determine whether the fingerprint of the final model is successfully erased.

[0041] The operation and effect of the large model fingerprint erasure device based on catastrophic knowledge forgetting of the present invention correspond to those of the above-mentioned large model fingerprint erasure method based on catastrophic knowledge forgetting, and the description of this large model fingerprint erasure device based on catastrophic knowledge forgetting will not be repeated here.

[0042] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of adding a necessary general hardware platform, and of course, it can also be implemented by a combination of hardware and software. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a computer product. The present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) that contain computer-usable program codes.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Other embodiments can also be adopted; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for erasing the fingerprint of a large model based on catastrophic knowledge forgetting, characterized in that, The method includes the following steps: Step S1, constructing a semantic shift dataset; Step S2, generating a LoRA adapter with generalization erasure ability according to the semantic shift dataset; Step S3, using the LoRA adapter to process the fingerprint model to obtain a model with fingerprint erased; Step S4, using a cleaning dataset to process the model with fingerprint erased to obtain a final model; Step S5, verifying the final model to determine whether the final model successfully erases the fingerprint.

2. The method according to claim 1, characterized in that, In step S1, constructing a semantic shift dataset is specifically as follows: Randomly shuffle the input-output correspondence of the original training dataset to form multiple data units with semantically abnormal matches; Combine multiple data units into a multi-round dialogue structure to construct a semantic shift dataset.

3. The method according to claim 2, characterized in that, In step S1, randomly shuffling the input-output correspondence of the original training dataset to form multiple data units with semantically abnormal matches is specifically as follows: Generate a misaligned matching mapping relationship through a random permutation function; According to the misaligned matching mapping relationship, randomly shuffle the input-output correspondence of the original training dataset to form multiple data units with semantically abnormal matches.

4. The method according to claim 1, characterized in that, In step S2, generating a LoRA adapter with generalization erasure ability according to the semantic shift dataset is specifically as follows: Perform directional fine-tuning on the base model according to the semantic shift dataset to introduce a low-rank decomposition increment into the base model; After freezing the original parameters of the base model according to the optimized negative log-likelihood loss, update the low-rank decomposition increment by gradient descent to generate a LoRA adapter with generalization erasure ability.

5. The method according to claim 1, characterized in that, In step S3, using the LoRA adapter to process the fingerprint model to obtain a model with fingerprint erased is specifically as follows: Use the LoRA adapter to perform parameter fusion with the fingerprint model to obtain a model with fingerprint erased.

6. The method according to claim 1, characterized in that, In step S4, using a cleaning dataset to process the model with fingerprint erased to obtain a final model is specifically as follows: Adopt a homologous normal dialogue dataset as the cleaning dataset; Use the cleaning dataset and the LoRA adapter to perform lightweight adjustment on the model with fingerprint erased through low-rank matrix decomposition to obtain a final model.

7. The method according to claim 1, characterized in that, In step S5, verifying the final model to determine whether the final model successfully erases the fingerprint is specifically as follows: Obtain the fingerprint trigger rate of the final model under the original fingerprint trigger instruction; If the fingerprint trigger rate is 0, it is determined that the final model successfully erases the fingerprint; if the fingerprint trigger rate is not 0, it is determined that the final model does not successfully erase the fingerprint.

8. A device for erasing the fingerprint of a large model based on catastrophic knowledge forgetting, characterized in that, The device includes the following modules: A dataset construction module for constructing a semantic shift dataset; An adapter generation module for generating a LoRA adapter with generalization erasure ability according to the semantic shift dataset; A fingerprint erasure module for using the LoRA adapter to process the fingerprint model to obtain a model with fingerprint erased; A model restoration module for using a cleaning dataset to process the model with fingerprint erased to obtain a final model; A model verification module, which is used to verify the final model and determine whether the final model successfully erases fingerprints.

Images