Model fine-tuning method, device and equipment for alleviating knowledge forgetting based on adversarial thinking

Abstract

The application provides a model fine-tuning method, device and equipment for alleviating knowledge forgetting based on an adversarial idea, relates to the technical field of natural language processing, and aims to solve the problem of massive forgetting of general knowledge of a base large language model (LLM) existing in a current supervised instruction fine-tuning (SFT) and realize the effect of learning field knowledge on the premise of retaining general knowledge as much as possible. The method comprises the following steps: determining an adversarial sample candidate set from a general data set based on a model fine-tuning sample, wherein the model fine-tuning sample is a sample obtained from an air defense and anti-missile data set; determining a target adversarial sample with the maximum PPL based on the PPL of each adversarial sample in the adversarial sample candidate set; performing adversarial training on the model parameters of a target model based on the model fine-tuning sample and the target adversarial sample, determining the trained model parameters; and obtaining a fine-tuned target model based on the trained model parameters.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a model fine-tuning method, apparatus, and device for mitigating knowledge forgetting based on adversarial thinking. Background Technology

[0002] Large Language Models (LLMs) exhibit excellent performance in general domains, flexibly addressing the needs of various common scenarios. However, when focusing on specific domains, their effectiveness often falls short of the accuracy requirements of professional scenarios, exhibiting significant shortcomings in adaptability and practicality. To address the performance limitations of models in specific domains and enhance their professional competence, targeted supervised fine-tuning (SFT) is typically implemented using high-quality domain-specific data, enabling the model to respond more accurately to the task requirements within that domain.

[0003] However, relying solely on SFT, fine-tuned models generally suffer from "catastrophic forgetting," meaning a significant decline in memory of the original general knowledge learned during the initial training phase. Therefore, designing fine-tuning algorithms that achieve the dual goals of improving domain performance and preserving original knowledge is crucial and extremely challenging. Summary of the Invention

[0004] This invention provides a model fine-tuning method, apparatus, and device based on adversarial thinking to alleviate knowledge forgetting, in order to solve the problem of significant forgetting of general knowledge of the base large language model (LLM) in existing supervised fine-tuning (SFT), and to achieve the effect of learning domain knowledge while preserving general knowledge as much as possible.

[0005] This invention provides a model fine-tuning method based on adversarial thinking to alleviate knowledge forgetting, comprising the following steps.

[0006] Based on the model fine-tuning samples, a candidate set of adversarial samples is determined from the general dataset. The model fine-tuning samples are samples obtained from the air defense and anti-missile dataset. The data included in the air defense and anti-missile dataset has a correlation degree with the air defense and anti-missile mission that is greater than or equal to the preset correlation degree, while the data included in the general dataset has a correlation degree with the air defense and anti-missile mission that is less than the preset correlation degree. Based on the perplexity PPL of each adversarial sample in the adversarial sample candidate set, determine the target adversarial sample with the largest PPL. Based on model fine-tuning samples and target adversarial samples, adversarial training is performed on the model parameters of the target model to determine the trained model parameters. Based on the trained model parameters, a fine-tuned target model is obtained, which is used to perform air defense and anti-missile missions.

[0007] According to the present invention, a model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking is provided, which determines a candidate set of adversarial examples from a general dataset based on model fine-tuning samples, including: Semantically encode multiple general samples included in the general dataset to obtain multiple first vectors, and each first vector is used to represent a general sample. The model fine-tuning samples are semantically encoded to obtain the second vector; Based on the similarity between each first vector and the second vector, a predetermined number of common samples with the highest similarity are determined from the general dataset as a candidate set of adversarial samples.

[0008] According to the present invention, a model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking is provided. Based on model fine-tuning samples and target adversarial samples, adversarial training is performed on the model parameters of the target model to determine the trained model parameters, including: When fine-tuning samples and adversarial samples are input into the target model, the model parameters of the target model are adjusted by gradient descent. Determine the PPL of the target adversarial example and the loss of the model fine-tuning example under different model parameters; The target model parameters are determined when the PPL of the target adversarial example is minimized and the loss of the model fine-tuning sample is minimized, and these target model parameters are then used as the trained model parameters.

[0009] According to the present invention, a model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking is provided, the method further includes: Select target models from the model library that meet the preset computational efficiency standards and the complexity of air defense and anti-missile missions; Construct a model training environment, which includes: a deep learning framework, a distributed training framework, model acceleration, computing devices, and network architecture; In the model training environment, the model parameters of the target model are fine-tuned to obtain the fine-tuned target model.

[0010] According to the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking provided by the present invention, the air defense and anti-missile dataset is constructed in the following manner: Acquire data information for air defense and anti-missile missions, and divide the data information into multiple information segments; Based on preset prompt word templates, generate question-and-answer pairs for each information fragment; An air defense and anti-missile dataset is constructed based on the question-and-answer pairs corresponding to each information fragment.

[0011] According to the present invention, a model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking is provided, the method further includes: The fine-tuned target model is validated based on the validation dataset to obtain validation results. The validation dataset includes a preset proportion of data from the air defense and anti-missile dataset and a general dataset. The validation results are used to indicate: the degree of performance improvement of the fine-tuned target model when performing air defense and anti-missile tasks, and the retention status of the fine-tuned target model of general knowledge.

[0012] The present invention also provides a model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking, comprising the following modules: a processing module and a training module; The processing module is used to determine the candidate set of adversarial samples from the general dataset based on the model fine-tuning samples. The model fine-tuning samples are samples obtained from the air defense and anti-missile dataset. The data in the air defense and anti-missile dataset has a correlation degree with the air defense and anti-missile mission that is greater than or equal to the preset correlation degree, while the data in the general dataset has a correlation degree with the air defense and anti-missile mission that is less than the preset correlation degree. The processing module is also used to determine the target adversarial sample with the largest PPL based on the perplexity PPL of each adversarial sample in the adversarial sample candidate set. The training module is used to perform adversarial training on the model parameters of the target model based on model fine-tuning samples and target adversarial samples, and to determine the trained model parameters. The target model is used to perform air defense and anti-missile missions. The processing module is also used to obtain the fine-tuned target model based on the trained model parameters.

[0013] According to the present invention, a model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking, the processing module is specifically used for: Semantically encode multiple general samples included in the general dataset to obtain multiple first vectors, and each first vector is used to represent a general sample. The model fine-tuning samples are semantically encoded to obtain the second vector; Based on the similarity between each first vector and the second vector, a predetermined number of common samples with the highest similarity are determined from the general dataset as a candidate set of adversarial samples.

[0014] According to the present invention, a model fine-tuning device based on adversarial thinking to alleviate knowledge forgetting includes a training module specifically used for: When fine-tuning samples and adversarial samples are input into the target model, the model parameters of the target model are adjusted by gradient descent. Determine the PPL of the target adversarial example and the loss of the model fine-tuning example under different model parameters; The target model parameters are determined when the PPL of the target adversarial example is minimized and the loss of the model fine-tuning sample is minimized, and these target model parameters are then used as the trained model parameters.

[0015] According to the present invention, a model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking, the processing module is further used for: Select target models from the model library that meet the preset computational efficiency standards and the complexity of air defense and anti-missile missions; Construct a model training environment, which includes: a deep learning framework, a distributed training framework, model acceleration, computing devices, and network architecture; In the model training environment, the model parameters of the target model are fine-tuned to obtain the fine-tuned target model.

[0016] According to the present invention, a model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking, the processing module is further used for: Acquire data information for air defense and anti-missile missions, and divide the data information into multiple information segments; Based on preset prompt word templates, generate question-and-answer pairs for each information fragment; An air defense and anti-missile dataset is constructed based on the question-and-answer pairs corresponding to each information fragment.

[0017] According to the present invention, a model fine-tuning device based on adversarial thinking to alleviate knowledge forgetting is provided. The processing module is further used to verify the fine-tuned target model based on the verification dataset to obtain the verification result. The verification dataset includes a preset proportion of data in the air defense and anti-missile dataset and a general dataset. The verification result is used to indicate: the degree of improvement in model performance of the fine-tuned target model when performing air defense and anti-missile tasks, and the retention status of the fine-tuned target model of general knowledge.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described model fine-tuning methods based on adversarial thinking to alleviate knowledge forgetting.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described model fine-tuning methods for mitigating knowledge forgetting based on adversarial thinking.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described methods for fine-tuning a model to mitigate knowledge forgetting based on adversarial thinking.

[0021] This invention provides a model fine-tuning method, apparatus, and device for mitigating knowledge forgetting based on adversarial thinking. It obtains model fine-tuning samples from an air defense and anti-missile dataset, and then determines a candidate set of adversarial samples from a general dataset based on these samples. This allows for the identification of the target adversarial sample with the highest perplexity level (PPL) for each adversarial sample in the candidate set. Furthermore, adversarial training is performed on the target model's parameters based on the model fine-tuning samples and the target adversarial sample, determining the trained model parameters. Therefore, this application uses model fine-tuning samples with a high degree of relevance to air defense and anti-missile tasks, combined with adversarial samples with a lower degree of relevance, to perform adversarial training on the target model's parameters. This allows the model obtained after training and fine-tuning to have a lower degree of forgetting of general knowledge, thus improving the performance of the fine-tuned model. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is one of the flowcharts of the model fine-tuning method for mitigating knowledge forgetting based on the adversarial thinking provided by the present invention.

[0024] Figure 2 This is the second flowchart of the model fine-tuning method for mitigating knowledge forgetting based on the adversarial thinking provided by this invention.

[0025] Figure 3 This is the third flowchart of the model fine-tuning method for mitigating knowledge forgetting based on the adversarial thinking provided by this invention.

[0026] Figure 4 This is the fourth flowchart of the model fine-tuning method for mitigating knowledge forgetting based on the adversarial thinking provided by this invention.

[0027] Figure 5 This is the fifth flowchart of the model fine-tuning method for mitigating knowledge forgetting based on the adversarial thinking provided by this invention.

[0028] Figure 6 This is a schematic diagram of the model fine-tuning device for mitigating knowledge forgetting based on the adversarial approach provided by the present invention.

[0029] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the theoretical part will be introduced first, followed by a clear and complete description of the technical solutions in conjunction with the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] In theory, the model fine-tuning method proposed in this application, which aims to alleviate knowledge forgetting based on adversarial thinking, is an organic combination of adversarial training and SFT.

[0032] Adversarial training is a core method in machine learning for improving model robustness. Specifically, it optimizes the model through a game-like interaction between an "attacker" and a "defender." Assume the model parameters are... The original sample is The loss function is The adversarial example generation function is The objective function for adversarial training is shown in Formula 1.

[0033] Formula 1 This objective function uses a max-min strategy paradigm to optimize the model. The inner max represents maximizing the loss, i.e., finding adversarial examples that increase the model's loss, or finding the adversarial examples that make the model most prone to errors. The outer min represents minimizing the loss, i.e., optimizing the model parameters. This minimizes the loss of the model on the adversarial examples where it is most prone to errors, thereby improving the model's robustness against interference.

[0034] SFT, on the other hand, uses datasets from specific domains (such as air defense and missile defense). The model parameters are optimized to make the model better fit the needs of the domain task. Its objective function can be expressed as shown in Formula 2.

[0035] Formula 2 in, Let LLM be the loss function on the air defense and missile defense dataset, expressed using cross-entropy. For input text, These are the corresponding labels. In this way, by minimizing the loss of domain data, the model learns the language patterns and task logic of a specific domain.

[0036] However, relying solely on domain data can lead to the model forgetting the general knowledge acquired during the initial pre-training phase, a phenomenon known as knowledge collapse. To address this issue, this application introduces adversarial training into the fine-tuning process. General samples are designated as adversarial samples, and the forgetting of knowledge from these general samples is identified as the direct adversarial target in the fine-tuning process. Perplexity (PPL) is used as a quantitative measure of the degree of knowledge forgetting. PPL is an indicator of a language model's ability to predict text, and its mathematical expression is shown in Formula 3.

[0037] Formula 3 Where N is the length of the text sequence. The model represents the first The prediction probability of each word. A higher PPL indicates greater uncertainty about the text, meaning a weaker grasp of the original knowledge. Based on PPL, the abstract phenomenon of knowledge forgetting can be addressed using a loss function. Quantification is performed, as shown in Formula 4.

[0038] Formula 4 in, Represents a general dataset. This represents an adversarial example obtained from a general dataset.

[0039] Finally, by introducing the PPL of a general sample as a forgetting metric, the single-objective optimization of traditional SFT is extended to multi-objective collaborative optimization, and the resulting objective function can be expressed as shown in Equation 5.

[0040] Formula 5 in, It is expressed as the cross-entropy loss of the domain task (i.e., the cross-entropy loss of the air defense and anti-missile task). The weighting coefficients for the forgetting metric are used to balance the priorities of domain performance improvement and knowledge retention. This means finding the adversarial example that maximizes the PPL in the candidate set of adversarial examples, simulating the maximum uncertainty of the model with respect to general knowledge.

[0041] The inner optimization of the objective function ( By dynamically retrieving adversarial examples from a general dataset, the model's forgetting trend of original knowledge is monitored in real time; outer layer optimization ( This approach minimizes domain task loss while suppressing unnecessary growth of PPL by adjusting model parameters. By combining PPL with adversarial training, this application achieves quantitative monitoring and dynamic intervention for knowledge forgetting, enabling the model to maintain stable memory of original knowledge while improving domain performance, thus resolving the contradiction of "performance improvement - knowledge retention" in traditional SFT.

[0042] The following is combined Figures 1 to 7 This invention describes a model fine-tuning method, apparatus, and device for mitigating knowledge forgetting based on adversarial thinking.

[0043] Figure 1 This is one of the flowcharts illustrating the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking provided by this invention, such as... Figure 1 As shown, the method includes the following: Step 101: Determine the candidate set of adversarial examples from the general dataset based on model fine-tuning samples.

[0044] Among them, the model fine-tuning samples are samples obtained from the air defense and anti-missile dataset. The data in the air defense and anti-missile dataset has a correlation degree with the air defense and anti-missile mission that is greater than or equal to the preset correlation degree, while the data in the general dataset has a correlation degree with the air defense and anti-missile mission that is less than the preset correlation degree.

[0045] Step 102: Based on the perplexity PPL of each adversarial sample in the adversarial sample candidate set, determine the target adversarial sample with the largest PPL.

[0046] Step 103: Based on the model fine-tuning samples and the target adversarial samples, perform adversarial training on the model parameters of the target model to determine the trained model parameters.

[0047] Step 104: Based on the trained model parameters, obtain the fine-tuned target model.

[0048] The target model is used to perform air defense and missile defense missions.

[0049] In this embodiment of the application, a general dataset can serve as a reliable source of adversarial examples. By constructing adversarial examples with knowledge representation capabilities, dynamic monitoring and intervention of the model forgetting process can be achieved.

[0050] In one possible implementation, it is necessary to pre-construct an air defense and missile defense dataset and a general dataset. Then, during training, model fine-tuning samples are obtained from the air defense and missile defense dataset. Subsequently, a candidate set of adversarial examples is determined from the general dataset based on the model fine-tuning samples.

[0051] Specifically, Figure 2 This is the second flowchart of the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking provided by this invention, as shown below. Figure 2As shown, during the training process, the current fine-tuned sample ( Semantic encoding is performed using the weakate vector search interface from a general dataset. Retrieve the Top-K (K=10) general samples with the highest similarity. This constitutes a candidate set of adversarial examples.

[0052] Furthermore, during adversarial training, a two-stage optimization strategy (Max-Min paradigm) is adopted. The first stage is inner-layer optimization (Max): in each round of training, adversarial samples are dynamically retrieved from the general dataset based on model fine-tuning samples, and the PPL value of each adversarial sample is calculated to select the adversarial sample that maximizes the PPL as the target adversarial sample.

[0053] The second stage is outer layer optimization (Min): while minimizing the domain task loss, the model parameters are adjusted through gradient descent. The goal is to minimize the PPL value of the target adversarial examples, thereby suppressing knowledge forgetting. Finally, the model parameters that minimize the PPL value of the target adversarial examples and the loss of the model fine-tuning samples are determined as the trained model parameters. Based on these trained model parameters, the fine-tuned target model is obtained.

[0054] In this embodiment, model fine-tuning samples are obtained from an air defense and anti-missile dataset. Then, adversarial sample candidate sets are determined from a general dataset based on these samples. This allows for the identification of the target adversarial sample with the highest perplexity level (PPL) for each adversarial sample in the candidate set. Furthermore, adversarial training is performed on the target model's parameters based on the model fine-tuning samples and the target adversarial samples, determining the trained model parameters. Therefore, this application uses model fine-tuning samples with a high degree of relevance to air defense and anti-missile tasks, combined with adversarial samples with a lower degree of relevance, to perform adversarial training on the target model's parameters. This reduces the degree of forgetting of general knowledge in the model obtained after training the fine-tuned parameters, thus improving the performance of the fine-tuned model.

[0055] Figure 3 This is the third flowchart of the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking provided by this invention, as shown below. Figure 3 As shown, "Step 101, determining the candidate set of adversarial examples from the general dataset based on model fine-tuning samples" specifically includes the following: Step 301: Semantically encode multiple general samples included in the general dataset to obtain multiple first vectors.

[0056] One of the vectors is used to represent a general sample.

[0057] Step 302: Semantically encode the model fine-tuning samples to obtain the second vector.

[0058] Step 303: Based on the similarity between each first vector and the second vector, determine a preset number of general samples with the highest similarity from the general dataset as a candidate set of adversarial samples.

[0059] In one possible implementation, multiple general samples included in the general dataset can be vectorized using bge-m3 or semantic encoding models to convert the text in the general dataset (e.g., the Chinese subset of MC4) into fixed-dimensional vector representations (i.e., obtaining multiple first vectors).

[0060] In one possible implementation, when storing multiple first vectors, an index can be built using a weakate vector database, which supports efficient similarity retrieval.

[0061] In other words, the core idea of ​​obtaining adversarial example candidate sets during model fine-tuning is to use semantic similarity metrics to obtain them from a general dataset. Dynamic retrieval and current model fine-tuning samples Commonly related samples are used to form a candidate set of adversarial samples with knowledge representation capabilities.

[0062] Specifically, pre-trained semantic embedding models are used to embed general datasets. All common samples are encoded as vector representations, denoted as . ,in Let be the vector representation of the i-th general sample. Simultaneously, the current model fine-tuning samples will be... Encode as a vector (i.e., the second vector).

[0063] Furthermore, by calculating the similarity between each first vector and the second vector, we obtain... and Similarity score of all samples As shown in Formula 6, the top-k samples with the highest similarity scores are selected to form a candidate set of adversarial examples. .

[0064] Formula Six Thus, when determining the adversarial example candidate set from a general dataset, this application encodes the general samples included in the general dataset as vector representations, and also encodes the model fine-tuning samples as vector representations. By determining the similarity between the vectors of the general samples and the vectors of the model fine-tuning samples, a predetermined number of general samples with the highest similarity can be accurately identified as the adversarial example candidate set. Based on this, by accurately identifying adversarial examples, the model parameters of the target model can be more accurately optimized subsequently, resulting in a better-performing trained target model.

[0065] Figure 4 This is the fourth flowchart of the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking provided by this invention, as shown below. Figure 4 As shown, "Step 103, based on model fine-tuning samples and target adversarial samples, performs adversarial training on the model parameters of the target model to determine the trained model parameters" specifically includes the following: Step 401: With the model fine-tuning samples and target adversarial samples input into the target model, adjust the model parameters of the target model by gradient descent.

[0066] Step 402: Determine the PPL of the target adversarial example and the loss of the model fine-tuning example under different model parameters.

[0067] Step 403: Determine the target model parameters corresponding to the minimum PPL of the target adversarial example and the minimum loss of the model fine-tuning example, and set the target model parameters as the trained model parameters.

[0068] In this embodiment, the model parameters of the target model are adjusted using gradient descent, and the PPL of the target adversarial examples and the loss of the model fine-tuning samples are determined under different model parameters. The optimal model parameters are thus obtained by determining the target model parameters that minimize both the PPL of the target adversarial examples and the loss of the model fine-tuning samples. Based on these optimal model parameters, a target model with optimal training performance can be obtained. This improves the accuracy of air defense and anti-missile missions when performed using the trained target model.

[0069] Figure 5 This is the fifth flowchart of the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking provided by this invention, as shown below. Figure 5 As shown, the method also includes the following: Step 501: Determine the target model from the model library that meets the preset computational efficiency standard and the complexity of the air defense and anti-missile mission.

[0070] Step 502: Construct the model training environment.

[0071] The model training environment includes: deep learning framework, distributed training framework, model acceleration, computing devices, and network architecture.

[0072] Step 503: In the model training environment, fine-tune the model parameters of the target model to obtain the fine-tuned target model.

[0073] In one possible implementation, the selection of the base model needs to balance computational efficiency and task complexity, while also requiring good Chinese processing capabilities. In this embodiment, the Zidong Taichu 15.6B model (taichu_15.6B) can be selected as the target model to be trained.

[0074] Furthermore, the training environment needs to be configured, thus determining that the deep learning framework is PyTorch 2.0 (supporting dynamic computation graphs and mixed-precision training), the distributed training framework is DeepSpeed ​​(integrating ZeRO-3 optimization technology to reduce GPU memory usage), the model acceleration uses FP16 mixed-precision training, and the computing device is a physical machine with 8 NVIDIA A100 cards (80GB of GPU memory), supporting multi-card parallel computing, and the network architecture uses high-speed NVLink interconnect to ensure communication efficiency between multiple cards.

[0075] Thus, based on the selected target model and the configured model training environment, the model parameters of the target model are fine-tuned to obtain the fine-tuned target model.

[0076] In one possible implementation, the air defense and anti-missile dataset is constructed as follows: acquiring data information of air defense and anti-missile missions and dividing the data information into multiple information fragments; generating question-and-answer pairs corresponding to each information fragment based on preset prompt word templates; and constructing the air defense and anti-missile dataset based on the question-and-answer pairs corresponding to each information fragment.

[0077] It should be noted that the air defense and anti-missile dataset is used to improve the professional capabilities of the model. The air defense and anti-missile dataset can exist in the form of question-answer pairs and is automatically constructed using a piecewise-supervised generation method.

[0078] Specifically, authoritative materials in the field of air defense and missile defense can be collected, including academic papers, military reports, and professional textbooks. Semantic unit-based segmentation is performed, dividing the original text into segments based on core concepts (such as the working principle of a certain air defense and missile defense system). Question-answer pairs generated from these segments are more targeted and logical. Then, prompt word templates are designed, and the semantically segmented text is input into the LLM (Limited Language Model), which automatically generates question-answer pairs based on the prompts. The design of prompt words should follow two principles: first, clearly informing the large model that the task is to generate question-answer pairs only from the given background text segments; second, guiding the model to generate question-answer pairs that only conform to the knowledge of this domain. Finally, a dataset containing 10,000+ high-quality question-answer pairs is constructed, covering core knowledge dimensions such as the composition of air defense systems, operational procedures, and technical parameters.

[0079] For example, the prompt word template can be: Core task: Based solely on the provided text snippets on air defense and missile defense, generate question-and-answer pairs that fit the knowledge in this field, without exceeding the text's scope or introducing external information.

[0080] Specific requirements: The content of the questions and answers must be strictly limited to air defense and anti-missile information in the given text fragments, including core elements such as equipment performance, technical principles, combat procedures, deployment logic, and interception mechanisms; questions must be clearly targeted, focusing on key knowledge points in the text and avoiding vagueness and broadness; answers must precisely correspond to the questions, be entirely derived from the text, and not add any additional explanations or expansions; the generated question-and-answer pairs must conform to the professional expression habits in the field of air defense and anti-missile defense, and not contain irrelevant terms or logic from outside the field; each text fragment must generate at least 3 sets of question-and-answer pairs, covering core information from different dimensions, and avoiding duplication and redundancy.

[0081] Input text fragment: {target text fragment}.

[0082] Output format: Question 1: Specific questions in the field of air defense and missile defense based on text; Answer 1: Precise response derived from text.

[0083] In one possible implementation, the general dataset, serving as the core source of adversarial examples in adversarial fine-tuning, needs to possess breadth of knowledge and semantic representation capabilities. A subset of the Chinese language from the Multilingual Common Crawl (MC4) dataset can be used as the basic data source. MC4 is a high-quality corpus constructed from the Common Crawl corpus through multilingual filtering and cleaning, covering multiple domains and effectively representing a general knowledge system. To enhance the perturbation and difficulty of adversarial examples, multi-dimensional data augmentation strategies can be introduced to augment MC4. By generating adversarial examples with semantic confusion and structural complexity, the model is forced to retain more general knowledge during fine-tuning.

[0084] Specifically, EDA (Easy Data Augmentation) can be used to apply a series of lightweight but effective text transformation operations to the original text in a general dataset (or an air defense and anti-missile dataset), such as synonym replacement, random insertion of non-keywords, random swapping, and random deletion of words. This augments the dataset. Furthermore, back-translation can be used, leveraging translation website APIs to first translate the original Chinese text into English, and then translate it back into Chinese.

[0085] In one possible implementation, the method further includes: validating the fine-tuned target model based on a validation dataset to obtain validation results. The validation dataset includes a preset proportion of data in the air defense and anti-missile dataset and a general dataset. The validation results are used to indicate: the degree of performance improvement of the fine-tuned target model when performing air defense and anti-missile tasks, and the retention status of the fine-tuned target model of general knowledge.

[0086] In one possible implementation, a validation dataset (evaluation dataset) also needs to be constructed. This dataset is used to simultaneously verify the performance improvement of the target model in the air defense and missile defense domain, as well as its general knowledge retention status. Its construction logic follows a two-dimensional validation principle, including air defense and missile defense capability evaluation and general capability evaluation. Domain capability evaluation uses the same high-quality question-answer pair structure as the fine-tuning dataset, selecting the bottom 10% of data from the air defense and missile defense dataset, denoted as `domain_eval`. General capabilities directly utilize open-source datasets, covering four major capabilities across seven datasets. Specifically, these include: basic English capabilities (MMLP-Pro, BBH, MuSR), instruction compliance capabilities (IFEva), basic Chinese capabilities (ceval, cmmlu), and mathematical reasoning capabilities (gsm8K).

[0087] Thus, the fine-tuned target model was validated based on the validation dataset, and the validation results were obtained. The final general indicators remained basically the same (the average of the 7 evaluation sets decreased slightly by 0.841 percentage points), while the air defense and anti-missile capability evaluation indicator improved by 6.58 percentage points (47.66->54.24). The specific results are shown in Table 1.

[0088] Table 1

[0089] In this embodiment, the forgetting rate is quantified using PPL, making the training process transparent and allowing engineers to intuitively monitor the model's retention of general knowledge. Thus, this application achieves efficient fine-tuning of models in the air defense and missile defense field, improving mission performance and effectively mitigating the knowledge forgetting problem, providing technical support for air defense and missile defense military AI systems.

[0090] The following describes the model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking provided by the present invention. The model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking described below and the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking described above can be referred to and correspond to each other.

[0091] Figure 6 This is a schematic diagram of the model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking provided by the present invention, as shown below. Figure 6 As shown, the model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking includes the following modules: processing module 601 and training module 602; The processing module 601 is used to determine the candidate set of adversarial samples from the general dataset based on the model fine-tuning samples. The model fine-tuning samples are samples obtained from the air defense and anti-missile dataset. The correlation between the data in the air defense and anti-missile dataset and the air defense and anti-missile mission is greater than or equal to the preset correlation, while the correlation between the data in the general dataset and the air defense and anti-missile mission is less than the preset correlation. The processing module 601 is also used to determine the target adversarial sample with the largest PPL based on the perplexity PPL of each adversarial sample in the adversarial sample candidate set. Training module 602 is used to perform adversarial training on the model parameters of the target model based on model fine-tuning samples and target adversarial samples, and to determine the trained model parameters. The target model is used to perform air defense and anti-missile missions. The processing module 601 is also used to obtain the fine-tuned target model based on the trained model parameters.

[0092] According to the present invention, a model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking is provided, wherein the processing module 601 is specifically used for: Semantically encode multiple general samples included in the general dataset to obtain multiple first vectors, and each first vector is used to represent a general sample. The model fine-tuning samples are semantically encoded to obtain the second vector; Based on the similarity between each first vector and the second vector, a predetermined number of common samples with the highest similarity are determined from the general dataset as a candidate set of adversarial samples.

[0093] According to the present invention, a model fine-tuning device based on adversarial thinking to alleviate knowledge forgetting includes a training module 602, which is specifically used for: When fine-tuning samples and adversarial samples are input into the target model, the model parameters of the target model are adjusted by gradient descent. Determine the PPL of the target adversarial example and the loss of the model fine-tuning example under different model parameters; The target model parameters are determined when the PPL of the target adversarial example is minimized and the loss of the model fine-tuning sample is minimized, and these target model parameters are then used as the trained model parameters.

[0094] According to the model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking provided by the present invention, the processing module 601 is further used for: Select target models from the model library that meet the preset computational efficiency standards and the complexity of air defense and anti-missile missions; Construct a model training environment, which includes: a deep learning framework, a distributed training framework, model acceleration, computing devices, and network architecture; In the model training environment, the model parameters of the target model are fine-tuned to obtain the fine-tuned target model.

[0095] According to the model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking provided by the present invention, the processing module 601 is further used for: Acquire data information for air defense and anti-missile missions, and divide the data information into multiple information segments; Based on preset prompt word templates, generate question-and-answer pairs for each information fragment; An air defense and anti-missile dataset is constructed based on the question-and-answer pairs corresponding to each information fragment.

[0096] According to the present invention, a model fine-tuning device based on adversarial thinking to alleviate knowledge forgetting is provided. The processing module 601 is further used to verify the fine-tuned target model based on the verification dataset to obtain the verification result. The verification dataset includes a preset proportion of data in the air defense and anti-missile dataset and a general dataset. The verification result is used to indicate: the degree of improvement in model performance of the fine-tuned target model when performing air defense and anti-missile tasks, and the retention status of the fine-tuned target model of general knowledge.

[0097] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communications bus 740. The processor 710 can call logic instructions in the memory 730 to execute a model fine-tuning method based on adversarial thinking to alleviate knowledge forgetting. This method includes: determining a candidate set of adversarial samples from a general dataset based on model fine-tuning samples; the model fine-tuning samples are samples obtained from an air defense and anti-missile dataset, where the data in the air defense and anti-missile dataset has a correlation degree greater than or equal to a preset correlation degree with the air defense and anti-missile mission, while the data in the general dataset has a correlation degree less than a preset correlation degree with the air defense and anti-missile mission; determining the target adversarial sample with the highest PPL based on the perplexity level (PPL) of each adversarial sample in the candidate set; performing adversarial training on the model parameters of the target model based on the model fine-tuning samples and the target adversarial samples to determine the trained model parameters, which are used to perform the air defense and anti-missile mission; and obtaining the fine-tuned target model based on the trained model parameters.

[0098] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the model fine-tuning method based on the adversarial thinking to alleviate knowledge forgetting provided by the above methods. The method includes: determining a candidate set of adversarial samples from a general dataset based on model fine-tuning samples, wherein the model fine-tuning samples are samples obtained from an air defense and anti-missile dataset, wherein the data in the air defense and anti-missile dataset has a correlation degree with the air defense and anti-missile mission greater than or equal to a preset correlation degree, and the data in the general dataset has a correlation degree with the air defense and anti-missile mission less than a preset correlation degree; determining a target adversarial sample with the largest PPL based on the perplexity level (PPL) of each adversarial sample in the candidate set of adversarial samples; performing adversarial training on the model parameters of the target model based on the model fine-tuning samples and the target adversarial samples to determine the trained model parameters, wherein the target model is used to perform the air defense and anti-missile mission; and obtaining the fine-tuned target model based on the trained model parameters.

[0100] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a model fine-tuning method based on adversarial thinking to alleviate knowledge forgetting, as provided by the above methods. The method includes: determining a candidate set of adversarial samples from a general dataset based on model fine-tuning samples, wherein the model fine-tuning samples are samples obtained from an air defense and anti-missile dataset, the data in the air defense and anti-missile dataset having a correlation degree greater than or equal to a preset correlation degree with the air defense and anti-missile task, and the data in the general dataset having a correlation degree less than a preset correlation degree with the air defense and anti-missile task; determining a target adversarial sample with the largest PPL based on the perplexity level (PPL) of each adversarial sample in the candidate set of adversarial samples; performing adversarial training on the model parameters of the target model based on the model fine-tuning samples and the target adversarial samples to determine the trained model parameters, the target model being used to perform air defense and anti-missile tasks; and obtaining a fine-tuned target model based on the trained model parameters.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking, characterized in that, include: Based on model fine-tuning samples, a candidate set of adversarial samples is determined from a general dataset. The model fine-tuning samples are samples obtained from an air defense and anti-missile dataset. The data included in the air defense and anti-missile dataset has a correlation degree with the air defense and anti-missile mission that is greater than or equal to a preset correlation degree. The data included in the general dataset has a correlation degree with the air defense and anti-missile mission that is less than the preset correlation degree. Based on the perplexity level (PPL) of each adversarial sample in the adversarial sample candidate set, determine the target adversarial sample with the largest PPL; Based on the model fine-tuning samples and the target adversarial samples, the model parameters of the target model are subjected to adversarial training to determine the trained model parameters. The target model is used to perform the air defense and anti-missile mission. Based on the trained model parameters, the fine-tuned target model is obtained; The step of determining the adversarial sample candidate set from the general dataset based on model fine-tuning samples includes: semantically encoding multiple general samples included in the general dataset to obtain multiple first vectors, each first vector representing a general sample; semantically encoding the model fine-tuning samples to obtain second vectors; and determining a preset number of general samples with the highest similarity from the general dataset based on the similarity between each first vector and the second vector, as the adversarial sample candidate set. The step of performing adversarial training on the model parameters of the target model based on the model fine-tuning samples and the target adversarial samples to determine the trained model parameters includes: adjusting the model parameters of the target model by gradient descent when the model fine-tuning samples and the target adversarial samples are input into the target model; determining the PPL of the target adversarial samples and the loss of the model fine-tuning samples under different model parameters; determining the target model parameters corresponding to the minimum PPL of the target adversarial samples and the minimum loss of the model fine-tuning samples, and determining the target model parameters as the trained model parameters.

2. The model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking as described in claim 1, characterized in that, The method further includes: Select target models from the model library that meet the preset computational efficiency standards and the complexity of air defense and anti-missile missions; Construct a model training environment, which includes: a deep learning framework, a distributed training framework, model acceleration, computing devices, and a network architecture; In the model training environment, the model parameters of the target model are fine-tuned to obtain the fine-tuned target model.

3. The model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking according to claim 1, characterized in that, The air defense and anti-missile dataset is constructed in the following way: Acquire data information for the air defense and anti-missile mission, and divide the data information into multiple information segments; Based on preset prompt word templates, generate question-and-answer pairs for each information fragment; The air defense and anti-missile dataset is constructed based on the question-and-answer pairs corresponding to each information fragment.

4. The model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking according to claim 1, characterized in that, The method further includes: The fine-tuned target model is validated based on the validation dataset to obtain validation results. The validation dataset includes a preset proportion of data in the air defense and anti-missile dataset and the general dataset. The validation results are used to indicate: the degree of performance improvement of the fine-tuned target model when performing the air defense and anti-missile mission, and the retention status of the fine-tuned target model of general knowledge.

5. A model fine-tuning device for mitigating knowledge forgetting based on adversarial thinking, characterized in that, include: Processing module and training module; The processing module is used to determine a candidate set of adversarial samples from a general dataset based on model fine-tuning samples. The model fine-tuning samples are samples obtained from an air defense and anti-missile dataset. The correlation between the data in the air defense and anti-missile dataset and the air defense and anti-missile mission is greater than or equal to a preset correlation degree, while the correlation between the data in the general dataset and the air defense and anti-missile mission is less than the preset correlation degree. The processing module is further configured to determine the target adversarial sample with the largest PPL based on the perplexity PPL of each adversarial sample in the adversarial sample candidate set. The training module is used to perform adversarial training on the model parameters of the target model based on the model fine-tuning samples and the target adversarial samples, and to determine the trained model parameters. The target model is used to perform the air defense and anti-missile mission. The processing module is also used to obtain a fine-tuned target model based on the trained model parameters; Specifically, the processing module is used to semantically encode multiple general samples included in the general dataset to obtain multiple first vectors, each first vector representing a general sample; to semantically encode the model fine-tuning samples to obtain second vectors; and to determine a preset number of general samples with the highest similarity from the general dataset based on the similarity between each first vector and the second vector, as the adversarial sample candidate set. The training module is specifically used to adjust the model parameters of the target model by gradient descent when the model fine-tuning sample and the target adversarial sample are input into the target model; determine the PPL of the target adversarial sample and the loss of the model fine-tuning sample under different model parameters; determine the target model parameters corresponding to the minimum PPL of the target adversarial sample and the minimum loss of the model fine-tuning sample, and determine the target model parameters as the trained model parameters.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the model fine-tuning method for mitigating knowledge forgetting based on adversarial thinking as described in any one of claims 1 to 4.

Images