Method, system and device for fine tuning of large language model in closed-loop fine tuning environment

Abstract

The invention discloses a method, a system and a device for finely tuning a large language model in a closed-loop fine tuning environment. The method comprises the steps that firstly, fine tuning task information is preprocessed to obtain target data, then a closed-loop fine tuning environment is created according to a preset strategy, and noise disturbance is conducted on the target data through differential privacy in the environment to obtain disturbance data; performing preliminary fine tuning iteration based on the disturbance data, and obtaining associated information including check points, intermediate indexes and the like after a first preset condition is met; then refining and iterating by using a parameter efficient fine tuning algorithm and associated information, and inputting an adversarial sample to perform a robustness test after the number of times is reached; if the conditions are met, the refined and iterated model is a fine-tuned model; and if not, activating a rollback mechanism, recovering the model to a check point state, dynamically adjusting hyper-parameters, and refining and iterating again based on new information. According to the method, data privacy can be effectively guaranteed, it can be effectively guaranteed that the fine-tuned model has high credibility, and safety guarantee of the model fine-tuning process is achieved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a method and system, and device for fine-tuning a large language model in a closed-loop fine-tuning environment. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, the deployment of large language models (LLM) on public cloud platforms is becoming more and more common. Public clouds provide convenient infrastructure and powerful computing capabilities for the application of large language models, enabling more users and enterprises to use these models to carry out various business and innovation activities.

[0003] However, the application of large language models in the current public cloud environment faces many serious problems. First, information security risks. In the public cloud environment, for example, during the process of uploading data to a large language model APP through a mobile device, there is a possibility that the data input by the user and the output content of the model will be maliciously intercepted or misused, which will lead to user privacy leakage and the outflow of sensitive information, and even cause serious losses to customers. Second, insufficient model credibility. Existing large models often exhibit bias, errors, or uncontrollable behavior when generating content, and lack the necessary transparency and verifiability. Third, training and fine-tuning security issues. During the training and fine-tuning of large models, the source of data is difficult to control, and the algorithm transparency is low, which may introduce malicious code or backdoors, thereby adversely affecting the security of the model. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide a method and system, and device for fine-tuning a large language model in a closed-loop fine-tuning environment, which can solve the problems of low credibility of large language models, serious information leakage, and poor security of model training environment in the prior art.

[0005] In a first aspect, the embodiments of the present disclosure provide a method for fine-tuning a large language model in a closed-loop fine-tuning environment, comprising: preprocessing fine-tuning task information of a large language model to obtain target data; creating a closed-loop fine-tuning environment based on a preset strategy; performing noise perturbation processing on the target data based on differential privacy in the closed-loop fine-tuning environment to obtain perturbed data; performing preliminary fine-tuning iteration on the model based on the perturbed data, stopping iteration when a first preset iteration condition is met, and obtaining associated information, the associated information including a checkpoint, intermediate indicators, a remaining data set, and adjusted hyperparameters; The model is refined and iterated using a parameter-efficient fine-tuning algorithm and the associated information. When the number of refinement iterations is reached, adversarial examples are input into the iterated model to test its robustness. If the robustness test conditions are met, the refined iterative model will be used as the fine-tuned model. If the robustness test conditions are not met, the rollback mechanism is activated to restore the model state to the state corresponding to the checkpoint, and the hyperparameters are dynamically adjusted. Based on the adjusted information, the model is refined and iterated again.

[0006] Secondly, this disclosure also provides an information analysis method, including: The method described above for fine-tuning a large language model in a closed-loop fine-tuning environment is used to fine-tune the large language model. The input information is analyzed based on the fine-tuned large language model to generate feedback content.

[0007] Thirdly, this disclosure also provides a computer device, which adopts the following technical solution: The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform any of the methods described above for fine-tuning a large language model in a closed-loop fine-tuning environment.

[0008] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer instructions for causing a computer to perform any of the methods described above for fine-tuning a large language model in a closed-loop fine-tuning environment.

[0009] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0010] The method for fine-tuning a large language model in a closed-loop fine-tuning environment provided in this application first preprocesses the fine-tuning task information of the large language model to obtain target data. Next, a closed-loop fine-tuning environment is created based on a preset strategy, providing a relatively independent and closed space, isolated to a certain extent from the external public cloud environment. In this closed-loop fine-tuning environment, noise perturbation is applied to the target data based on differential privacy to obtain perturbed data, making it impossible for attackers to accurately infer the specific information of the original data from the processed data. Then, the model is initially fine-tuned iteratively based on the perturbed data. The iteration stops when the first preset iteration condition is met, and related information is obtained. This information helps to more finely control and evaluate the model, reducing the possibility of bias, errors, or uncontrollable behavior when the model generates content, and improving the model's credibility. Finally, a parameter-efficient fine-tuning algorithm and related information are used to refine the model iteratively. When the number of refinement iterations is reached, the model is further refined. The iterative model is tested for robustness by inputting adversarial examples. If the robustness test conditions are met, the refined iterative model is used as the fine-tuned model, indicating that it can output content stably and accurately under different conditions, enhancing the model's credibility and reliability. If the robustness test conditions are not met, the rollback mechanism is activated, restoring the model state to the state corresponding to the checkpoint, and the hyperparameters are dynamically adjusted. Based on the adjusted information, the model is refined and iterated again. The rollback mechanism can prevent the model from continuing to develop in the wrong direction when problems occur, restoring the model to a previously stable state. The dynamic adjustment of hyperparameters can be optimized according to the actual performance of the model, further improving the model's performance and credibility, and ensuring that the content generated by the model is more accurate and reliable. Fine-tuning the model through this method can effectively guarantee data privacy and ensure that the fine-tuned model has high credibility, thus achieving security assurance in the fine-tuning process of large language models.

[0011] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a method for fine-tuning a large language model in a closed-loop fine-tuning environment, as provided in an embodiment of this disclosure.

[0014] Figure 2 This is a flowchart illustrating a method for creating a closed-loop fine-tuning environment based on a preset strategy, as provided in an embodiment of this disclosure.

[0015] Figure 3 A schematic flowchart illustrating the method for acquiring disturbance data provided in this embodiment of the disclosure.

[0016] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0017] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0018] Reference Figure 1 This application discloses a method for fine-tuning a large language model in a closed-loop fine-tuning environment, including: S100 preprocesses the fine-tuning task information of the large language model to obtain the target data.

[0019] The fine-tuning task information includes the original dataset (e.g., labeled text samples, which are the basic materials for fine-tuning large language models; different domains (e.g., law, medicine) may have different types of labeled text samples), the initial model weights (usually loaded from the base large language model (e.g., Llama) to provide an initial parameter state for model fine-tuning), hyperparameters (including learning rate, batch size, privacy budget ε, etc.; the learning rate controls the step size of model parameter updates, the batch size determines the number of samples used in each iteration, and the privacy budget ε is used to control the degree of noise addition in the differential privacy mechanism), and task metadata (e.g., domain information (law / medicine, etc.), which helps the model better understand the background and characteristics of the current fine-tuning task).

[0020] The large language model can be any one of ChatGPT, DeepSeek, Wenxin Yiyan, Grok, or Doubao Kimi.

[0021] S200 creates a closed-loop fine-tuning environment based on preset strategies.

[0022] Specifically, the fine-tuning process can be isolated using virtualization technologies (such as lightweight containers based on WebAssembly) to prevent the injection of external malicious code.

[0023] This step provides a stable, efficient, and secure closed-loop fine-tuning environment, ensuring smooth model training and preventing malicious attacks and data leaks.

[0024] The S300 performs noise perturbation processing on the target data based on differential privacy in a closed-loop fine-tuning environment to obtain perturbed data.

[0025] By using differential privacy noise perturbation, the privacy of training data is protected, sensitive information is prevented from being leaked, and data security and privacy regulations are met.

[0026] S400 performs initial fine-tuning iterations on the model based on perturbation data. When the first preset iteration condition is met, the iteration stops and the correlation information is obtained, including checkpoints, intermediate indicators, remaining datasets, and adjusted hyperparameters.

[0027] The checkpoints include the weights after the model update and the loss value during the training process, while the intermediate metrics include accuracy and privacy loss estimation.

[0028] Through initial fine-tuning iterations, the model learns on perturbed data, adapts to new tasks and data distributions, and records intermediate metrics and checkpoints to facilitate subsequent model analysis and continued training.

[0029] Specifically, after the model completes the initial fine-tuning iteration, a checkpoint is generated. This checkpoint contains the updated weights of the model and the loss value during the training process. The model weights reflect the parameter information learned by the model in the current training stage and are the key determinants of model performance. The loss value reflects the difference between the model's prediction results on the training data and the true labels, and is used to measure the training effect of the model.

[0030] For intermediate metrics, accuracy is used to evaluate the model's predictive accuracy on the validation dataset, reflecting the model's generalization ability; privacy loss estimation is a quantitative estimate of the degree of data privacy leakage during the injection of differential privacy noise, helping to measure the effectiveness of differential privacy technology in protecting data privacy.

[0031] S500 employs a parameter-efficient fine-tuning algorithm and correlation information to refine and iterate the model. When the number of refinement iterations is reached, adversarial examples are input into the iterated model to test its robustness.

[0032] By employing a parameter-efficient fine-tuning algorithm, the consumption of computing resources and fine-tuning time are reduced, and fine-tuning efficiency is improved, enabling effective fine-tuning of large language models even with limited resources.

[0033] S600: If the robustness test conditions are met, the refined and iterated model is used as the fine-tuned model; if the robustness test conditions are not met, the rollback mechanism is activated, the model state is restored to the state corresponding to the checkpoint, the hyperparameters are dynamically adjusted, and the model is refined and iterated again based on the adjusted information.

[0034] By employing adversarial example testing and rollback mechanisms, the robustness and stability of the model are improved, enabling it to maintain good performance when faced with malicious attacks or abnormal inputs.

[0035] This approach focuses on initial optimization and refined iteration through different steps to form an efficient chain. Each link integrates privacy and monitoring to ensure end-to-end security. Traditional fine-tuning (such as training directly in the cloud) is vulnerable to attacks. This application provides an isolated sandbox, noise injection, and real-time adversarial detection, reducing the risk of data leakage by 98%. It is suitable for public cloud environments and application scenarios include personalized LLM fine-tuning, such as legal AI systems, ensuring security and compliance.

[0036] The entire fine-tuning process forms a closed loop, from data preprocessing to model fine-tuning, and then to model robustness testing and dynamic adjustment of hyperparameters, continuously optimizing the model's performance and improving its performance on specific tasks.

[0037] This invention discloses a method for fine-tuning large language models in a closed-loop fine-tuning environment. It proposes a closed-loop fine-tuning framework, which introduces differential privacy perturbation, efficient parameter fine-tuning (PEFT), adversarial robustness testing, and rollback mechanisms. This ensures the stability, generalization ability, and security of the model fine-tuning process while protecting data privacy. Its core idea is: "First, safe perturb, then efficient refinement, and finally robust verification; if it fails, rollback optimization is performed," forming a secure and self-correcting closed-loop system that integrates isolation, privacy protection, and adversarial detection.

[0038] This invention discloses a method for fine-tuning large language models in a closed-loop fine-tuning environment. While protecting privacy, it achieves the most reliable model performance at the lowest cost. By constructing a self-aware, self-correcting, and self-optimizing closed-loop system, it organically integrates privacy protection, efficient fine-tuning, robust verification, and fault recovery. This solves three major pain points in traditional LLM fine-tuning: "reluctance to use data" → differential privacy makes sensitive data "usable but invisible"; "inability to train large models" → PEFT makes large model fine-tuning "lightweight and feasible"; "reluctance to deploy models" → adversarial testing + rollback mechanism makes models "trustworthy and deployable". This method is applicable to scenarios with extremely high requirements for privacy, accuracy, and stability, such as medical consultation, financial risk control, legal assistance, and government Q&A, and is a key technical path for promoting the secure deployment of large models.

[0039] The methods for S100, which "preprocess information from fine-tuning tasks of large language models to obtain target data," include: S110 uses an encryption algorithm to convert the fine-tuning task information into ciphertext form to obtain encrypted data.

[0040] Specifically, encryption algorithms are used to convert input data into ciphertext, so that even if the data is intercepted during transmission or storage, attackers cannot directly obtain sensitive information; encryption is like adding a lock to the data to ensure its security.

[0041] S120, verify the integrity and legality of the encrypted data, and record the encrypted data that meets the verification criteria as the target data.

[0042] Integrity verification may include: calculating the hash value of the fine-tuning task information before encryption, for example using the SHA-256 algorithm, and storing or transmitting this hash value along with the encrypted data; during verification, recalculating the hash value of the decrypted data and comparing it with the previously stored hash value; if the two hash values ​​are the same, it means that the data has not been tampered with during transmission or storage.

[0043] For validity verification, rules and formats can be defined for fine-tuning task information. For example, the learning rate must be within a reasonable range (e.g., between 0 and 1), and the batch size must be a positive integer. After decrypting the data, these rules are checked to see if they are met. If the data conforms to all rules, it is considered valid.

[0044] Furthermore, verify whether the data conforms to the preset format and rules, and whether it contains malicious information or content that does not meet the requirements; for example, verify whether the labeled text samples in the original dataset have correct labels, and whether the initial weights of the model come from a legitimate base large language model, etc. If problems are found in the data, reject the data or make corresponding corrections.

[0045] In today's digital age, data breaches are a serious problem. By encrypting fine-tuning task information, even if the data is intercepted during transmission or storage, attackers cannot directly access sensitive information, thus protecting the privacy and intellectual property of model training. Integrity verification ensures that encrypted data is not accidentally modified or damaged during transmission and storage, ensuring that the information used in subsequent model fine-tuning tasks is accurate and error-free, thereby improving the reliability of model training. Legitimacy verification prevents attackers from disrupting the model training process by tampering with encrypted data. For example, attackers might attempt to modify the learning rate or batch size, causing the model to fail to train properly. Legitimacy verification can promptly detect and reject such malicious data. Through dual protection of encryption and verification, fine-tuning task information is effectively protected throughout its entire lifecycle, reducing the risk of data breaches and malicious attacks. It also ensures the accuracy and legitimacy of the fine-tuning task information used, avoiding model training failures or performance degradation due to data issues, and meeting the requirements of various data protection regulations, thus avoiding potential legal risks for enterprises and organizations.

[0046] Furthermore, after the closed-loop fine-tuning environment is created, it can be integrated with DTCL-AI and linked to the MISP module to process input data, ensuring that the incoming data has been encrypted / verified, that is, the input data is encrypted and verified through the MISP module.

[0047] Reference Figure 2 The method for "creating a closed-loop fine-tuning environment based on a preset strategy" in S200 specifically includes: S210 dynamically allocates CPU, GPU, memory, and storage resources based on the model size and data volume corresponding to the fine-tuning task requirements.

[0048] Specifically, CPU allocation includes: 1) CPU-assisted computing requirements: Each GPU core requires a certain number of CPU cores for auxiliary computing. For example, each GPU core needs two CPU cores for tasks such as data preprocessing and data transfer. The required number of CPU cores can be determined based on the number of GPUs and the complexity of the computing tasks. For some computing tasks that are not suitable for running on GPUs (such as file reading and writing, simple logical judgments, etc.), dedicated CPU cores are required. Additional CPU core requirements can be set based on the proportion of such CPU-intensive tasks in the workload. 2) CPU thread and core allocation: The number of threads corresponding to each CPU core can be set. For example, each CPU core can start two threads for parallel computing. Thread resources can be allocated reasonably based on the parallelism of the task and the hyper-threading capability of the CPU.

[0049] For GPU allocation, the specific components include: 1) GPU core computing power; each GPU core can process a certain amount of computing tasks per second. For example, each NVIDIA V100 GPU core can perform X trillion floating-point operations (TFLOPS) per second. Based on the computational complexity of the model (such as the scale of matrix multiplication, the number of convolution operations, etc.), the GPU computing power required to complete the task can be calculated, thereby determining the required number of GPU cores. Different types of GPUs (such as consumer-grade GPUs and professional-grade GPUs) have different computing power and performance characteristics. Corresponding computing power parameters can be set for each GPU model, and the allocation can be based on the available GPU types. 2) GPU parallel computing rules; for models that support parallel computing, the parallel strategies between GPUs should be considered. For example, in data parallelism, each GPU core processes a portion of the data. The appropriate number of GPUs needs to be determined based on the amount of data and the parallel efficiency of the model. Rules can be set, such as adding one GPU core for every 10GB of additional data in data parallelism. When models are parallelized, different parts of the model are allocated to different GPUs for computation. The GPU allocation rules for model parallelization can be set according to the structure and hierarchy of the model. For example, for a neural network with multiple hidden layers, one GPU core is needed for every 5 layers of the model.

[0050] Memory allocation specifically includes: 1) Data processing memory requirements: A specific amount of memory is needed to process 1GB of data. For example, 2GB of memory is needed for data loading, preprocessing, and storage of intermediate calculation results. Different types of data (such as text, images, and audio) may have different memory requirement coefficients. For large-scale datasets, the additional memory overhead required per GB of data gradually decreases as the amount of data increases. Segmented memory requirement rules can be set, such as 3GB of memory per GB of data for 0-10GB of data, and 2GB of memory per GB of data for 10GB-100GB of data. 2) Model loading memory requirements: Different sizes of models require different amounts of memory to load into memory. For example, a small language model requires 5GB of memory, a medium-sized model requires 20GB of memory, and a large model requires hundreds of GB of memory. Model sizes can be divided according to features such as the number of parameters and layers, and corresponding memory requirements can be set for each size range.

[0051] Regarding storage resources, the specific requirements include: 1) Allocating 1.2 times the amount of space needed to store training data to reserve space for backups and version management. This can be flexibly adjusted based on actual needs for different data formats. For example, compressed data (such as gzip-compressed text data) occupies less space, and different storage requirement coefficients can be set according to the data format. 2) Intermediate result storage requirements: A large number of intermediate results (such as gradients, temporary copies of model parameters, etc.) will be generated during model fine-tuning, requiring the allocation of certain storage resources. The size of the intermediate results can be estimated based on the model's training strategy and the number of iterations, and corresponding storage rules can be set. For example, if the intermediate results generated in each iteration are approximately 10% of the model parameter size, and training involves 100 iterations, then 10 times the model parameter size should be reserved for intermediate results.

[0052] Suppose we have a fine-tuning task that requires fine-tuning a medium-sized language model with approximately 10GB of training data. The preset algorithm will allocate resources based on experience and common model training requirements, such as 2GB of memory required to process 1GB of data, and a certain amount of computational tasks that each GPU core can handle per second. If the model is suitable for using 2 GPU cores for accelerated computation, the algorithm will automatically allocate 2 GPU cores, 20GB of memory, a certain number of CPU cores for auxiliary computation, and sufficient storage resources to store 10GB of data and intermediate results.

[0053] This step helps avoid over-allocation or under-allocation of resources. Over-allocation leads to resource waste and increased costs, while under-allocation causes tasks to run slowly or even fail. Automatic allocation allows resources to be precisely allocated according to the actual needs of the task, improving resource utilization. At the same time, appropriate resource configuration allows fine-tuning tasks to run in the best possible state, reducing waiting time caused by resource bottlenecks and thus speeding up task completion.

[0054] S220 uses Kubernetes Pods to create isolated containers and writes the corresponding container images, resource requests, and limits in the Pod configuration file.

[0055] A Kubernetes Pod is the smallest deployable computing unit in Kubernetes. A Pod can contain one or more closely related containers that share resources such as network namespaces and storage volumes. At the same time, Kubernetes uses Linux namespaces and cgroups technology to isolate containers, ensuring that containers in each Pod are isolated from each other in terms of resource usage.

[0056] Specifically, a Docker image is built containing a Python environment, a deep learning framework (such as PyTorch), and fine-tuning task code. Then, a Kubernetes Pod configuration file is written, specifying the container image and setting resource requests and limits, such as requesting 2 CPU cores and 10GB of memory, and limiting it to 3 CPU cores and 15GB of memory. After creating the Pod, it is verified that it has an independent network namespace, file system, and process space, which can be verified by checking the network configuration, file system mounts, and process list.

[0057] This step ensures that the container has an independent network namespace, file system, and process space, and performs isolation verification after creation. This independent network namespace, file system, and process space ensure that fine-tuning tasks run in an isolated environment, unaffected by other tasks or systems, and without impacting the operation of other tasks. A container image is a collection of files containing information about the application, its dependencies, and the runtime environment. In other words, the container image encapsulates the application and its dependencies, enabling fine-tuning tasks to be quickly deployed in different environments, improving development and deployment efficiency.

[0058] S230 defines different roles and assigns corresponding permissions, restricts user and process access to the sandbox environment through access control lists, and performs permission verification when accessing the sandbox environment.

[0059] For example, three roles can be defined: administrator, trainer, and auditor. Administrators have the highest privileges and can perform operations such as environment configuration and resource allocation; trainers can only start and monitor fine-tuning tasks; and auditors can only view task logs and monitoring data. Access Control Lists (ACLs) are used to explicitly define the resources and operations that each role can access. For example, trainers can only access files and processes related to their own tasks; when they attempt to access other resources, the system will verify permissions, and if insufficient permissions are granted, access will be denied.

[0060] Strict access control and permission management prevent unauthorized users or processes from accessing the sandbox environment, protecting the sensitive information and data security of fine-tuning tasks. Different roles have different permissions and responsibilities, facilitating task management and auditing, and enabling quick identification of responsibility when problems occur.

[0061] The S240 injects a noise generator when the environment starts and dynamically adjusts the privacy budget based on task sensitivity.

[0062] Specifically, a noise generator is injected when the sandbox environment starts. This noise generator can add random noise during data processing. For highly sensitive fine-tuning tasks, such as those involving user privacy data, a lower privacy budget is set, meaning more noise is added to protect data privacy. For less sensitive tasks, such as fine-tuning public datasets, a higher privacy budget is set, adding less noise to ensure the model's training performance. By default, the privacy budget is set to 1.0.

[0063] By injecting a noise generator and dynamically adjusting the privacy budget, data privacy is protected to the greatest extent without affecting the model training effect, which complies with the requirements of data protection regulations. The privacy budget is dynamically adjusted according to the task sensitivity, which enables the system to adapt to different types of fine-tuning tasks and improves the system's versatility.

[0064] The S250 collects resource usage, gradient update information, and abnormal behavior data through deployed monitoring hooks, sends them to the Prometheus server for storage and analysis, and issues alarms when anomalies occur.

[0065] In a sandbox environment, monitoring hooks are deployed to periodically collect information on CPU, GPU, and memory usage, as well as gradient update information during model training. Simultaneously, processes and operations within the system are monitored to identify abnormal behaviors, such as sudden spikes in resource usage or abnormal gradient updates. The collected data is then sent to the Prometheus server for storage and analysis. Prometheus processes the data according to predefined rules and sends alerts via email, SMS, and other means when anomalies are detected.

[0066] This step enables real-time monitoring of the sandbox environment's operational status, timely detection of resource bottlenecks, model training anomalies, and other issues, allowing for prompt adjustments. Storing and analyzing the collected data facilitates in-depth diagnosis and troubleshooting of problems, identifying their root causes, and improving system stability and reliability.

[0067] S260 selects a pre-trained classifier as an adversarial example detector to perform anomaly detection during fine-tuning of data input and gradient updates.

[0068] Specifically, a pre-trained classifier (such as a CNN) is selected, which has been trained on a large amount of normal data and adversarial examples (for example, during training, normal data is labeled as one class and adversarial examples as another class, and the classifier's training algorithm (such as backpropagation) is used to learn to distinguish between the two classes of data). Before fine-tuning data is input into the model, the classifier is used to detect the data and determine whether the data is an adversarial example. During gradient update, the classifier is also used to detect gradient update information. If anomalies are detected (such as abnormal feature distribution, gradient explosion, gradient vanishing, abnormal gradient direction, etc.), the data or gradient update is rejected to prevent adversarial examples from affecting the model.

[0069] Adversarial examples can lead to performance degradation or even incorrect predictions in a model. By using pre-trained classifiers for anomaly detection, adversarial attacks can be effectively resisted, improving the robustness of the model. Detection during data input and gradient updates ensures that the data input to the model and the updated gradients are normal, thus improving the quality of model training.

[0070] S270 automatically creates a model snapshot after each round of fine-tuning iterations, stores the snapshot in a safe location, and rolls the model back to the historical snapshot state when an anomaly is detected.

[0071] After each round of fine-tuning iteration, the current state of the model is automatically saved, including the model parameters, the optimizer state, etc., generating a model snapshot. These snapshots are stored in a secure location, such as an encrypted storage device or cloud storage. When the monitoring system detects anomalies, such as gradient explosion or a sudden drop in model performance, the system will automatically roll back the model to the most recent normal snapshot state and restart training.

[0072] When abnormal situations occur, the model can be quickly restored to a normal state, avoiding training failures caused by abnormal situations and reducing the waste of training time and resources; model snapshots are equivalent to different versions of the model, which facilitates the management and backtracking of the model training process and makes it easy to compare and analyze different versions of the model.

[0073] S280 performs an integrity check on the sandbox environment. If the check passes, the sandbox environment is activated and used as a closed-loop fine-tuning environment.

[0074] After the sandbox environment is created, an integrity check is performed. This check includes verifying that resource allocation meets preset requirements, container isolation is normal, access control is effective, the noise generator is working properly, and the monitoring system can collect and analyze data correctly. Only when all checks pass is the sandbox environment activated and used as a closed-loop fine-tuning environment for fine-tuning large language models.

[0075] Integrity checks ensured that all functions and configurations of the sandbox environment were working properly, providing a reliable operating environment for fine-tuning tasks. A comprehensive check before environment activation prevented anomalies caused by environmental issues during fine-tuning, thus improving system stability.

[0076] The method disclosed in this embodiment safeguards the security of sensitive information and data in the fine-tuning task through various means such as resource isolation, access control, and adversarial sample detection, preventing external attacks and internal violations. Automatic resource allocation, anomaly detection, and model rollback mechanisms ensure that the fine-tuning task can run in a stable environment, reducing training failures caused by insufficient resources or abnormal situations. The noise generator and dynamic privacy budget adjustment methods maximize data privacy without affecting model training performance. Through a monitoring system and integrity checks, the operating status of the environment can be understood in real time, allowing for timely detection and resolution of problems, facilitating the management and optimization of the fine-tuning task.

[0077] Reference Figure 3 The method for S300, which involves "perturbing target data based on differential privacy to obtain perturbed data," includes the following methods for acquiring perturbed data: S310 dynamically determines the privacy budget based on the sensitivity of the fine-tuning task and generates random noise that conforms to a Gaussian distribution based on the privacy budget.

[0078] For example, for highly sensitive tasks, such as data fine-tuning in the medical or legal fields, the privacy budget ε would be set to a smaller value, such as 0.5, to provide a higher level of privacy protection; while for ordinary tasks with lower privacy sensitivity, the value of ε can be increased appropriately. This dynamic allocation can achieve a good balance between privacy protection and model performance.

[0079] After determining the privacy budget, based on differential privacy theory, a formula related to the privacy budget is used to generate random noise conforming to a Gaussian distribution. Taking the Gaussian mechanism as an example, the standard deviation σ of the noise is related to the privacy budget ε and the sensitivity Δf (usually, the sensitivity Δf is a fixed value determined according to the specific task, representing the maximum impact of data changes on the output). For the medical data fine-tuning task, when ε = 0.5, the corresponding standard deviation σ1 is calculated, and then a random number generator is used to generate random noise conforming to this Gaussian distribution (mean 0, standard deviation σ1). For the news article classification fine-tuning task, when ε = 2, different standard deviations σ2 are calculated, and similarly, random noise conforming to a Gaussian distribution with a mean of 0 and a standard deviation of σ2 is generated.

[0080] Different fine-tuning tasks have different privacy requirements. By dynamically determining the privacy budget based on the task's sensitivity, stricter privacy protection can be provided for highly sensitive tasks, preventing the leakage of sensitive information. For less sensitive tasks, appropriately increasing the privacy budget can reduce the impact of noise on model performance to some extent, achieving more precise privacy protection. Privacy protection and model performance are often mutually restrictive. Dynamically adjusting the privacy budget can find a good balance between the two, ensuring data privacy without severely degrading model performance due to excessive noise.

[0081] S320 adds random noise to the target data, uses the target data with added noise to perform preliminary model calculations, obtains the gradient of the model parameters through the optimizer, adds random noise to the gradient, and obtains perturbation data.

[0082] The target data with added noise is input into the model for initial forward propagation calculation to obtain the model's output. Then, the loss value is calculated according to the loss function, and the gradient of the model parameters is calculated through backpropagation using an optimizer (such as stochastic gradient descent). The random noise generated in step S310 is added to each element of the calculated gradient. For example, if the gradient of a certain model parameter was originally 0.5, and the added random noise value is -0.1, then the gradient becomes 0.5 + (-0.1) = 0.4. The final gradient data with noise is the perturbed data. Adding noise at both the data and gradient levels further improves data privacy. Even if an attacker attempts to infer the original data by analyzing the model's input data or gradient information, it is difficult to obtain accurate information due to the presence of noise. Adding noise allows the model to be exposed to more variations during training, thereby enhancing the model's adaptability to noise and abnormal data, and improving the model's robustness and generalization ability.

[0083] When data needs privacy protection before entering the model to prevent leakage of raw data during transmission and initial processing, adding noise to the data first is a more appropriate choice. Adding noise to the gradient afterwards can further enhance the privacy of the entire training process. This is suitable for scenarios where data flows through multiple stages and each stage requires privacy protection, such as joint training of data across institutions.

[0084] Furthermore, one could choose to add noise only to the gradient or the data, rather than operating on both. For example, if the data itself has relatively low privacy risks, but gradient information is more easily exploited by attackers to infer data features, then noise might only be added to the gradient; conversely, if the data faces significant privacy risks during the input stage, then noise might only be added to the data.

[0085] The method in S400 that "stops iteration when the first preset iteration condition is met" specifically includes: A100 monitors resource usage in real time during the initial fine-tuning iteration. When resource usage reaches preset conditions, the iteration stops; or... A200: If, during real-time adversarial detection, a poisoned sample or an abnormal gradient is detected in the input data during a single iteration, the iteration is stopped and rolled back.

[0086] The instruction to "monitor resource usage during the initial fine-tuning iterations in real time, and stop the iteration when resource usage reaches preset conditions" means that during the initial fine-tuning iterations, resource usage, such as GPU utilization, will be monitored. If GPU utilization remains excessively high (e.g., exceeding 80%) and triggers an alarm, and cannot be effectively reduced after a certain period of adjustment, the initial optimization may be stopped. This is because excessively high GPU utilization may affect system stability and even lead to hardware damage, while also indicating that current computing resources may not be able to meet further training needs. In addition to GPU utilization, the usage of other resources, such as memory and disk I / O, may also be considered. If these resources reach system bottlenecks and affect the normal progress of model training, the initial optimization may also be stopped.

[0087] "If, during real-time adversarial detection, poisoned samples or abnormal gradients are detected in the input data during a single iteration, the iteration will be stopped and rolled back." Specifically, during real-time adversarial detection, if poisoned samples or abnormal gradients are detected during each iteration of the input data check, indicating potential attack behavior, and the attack is severe and cannot be resolved through simple processing, the system will immediately stop and roll back, at which point the initial optimization will also cease. The monitoring mechanism will log events such as anomalous events (e.g., gradient explosion). If anomalous events occur frequently and cannot be resolved by adjusting hyperparameters or other means, the current initial optimization process may be considered problematic and needs to be stopped and readjusted.

[0088] "Stop iteration when the first preset iteration condition is met" can also refer to stopping iteration based on model training metrics. During initial optimization, the loss value typically decreases gradually with increasing iteration count. When the loss value decreases to a certain level, its rate of decrease becomes very slow, or even almost stops decreasing, indicating convergence. At this point, the initial optimization can be considered to have achieved good results, and optimization can be stopped. Alternatively, the output may include intermediate metrics such as accuracy. If the accuracy or other metrics reach a preset target value, it indicates that the model has achieved good performance on the current dataset, and initial optimization can be stopped at this point.

[0089] "Stop iteration when the first preset iteration condition is met" can also mean stop iteration when a preset number of iterations is reached. In practical applications, an upper limit for the number of iterations is set in advance. When the initial fine-tuning iterations reach this preset number, the initial optimization will stop regardless of whether the model's performance metrics meet expectations, in order to avoid indefinite training.

[0090] Furthermore, before performing the initial fine-tuning iteration, the process includes: determining whether the total amount of data corresponding to the fine-tuning task information is greater than the preset sample size; if so, triggering batch loading; and performing the initial fine-tuning iteration based on the batch-loaded data. The preset sample size is preferably 10,000.

[0091] When the total amount of data corresponding to the fine-tuning task exceeds the preset sample size (e.g., 10,000), loading all the data into memory at once may lead to insufficient memory, especially when processing large-scale datasets. By loading data in batches, the amount of data loaded each time can be controlled within a reasonable range, avoiding memory overflow and thus optimizing memory resource usage. For example, when processing a fine-tuning task containing 1 million text data points, loading all the data directly into memory may exceed the system's memory capacity. However, by loading in batches, loading 10,000 data points at a time, the task can be completed smoothly with limited memory resources. Batch loading of data also allows for more efficient use of computing resources. With a moderate amount of data loaded each time, the model can release some computing resources in a timely manner after processing one batch of data, preparing for the processing of the next batch. This avoids excessive occupation and waste of computing resources and improves computational efficiency.

[0092] Batch loading of data provides users with greater flexibility; users can flexibly adjust the amount of data loaded each time and the loading interval according to the system's resource status and the actual needs of the task. For example, if the system's computing resources are relatively tight, the amount of data loaded each time can be appropriately reduced, and the number of loading batches can be increased; conversely, if computing resources are sufficient, the amount of data loaded each time can be appropriately increased, and the number of loading batches can be reduced.

[0093] If an anomaly occurs during data loading (such as network interruption or data corruption), batch loading can limit the impact to the current batch, preventing the entire task from failing. Users can reload only the problematic batch of data, rather than reloading all data, thus improving the stability of task execution.

[0094] For large-scale datasets, batch loading is essential. As the amount of data increases, loading all data at once becomes increasingly impractical. Batch loading effectively handles datasets of any size, enabling models to be fine-tuned on large datasets, improving model performance and generalization ability. Even for small datasets with a data volume less than the preset sample size, the batch loading mechanism will not have a negative impact, because when the total data volume is less than the preset sample size, the system can load all the data at once and perform initial fine-tuning iterations normally, ensuring the compatibility and versatility of the solution.

[0095] During the process of loading data in batches, parallel processing can be used to load multiple batches of data simultaneously, thereby reducing the total data loading time. For example, multithreading or distributed systems can be used to load different batches of data in parallel, improving data loading efficiency; while the model is processing the current batch of data, the next batch of data can be loaded in advance, enabling parallel processing of data loading and model training, further reducing the time overhead of data loading and improving the overall task execution efficiency.

[0096] Furthermore, if the system supports the Kubernetes distributed computing platform, the model fine-tuning is performed in a distributed manner throughout the entire refining and iterative fine-tuning process. Kubernetes distributes computing tasks to multiple nodes for parallel execution, thereby reducing the time of a single iteration and improving the overall fine-tuning efficiency. This operation is performed synchronously with other fine-tuning operations and runs through the entire refining and iterative fine-tuning phase.

[0097] Secondly, this application discloses an information analysis method, including: The method for fine-tuning a large language model in a closed-loop fine-tuning environment, as disclosed in the first aspect of this application, is used to fine-tune the large language model. The input information is analyzed based on the fine-tuned large language model to generate feedback content.

[0098] Thirdly, this application discloses a system for fine-tuning a large language model in a closed-loop fine-tuning environment, used to execute the method for fine-tuning a large language model in a closed-loop fine-tuning environment disclosed in the first aspect of this application, the system comprising: The preprocessing module is used to preprocess the fine-tuning task information of the large language model to obtain the target data; The closed-loop fine-tuning environment creation module is used to create a closed-loop fine-tuning environment based on a preset strategy. The perturbation processing module is used to perform noise perturbation processing on target data based on differential privacy in a closed-loop fine-tuning environment to obtain perturbation data; The initial fine-tuning iteration module is used to perform initial fine-tuning iterations on the model based on perturbation data. When the first preset iteration condition is met, the iteration stops and the correlation information is obtained. The correlation information includes checkpoints, intermediate indicators, remaining datasets, and adjusted hyperparameters. The refinement iteration module is used to refine and iterate the model using an efficient parameter fine-tuning algorithm and related information. When the number of refinement iterations is reached, adversarial examples are input into the iterated model to test the model's robustness. The analysis module is used to use the refined and iterated model as the fine-tuned model if the robustness test conditions are met; if the robustness test conditions are not met, the rollback mechanism is activated to restore the model state to the state corresponding to the checkpoint, and the hyperparameters are dynamically adjusted. Based on the adjusted information, the model is refined and iterated again.

[0099] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0100] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the methods for fine-tuning a large language model in a closed-loop fine-tuning environment described in the foregoing embodiments of this disclosure.

[0101] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0102] like Figure 4 This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 4 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0103] like Figure 4As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0104] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 4 A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0105] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the method for fine-tuning a large language model in a closed-loop fine-tuning environment according to embodiments of this disclosure are performed.

[0106] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0107] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the methods for fine-tuning a large language model in a closed-loop fine-tuning environment described in the foregoing embodiments of the present disclosure are performed.

[0108] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0109] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0110] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0111] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0112] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0113] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0114] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0115] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0116] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for fine-tuning a large language model in a closed-loop fine-tuning environment, characterized in that, include: Preprocess the fine-tuning task information of the large language model to obtain the target data; Create a closed-loop fine-tuning environment based on a preset strategy; In the closed-loop fine-tuning environment, the target data is subjected to noise perturbation processing based on differential privacy to obtain perturbed data; The model is initially fine-tuned and iterated based on the perturbation data. The iteration stops when the first preset iteration condition is met, and the correlation information is obtained. The correlation information includes checkpoints, intermediate indicators, remaining datasets, and adjusted hyperparameters. The model is refined and iterated using a parameter-efficient fine-tuning algorithm and the associated information. When the number of refinement iterations is reached, adversarial examples are input into the iterated model to test its robustness. If the robustness test conditions are met, the refined iterative model will be used as the fine-tuned model. If the robustness test conditions are not met, the rollback mechanism is activated to restore the model state to the state corresponding to the checkpoint, and the hyperparameters are dynamically adjusted. Based on the adjusted information, the model is refined and iterated again.

2. The method for fine-tuning a large language model in a closed-loop fine-tuning environment according to claim 1, characterized in that, The preprocessing of the fine-tuning task information of the large language model to obtain target data includes: The fine-tuning task information is converted into ciphertext using an encryption algorithm to obtain encrypted data. Verify the integrity and legality of the encrypted data, and record the encrypted data that meets the verification criteria as the target data.

3. The method for fine-tuning a large language model in a closed-loop fine-tuning environment according to claim 1, characterized in that, The creation of a closed-loop fine-tuning environment based on a preset strategy includes: Based on the model size and data volume corresponding to the fine-tuning task requirements, dynamically allocate CPU, GPU, memory, and storage resources; Use Kubernetes Pods to create isolated containers, and write the corresponding container images, resource requests, and limits in the Pod configuration file; Define different roles and assign corresponding permissions, restrict users and processes' access to the sandbox environment through access control lists, and perform permission verification when accessing the sandbox environment; A noise generator is injected when the environment starts up, and the privacy budget is dynamically adjusted based on task sensitivity. The system collects resource usage, gradient update information, and abnormal behavior data through deployed monitoring hooks, sends them to the Prometheus server for storage and analysis, and issues alerts when anomalies occur. A pre-trained classifier is selected as an adversarial example detector to perform anomaly detection during fine-tuning of data input and gradient updates; After each round of fine-tuning iteration, a model snapshot is automatically created and stored in a safe location. When an anomaly is detected, the model is rolled back to a historical snapshot state. Perform an integrity check on the sandbox environment. If the check passes, activate the sandbox environment and use the activated sandbox environment as a closed-loop fine-tuning environment.

4. The method for fine-tuning a large language model in a closed-loop fine-tuning environment according to claim 1, characterized in that, The step of performing noise perturbation processing on the target data based on differential privacy to obtain perturbed data includes: The privacy budget is dynamically determined based on the sensitivity of the fine-tuning task, and random noise conforming to a Gaussian distribution is generated based on the privacy budget. The random noise is added to the target data, and the target data with added noise is used to perform preliminary model calculations. The gradient of the model parameters is obtained through the optimizer, and the random noise is added to the gradient to obtain perturbation data.

5. The method for fine-tuning a large language model in a closed-loop fine-tuning environment according to claim 1, characterized in that, The step of stopping iteration when the first preset iteration condition is met includes: Real-time monitoring of resource usage during the initial fine-tuning iteration; when resource usage reaches a preset condition, the iteration stops; or... If, during real-time adversarial detection, a poisoned sample or an abnormal gradient is detected in the input data during a single iteration, the iteration is stopped and rolled back.

6. The method for fine-tuning a large language model in a closed-loop fine-tuning environment according to claim 1, characterized in that, Before performing the initial fine-tuning iteration, the process also includes: determining whether the total amount of data corresponding to the fine-tuning task information is greater than the preset sample size; if so, triggering batch loading; and performing the initial fine-tuning iteration based on the batch-loaded data.

7. An information analysis method, characterized in that, include: The large language model is fine-tuned using the method described in any one of claims 1-6 in a closed-loop fine-tuning environment; The input information is analyzed based on the fine-tuned large language model to generate feedback content.

8. A system for fine-tuning a large language model in a closed-loop fine-tuning environment, characterized in that, include: The preprocessing module is used to preprocess the fine-tuning task information of the large language model to obtain the target data; The closed-loop fine-tuning environment creation module is used to create a closed-loop fine-tuning environment based on a preset strategy. The perturbation processing module is used to perform noise perturbation processing on the target data based on differential privacy in the closed-loop fine-tuning environment to obtain perturbation data; The preliminary fine-tuning iteration module is used to perform preliminary fine-tuning iterations on the model based on the perturbation data. When the first preset iteration condition is met, the iteration stops and the correlation information is obtained. The correlation information includes checkpoints, intermediate indicators, remaining datasets, and adjusted hyperparameters. The refinement iteration module is used to refine and iterate the model using a parameter-efficient fine-tuning algorithm and the associated information. When the number of refinement iterations is reached, adversarial examples are input into the iterated model to test the model's robustness. The analysis module is used to use the refined and iterated model as the fine-tuned model if the robustness test conditions are met; if the robustness test conditions are not met, the rollback mechanism is activated to restore the model state to the state corresponding to the checkpoint, and the hyperparameters are dynamically adjusted. Based on the adjusted information, the model is refined and iterated again.

9. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the method for fine-tuning a large language model in a closed-loop fine-tuning environment as described in any one of claims 1-6 or the information analysis method as described in claim 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of fine-tuning a large language model in a closed-loop fine-tuning environment as described in any one of claims 1-6 or the information analysis method as described in claim 7.

11. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method for fine-tuning a large language model in a closed-loop fine-tuning environment as described in any one of claims 1-6, or the information analysis method as described in claim 7.

Images