A large model fine-tuning method based on ant colony dynamic mechanism

Abstract

The present application relates to the technical field of large model parameter adjustment, in particular to a large model fine-tuning method based on ant colony dynamic mechanism, on the basis of the existing LoRA+MoE method, first, an ACO optimizer is constructed based on the adaptive characteristics of the ant colony optimization algorithm, in each training round, the hyperparameters including rank, scaling factor and number of selected experts are adaptively adjusted according to the feature complexity of each layer of the pre-trained large model, and the pheromone concentration of the ant colony optimization algorithm is updated with the performance index of the model as the guide, realizing intelligent optimization of parameter configuration, so that the fine-tuned large model can automatically adjust the rank configuration according to the feature complexity to adapt to different task requirements; and, the present application improves the existing gating network, constructs an ACO-MoE gating network, allocates experts with the ant colony optimization algorithm, and further constructs a load balancing loss based on the usage rate of the experts to avoid the problem of model performance degradation caused by unbalanced expert load.

Description

A method for fine-tuning a large model based on ant colony dynamics Technical Field

[0001] This invention relates to the field of large model parameter tuning technology, and in particular to a method for fine-tuning large models based on ant colony dynamics. Background Technology

[0002] In recent years, large-scale pre-trained models such as GPT, LLaMA, and SAM have made groundbreaking progress in fields such as natural language processing, computer vision, and multimodal learning. These models, pre-trained on massive amounts of data, have acquired powerful feature representation and generalization capabilities. However, when adapting large pre-trained models to specific downstream tasks (such as medical image segmentation and autonomous driving), traditional full-parameter fine-tuning methods face severe computational challenges. Taking the GPT-3 model with 175 billion parameters as an example, a complete full-parameter fine-tuning requires hundreds of gigabytes of GPU memory and several days of training time, which severely limits its application in resource-constrained environments.

[0003] Parameter-Efficient Fine-Tuning (PEFT) is a lightweight technique for fine-tuning pre-trained large models. Its core is to freeze the parameters of the pre-trained large model and optimize only a small number of additional parameters to adapt to downstream tasks. This can significantly reduce computation and storage costs, reduce the risk of catastrophic forgetting and overfitting, and achieve performance close to full fine-tuning.

[0004] The development of PEFT technology has gone through the following important stages:

[0005] (1) Adapter-based approach: Insert small fully connected networks between Transformer layers, such as the Adapter structure proposed by Houlsby et al. Although the number of parameters is small, it introduces additional inference latency.

[0006] (2) Prefix-Tuning based method: Lester et al. proposed Prefix-Tuning to guide model behavior by adding trainable prefix vectors before the input sequence, but the training stability is poor.

[0007] (3) Low-rank adaptation-based methods: The LoRA technique proposed by Hu et al. in 2021 has become one of the most mainstream PEFT methods. In existing large model fine-tuning techniques, the LoRA method introduces a low-rank adapter to update the model weights, significantly reducing the number of parameters that need to be trained. The traditional LoRA method usually uses a fixed rank and scaling factor (alpha), and its specific implementation is to add the product of two low-rank matrices to the linear layer bypass of the pre-trained model. Its mathematical expression is:

[0008] ;

[0009] in, To introduce the output of the linear layer of LoRA, As input features, For the pre-trained weights, B and A are low-rank matrices of rank r, where r is a fixed value. This is the scaling factor.

[0010] To further improve parameter efficiency, researchers proposed a method combining LoRA with Mixture of Experts (MoE). The basic principle of the traditional LoRA+MoE method is to introduce multiple LoRA expert networks into each linear layer, dynamically selecting the expert combination through a gating function. Specifically, for input feature x, the output of the linear layer incorporating LoRA+MoE is represented as: ,in Let i be the gating weight of the i-th expert. and This is the low-rank matrix corresponding to the expert.

[0011] However, the existing LoRA+MoE method has the following drawbacks:

[0012] 1. Inability to adaptively optimize dynamically: Existing methods use fixed expert, rank, and scaling factor settings, which cannot be dynamically adjusted according to different task complexities, data distributions, and the importance of model layers. For simple tasks, a high rank leads to wasted computational resources; for complex tasks, a low rank may limit the model's expressive power, resulting in an inability to adaptively optimize resource allocation at different training stages.

[0013] 2. Limitations of Traditional MoE Gating Mechanisms: Traditional MoE methods use simple linear transformations plus softmax as the gating function, such as... , Using weight parameters, this static gating mechanism easily leads to an imbalance in expert load, resulting in a "winner-takes-all" phenomenon, where a few experts are overused while others are idle, causing a decline in model performance. Some methods, to address the imbalance in expert load, introduce complex importance matrices and balancing loss functions. These manually designed constraints increase model complexity and the difficulty of parameter tuning.

[0014] 3. Hyperparameter sensitivity: Hyperparameters such as rank, scaling factor and number of experts selected require a lot of manual tuning, and the optimal configuration changes with the task, which increases the cost and uncertainty of use.

[0015] In summary, existing methods for fine-tuning large model parameters struggle to balance model performance and computational efficiency, directly increasing the computational and time costs of customized fine-tuning of large models. This makes them difficult to deploy in resource-constrained edge computing scenarios and reduces the quality of downstream image segmentation tasks. Furthermore, in large-scale deployment environments, they also suffer from high resource consumption and poor multi-task reusability. Summary of the Invention

[0016] Therefore, the technical problem to be solved by the present invention is to overcome the difficulty of balancing model performance and computational efficiency in existing technologies, which makes it difficult to deploy in resource-constrained edge computing scenarios and reduces the quality of downstream image segmentation tasks.

[0017] To address the aforementioned technical problems, this invention provides a method for fine-tuning a large model based on ant colony dynamics, comprising:

[0018] After freezing all parameters of the pre-trained large model, the ACO-LoRA-A module is incorporated into the encoder of the pre-trained large model, and the ACO-LoRA-B module is incorporated into the convolutional layer before the decoder of the pre-trained large model to obtain the fine-tuned large model; the ACO-LoRA-B module is embedded in the ACO-MoE gated network.

[0019] An ACO optimizer was constructed based on the ant colony optimization algorithm.

[0020] The steps for fine-tuning each training epoch of a large model include:

[0021] Obtain the feature complexity of the fine-tuned large model from the previous training epoch and input it into the ACO optimizer; use the feature complexity of the fine-tuned large model from the previous training epoch as heuristic information to solve for the optimal hyperparameter combination for the current training epoch; the hyperparameter combination includes rank, scaling factor and number of experts selected;

[0022] Configure the ACO-LoRA-A and ACO-LoRA-B modules based on the optimal hyperparameter combination for the current training round;

[0023] The training image dataset is input into the fine-tuned large model, and forward propagation is performed to update the learnable parameters of the fine-tuned large model with the total loss function.

[0024] After the current training round ends, the pheromone concentration in the ACO optimizer is updated based on the quality evaluation metrics of the fine-tuned large model for the current training round.

[0025] Preferably, in the ACO optimizer, the state transition probability formula for the ant colony optimization algorithm is:

[0026] ;

[0027] Where i is the index in the rank candidate set, j is the index in the scaling factor candidate set, and m is the index in the number of experts to be selected candidate set. The probability of choosing the i-th rank, the j-th scaling factor, and the m-th number of experts for the ant as the combined solution is given. Let the rank of i, the scaling factor of j, and the number of selected experts be used as the pheromone concentration of the combination. For heuristic information, For the s-th rank, Let u be the scaling factor. Let v be the number of experts selected. For an optional set of combinations, Let the s-th rank, the u-th scaling factor, and the v-th number of selected experts be used as the combined pheromone concentration. Pheromones are the heuristic factor. For inspiring information factors.

[0028] Preferably, the ACO-LoRA-A module adopts the standard LoRA parallel branch.

[0029] Preferably, the ACO-LoRA-B module is embedded in the ACO-MoE gating network, and the data processing includes:

[0030] Based on learnable matrices The input features of the ACO-LoRA-B module are used with learnable matrices. Calculate intermediate image features by taking the rank of the ACO-LoRA-B module;

[0031] The intermediate image features are input into the ACO-MoE gated network to obtain the gated features;

[0032] Based on gated features, using learnable matrices Calculate the output characteristics of the ACO-LoRA-B module by taking the rank of the ACO-LoRA-B module.

[0033] Preferably, the intermediate image features are input into the ACO-MoE gating network to obtain gating features, and the steps include:

[0034] The clean Logits are obtained by adding the product of the intermediate image features and the gating weights to the training noise.

[0035] Based on clean Logits and gated pheromone concentration, the probability of expert selection is calculated using the following formula:

[0036] ;

[0037] in, Let be the probability of the nth expert's choice. Let n be the gating pheromone concentration of the nth expert. To clean Logits, The Softmax activation function is used. Pheromones are the heuristic factor. To inspire information factors;

[0038] Based on the expert selection probability, select the Top-K experts and calculate the gating features using the following formula:

[0039] ;

[0040] in, Here, K represents the number of experts selected, which is the gated feature. Let the weight be the weight of the k-th expert. For the kth expert, These are intermediate image features.

[0041] Preferably, updating the gating pheromone concentration of the ACO-MoE gating network in each training batch includes:

[0042] Based on the local gated volatiles, update the gated pheromone concentrations of all experts;

[0043] Based on expert quality assessment values, the gating pheromone concentration of the experts selected in the current training batch is enhanced.

[0044] Preferably, the total loss function includes the main task structure loss and the load balancing loss.

[0045] Preferably, the formula for calculating the load balancing loss is:

[0046] ;

[0047] in, For load balancing losses, The total number of experts, Let n be the usage rate vector for the nth expert. To calculate the standard deviation, To calculate the mean.

[0048] Preferably, the present invention further includes: if the quality evaluation index of the fine-tuned large model does not improve within multiple training rounds, adaptive scheduling is performed, including:

[0049] Perform a soft reset on the pheromone concentration in the ACO-MoE gated network and the ACO optimizer;

[0050] Dynamically adjust the loss weights in the total loss function.

[0051] Preferably, the present invention further includes: when the training rounds reach a preset number, performing a diversified search, the steps of which include:

[0052] Global candidate solutions are generated using strategies such as completely random restart, reverse information search, or optimal solution mutation.

[0053] A training epoch is performed using the global candidate solutions, and the quality evaluation index of the fine-tuned large model in that training epoch is calculated.

[0054] If the quality evaluation index of the fine-tuned large model in this training round is less than that of the fine-tuned large model in the previous training round, then the pheromone concentration of the global candidate solution is reduced in the ACO optimizer.

[0055] If the quality evaluation index of the fine-tuned large model in this training round is greater than that of the fine-tuned large model in the previous training round, then the pheromone concentration in the ACO optimizer is updated based on the quality evaluation index of the fine-tuned large model in this training round.

[0056] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0057] This invention presents a large-scale model fine-tuning method based on ant colony dynamics. Building upon the existing LoRA+MoE method, it first constructs an ACO optimizer based on the adaptive characteristics of the ant colony optimization algorithm. In each training epoch, it adaptively adjusts hyperparameters, including rank, scaling factor, and the number of experts selected, according to the feature complexity of each layer of the pre-trained large model. Furthermore, it updates the pheromone concentration of the ant colony optimization algorithm based on the model's performance metrics, achieving intelligent parameter tuning. This allows the fine-tuned large model to automatically adjust the rank configuration according to feature complexity to adapt to different task requirements. Moreover, this invention improves existing gating networks by constructing an ACO-MoE gating network, using the ant colony optimization algorithm for expert allocation, and further constructs a load balancing loss based on expert utilization to avoid performance degradation caused by unbalanced expert load. This invention achieves adaptive dynamic tuning of hyperparameters without manual intervention, optimizes the use of computational resources, and effectively improves the performance of the fine-tuned pre-trained large model. This enables the fine-tuned large model to efficiently adapt to different task requirements while ensuring task completion quality. Attached Figure Description

[0058] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0059] Figure 1 is a structural flowchart of a large model fine-tuning method based on ant colony dynamics mechanism according to the present invention.

[0060] Figure 2 is a schematic diagram of the tuning of the ACO optimizer proposed in this invention. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0062] Efficient fine-tuning of large model parameters is an important research direction in the field of deep learning, but it generally faces a series of problems such as high computational resource consumption, low parameter utilization efficiency, and complex hyperparameter tuning. With the continuous expansion of model size, these problems seriously affect the practical deployment effect of large models and hinder the application of artificial intelligence technology in resource-constrained environments. Therefore, research on efficient parameter fine-tuning techniques has significant practical implications.

[0063] To address the problems of existing technologies, and referring to Figure 1, this invention provides a method for fine-tuning a large model based on ant colony dynamics, comprising:

[0064] After freezing all parameters of the pre-trained large model, an ACO-LoRA-A module is inserted into the encoder of the pre-trained large model, and an ACO-LoRA-B module is inserted into the decoder of the pre-trained large model to obtain a fine-tuned large model; the ACO-LoRA-A module adopts the standard LoRA parallel branch, and the ACO-LoRA-B module is embedded in the ACO-MoE gated network.

[0065] An ACO optimizer was constructed based on the ant colony optimization algorithm.

[0066] The steps for fine-tuning each training epoch of a large model include:

[0067] Obtain the feature complexity of the fine-tuned large model from the previous training epoch and input it into the ACO optimizer; use the feature complexity of the fine-tuned large model from the previous training epoch as heuristic information to solve for the optimal hyperparameter combination for the current training epoch; the hyperparameter combination includes rank, scaling factor and number of experts selected;

[0068] Configure the ACO-LoRA-A and ACO-LoRA-B modules based on the optimal hyperparameter combination for the current training round;

[0069] The training image dataset is input into the fine-tuned large model, and forward propagation is performed to update the learnable parameters of the fine-tuned large model with the total loss function.

[0070] Based on the current training round, fine-tune the quality evaluation metrics of the large model and update the pheromone concentration in the ACO optimizer.

[0071] Specifically, the preparation stage of this invention is system initialization and configuration, which includes two parts: model loading and structure construction, and ACO optimizer configuration.

[0072] The model loading and structure construction steps include: loading a pre-trained large model (SAM2 is used in this embodiment); freezing all parameters of the pre-trained large model; inserting an ACO-LoRA-A module into the encoder of the pre-trained large model and an ACO-LoRA-B module into the decoder of the pre-trained large model to obtain a fine-tuned large model; the ACO-LoRA-A module uses a standard LoRA parallel branch to handle the attention mechanism; the ACO-LoRA-B module embeds an ACO-MoE gating network to handle feature reconstruction.

[0073] The ACO optimizer configuration steps include:

[0074] Initialize the ACO optimizer, setting the candidate space and related parameters of the solution set. The initialization settings in this embodiment are as follows:

[0075] The set of candidate values ​​for rank is: ;

[0076] The candidate set of scaling factor values ​​is: ;

[0077] The candidate set of values ​​for the number of experts is: ;

[0078] The number of ants is set to 20, and the evaporation rate is... Initialize the pheromone concentration matrix.

[0079] Before the start of each training epoch, the ACO optimizer is used to select the optimal combination of hyperparameters (rank r, scaling factor, etc.) for the ACO-LoRA-A and ACO-LoRA-B modules in the encoder and decoder based on historical performance and the current pheromone distribution. (Number of experts selected, K). The core mechanism of the ACO optimizer is expressed by the formula:

[0080] ;

[0081] in, and These represent the hyperparameter configurations of the encoder and decoder, respectively. H represents the current training round count, and H represents historical performance information. For ACO optimizer.

[0082] Referring to Figure 2, the ACO optimizer obtains the feature complexity of each layer of the large model before training begins or during the previous training round for fine-tuning, serving as heuristic information for the Ant Colony Optimization (ACO) algorithm; the formula for calculating feature complexity is:

[0083] ;

[0084] in, To calculate the standard deviation, For image feature tensors, It is the sigmoid activation function. The feature complexity is denoted as .

[0085] In the ACO optimizer, using rank, scaling factor, and the number of selected experts as path nodes, the state transition probability formula for the ant colony optimization algorithm is:

[0086] ;

[0087] Where i is the index in the rank candidate set, j is the index in the scaling factor candidate set, and m is the index in the number of experts to be selected candidate set. The probability of choosing the i-th rank, the j-th scaling factor, and the m-th number of experts for the ant as the combined solution is given. Let the rank of i, the scaling factor of j, and the number of selected experts be used as the pheromone concentration of the combination, representing historical experience. For heuristic information, For the s-th rank, Let u be the scaling factor. Let v be the number of experts selected. For an optional set of combinations, Let the s-th rank, the u-th scaling factor, and the v-th number of selected experts be used as the combined pheromone concentration. This is a pheromone-inspired factor used to control the weighting of historical experience. The heuristic information factor is used to control the weights of the current feature state. The denominator of the state transition probability formula represents the sum of fitness over all possible combinations, used for probability normalization.

[0088] The output of the ACO optimizer , K configures the hyperparameters of the ACO-LoRA-A module of the encoder and the ACO-LoRA-B module of the decoder for the current training round, where the hyperparameters of the ACO-LoRA-A module are defined as follows: The hyperparameters of the ACO-LoRA-B module are defined as follows: .

[0089] The training dataset is input into the fine-tuned large model, forward propagation is performed, and the learnable parameters of the fine-tuned large model are updated with the total loss function. The steps include:

[0090] S1: Encoder containing the ACO-LoRA-A module.

[0091] The original image is input into the encoder to obtain encoded image features. An ACO-LoRA-A module is then incorporated alongside the linear layer of the encoder. The output features of the ACO-LoRA-A module are added to the output features of the linear layer to obtain the final output features of the linear layer. In this embodiment, the linear layers with the added ACO-LoRA-A module include the qkv and proj linear layers.

[0092] The ACO-LoRA-A module uses a standard LoRA parallel branch as the branch of the encoder's linear layer, expressed by the formula:

[0093] ;

[0094] in, The input features are the linear layers of the encoder. This refers to the data processing procedure of the branch where the ACO-LoRA-A module is located. For data processing in linear layers, and For learnable matrices, For slicing operations, For the rank of the ACO-LoRA-A module, The scaling factor for the ACO-LoRA-A module. For pre-trained weights, This represents the final output feature of the linear layer of the encoder.

[0095] This section teaches how to directly multiply matrices A and B using slicing operations. Only a portion of the parameter space is activated.

[0096] S2: An intermediate convolutional layer containing the ACO-LoRA-B module.

[0097] Encoding image features requires passing through intermediate convolutional layers before being input into the decoder. This invention incorporates an ACO-LoRA-B module alongside the intermediate convolutional layer, adding the output features of the ACO-LoRA-B module to the output features of the intermediate convolutional layer to obtain the final output features of the intermediate convolutional layer.

[0098] Specifically, the ACO-LoRA-B module is embedded in the ACO-MoE gated network to implement the "projection-distribution-aggregation-projection" process, and its data processing includes:

[0099] Based on learnable matrices The input features of the ACO-LoRA-B module are used with learnable matrices. The rank of the ACO-LoRA-B module is used to reduce the dimensionality of the input features to a value through multiplication or convolution. The intermediate image features are obtained, and the formula is expressed as:

[0100] ;

[0101] in, For intermediate image features, For convolution operations, For slicing operations, For the rank of the ACO-LoRA-B module, The input features are those of the intermediate convolutional layers;

[0102] The intermediate image features are input into the ACO-MoE gated network to obtain the gated features;

[0103] Based on gated features, using learnable matrices Given the rank of the ACO-LoRA-B module, calculate the output features of the ACO-LoRA-B module using the formula:

[0104] ;

[0105] in, The output characteristics of the ACO-LoRA-B module, As a gating feature, This is a slicing operation.

[0106] After incorporating the ACO-LoRA-B module, the overall processing flow of the intermediate convolutional layers is as follows:

[0107] ;

[0108] in, The scaling factor for the ACO-LoRA-B module. These are the pre-trained weights for the intermediate convolutional layers. This represents the final output feature of the intermediate convolutional layer.

[0109] Specifically, the intermediate image features are input into the ACO-MoE gating network to obtain gating features. The steps include:

[0110] The clean Logits are obtained by adding the product of the intermediate image features and the gating weights to the training noise, as expressed by the formula:

[0111] ;

[0112] in, To clean Logits, For gating weights, For training noise;

[0113] Based on clean Logits and gated pheromone concentration, the probability of expert selection is calculated using the following formula:

[0114] ;

[0115] in, Let be the probability of the nth expert's choice. Let n be the gating pheromone concentration of the nth expert. The Softmax activation function is used. Pheromones are the heuristic factor. To inspire information factors;

[0116] Based on the expert selection probability, select the Top-K experts and calculate the gating features using the following formula:

[0117] ;

[0118] in, Here, K represents the number of experts selected, which is the gated feature. Let the weight be the weight of the k-th expert. For the kth expert, These are intermediate image features.

[0119] Preferably, updating the gating pheromone concentration of the ACO-MoE gating network in each training batch includes:

[0120] Based on the local gated pheromone volatility, the gated pheromone concentrations of all experts are updated to simulate forgetting and prevent the solidification of old experiences. The formula is expressed as:

[0121] ;

[0122] in, For gating pheromone concentration, For the controlled volatilization rate of the local department;

[0123] Based on expert quality assessment values, the gating pheromone concentration of the experts selected in the current training batch is enhanced, as expressed by the formula:

[0124] ;

[0125] in, Let be the gating pheromone concentration of the k-th selected expert. It is a local increment constant. The quality assessment value for the k-th selected expert is calculated based on that expert's contribution.

[0126] Updating the gating pheromone concentration ensures that the expert selection strategy can quickly adapt to changes in batch training data, achieving "adjusting the route while training".

[0127] S3: Decoder.

[0128] The output features of the intermediate convolutional layers are input into the decoder to obtain the output mask.

[0129] S4: Loss function.

[0130] The learnable parameters of the fine-tuned large model are updated using the total loss function. The total loss function of this invention includes the main task structure loss and the load balancing loss, and is expressed by the following formula:

[0131] ;

[0132] in, For the total loss function, The main task structure loss, For load balancing losses, For load balancing weights.

[0133] The main task structure loss adopts weighted IoU and BCE loss.

[0134] The load balancing loss is used to prevent expert "collapse" (i.e., only a few experts are used), and is calculated using the coefficient of variation squared loss, expressed by the formula:

[0135] ;

[0136] in, For load balancing losses, The total number of experts, Let n be the usage rate vector for the nth expert. To calculate the standard deviation, To calculate the mean.

[0137] S5: Backpropagation and Learnable Parameter Update.

[0138] Preferably, this implementation uses the AdamW optimizer to update the learnable parameters of the fine-tuned large model. The learnable parameters include: a learnable matrix. and numerical values, expert weights Gating weights .

[0139] After the current training round ends, the pheromone concentration in the ACO optimizer is updated based on the quality evaluation metrics of the fine-tuned large model for the current training round.

[0140] In this embodiment, the average performance index (IoU) of the entire training round is used as the quality evaluation index.

[0141] The ACO optimizer's pheromone update mechanism includes an elite strategy and diversified search.

[0142] In regular training rounds, an elite strategy is used to update pheromone concentrations, including:

[0143] Pheromone evaporation: The pheromone concentration of all candidate combinations decreases, as expressed by the formula:

[0144] ;

[0145] in, For the index of training rounds, This refers to the evaporation rate.

[0146] Elite ant pheromone enhancement: Based on the current training round, fine-tuning the quality evaluation metrics of the large model, pheromones are added to the solutions obtained from the hyperparameter combinations. The formula is expressed as:

[0147] ;

[0148] in, The total number of ants. The pheromone increment constant, Let the quality evaluation index of the fine-tuned large model be obtained for the p-th ant representative combination solution.

[0149] The better the performance of the hyperparameter combination, the more pheromone the corresponding combination solution will have.

[0150] Preferably, if the quality evaluation metrics of the fine-tuned large model do not improve within multiple training rounds, adaptive scheduling is performed, including:

[0151] Perform a soft reset on the pheromone concentration in the ACO-MoE gated network and the ACO optimizer;

[0152] The loss weights in the total loss function are dynamically adjusted, and the exploration intensity is reduced in the later stages of training to focus on convergence.

[0153] Preferably, a diversity search is performed once when the preset number of training epochs is reached. For example, a diversity search is performed every ten training epochs, and the steps include:

[0154] Global candidate solutions are generated using strategies such as completely random restart, reverse information search, or optimal solution mutation.

[0155] A training epoch is performed using the global candidate solutions, and the quality evaluation index of the fine-tuned large model in that training epoch is calculated.

[0156] If the quality evaluation index of the fine-tuned large model in this training round is less than that of the fine-tuned large model in the previous training round, then the pheromone concentration of the global candidate solution is reduced in the ACO optimizer.

[0157] If the quality evaluation index of the fine-tuned large model in this training round is greater than that of the fine-tuned large model in the previous training round, then the pheromone concentration in the ACO optimizer is updated based on the quality evaluation index of the fine-tuned large model in this training round.

[0158] Specifically, the Random Walk strategy involves ignoring the current pheromone matrix τ and uniformly generating a set of configurations in the candidate space. The Low-Pheromone Search strategy involves intentionally selecting the path with the lowest pheromone concentration. The Mutation strategy involves obtaining the current historical best solution and randomly perturbing it. For example... , Let rank be the result of the perturbation. For the perturbation step, To determine the rank of the optimal solution, we can change the optimal value of r from 16 to 8 or 32. This allows for a local search in the vicinity of the optimal solution to see if a better neighborhood solution can be found.

[0159] After the diversification search concludes, at the start of the next training round, the standard pheromone-based state transition probability formula is reverted. At this point, if a better new path is discovered through the diversification search, the standard algorithm will naturally guide subsequent ants to that new path.

[0160] Traditional fine-tuning methods often struggle to balance model performance and computational efficiency. This invention combines the advantages of ant colony optimization (ACO) and efficient parameter fine-tuning to fundamentally improve the LoRA architecture for large model adaptation, enhancing parameter utilization efficiency and model performance. It also completes the design and implementation of a full training process. The core of this invention lies in leveraging the self-organizing and positive feedback characteristics of ACO to achieve intelligent exploration and balance in parameter configuration. By adaptively emphasizing or weakening certain configuration paths through the learned pheromone distribution, it improves parameter utilization efficiency and can be effectively applied to fields such as natural language processing, computer vision, and multimodal learning.

[0161] To verify the effectiveness of the method of this invention, this embodiment uses the existing medical image segmentation dataset Polyp for training and validates it on multiple publicly available test sets. Training images were uniformly resized to 352×352 pixels, and training was performed on an Ubuntu 20.04 system and an NVIDIA V100 Tensor Core GPU using Python and the PyTorch framework. A total of 50 training epochs were conducted, employing cosine annealing for learning rate scheduling, with an initial learning rate of 0.001.

[0162] In the comparative experiments, in addition to comparing with the previous SAM and SAM2 methods, this embodiment also selected a variety of image segmentation methods for comparison, and calculated IoU (Intersection over Union) and accuracy on multiple downstream tasks to evaluate the final results.

[0163] Subjective judgment has certain limitations; only by calculating corresponding image segmentation evaluation metrics can more objective and accurate results be obtained. Table 1 shows the comparison results of various methods. A higher mIoU value indicates that the segmentation result is closer to the true annotation, and higher accuracy means better segmentation performance. It can be seen that the method proposed in this invention not only performs excellently in subjective visual evaluation but also achieves the best results in the comparison of objective metrics.

[0164] Table 1. Comparison Results of Different Methods

[0165]

[0166] To verify the effectiveness of ACO dynamic hyperparameters, this embodiment compares them with the fixed-rank method, and the results are shown in Table 2.

[0167] Table 2. Experimental Results of Dynamic Hyperparameter Comparison

[0168]

[0169] The experimental results show that the method of this invention has certain improvements compared to other methods, and dynamic rank adjustment contributes to the performance improvement. It effectively reduces the workload of manual parameter tuning, and the hyperparameter selection is fully automated, improving the convenience of using the model. The method of this invention supports multiple candidate value configurations, and the system automatically explores the optimal combination.

[0170] After 50 epochs of training, this invention achieves an average IoU of 0.808 and an average Dice of 0.871 on the Polyp medical image segmentation dataset, outperforming the fixed-rank LoRA method. Compared to the untuned SAM2 method, these figures represent improvements of 77.9% and 67.1%, respectively, while reducing the number of parameters to approximately 3% of the total. The modular design allows this invention to flexibly adapt to different large model architectures and downstream tasks, achieving truly efficient adaptive parameter fine-tuning. Furthermore, the enhanced ACO algorithm effectively avoids local optima.

[0171] To improve the adaptability of parameter configuration, this invention introduces dynamic rank adjustment and adaptive expert selection mechanisms into the traditional LoRA architecture, and experimental results on multiple downstream tasks verify the effectiveness of the proposed method. Furthermore, based on the ant colony dynamics mechanism of this invention, a complete large-scale model fine-tuning system is designed and implemented, providing core functions such as model loading, parameter configuration, training monitoring, and performance evaluation.

[0172] In summary, the large model fine-tuning method based on ant colony dynamics described in this invention, building upon the existing LoRA+MoE method, first constructs an ACO optimizer based on the adaptive characteristics of the ant colony optimization algorithm. In each training epoch, it adaptively adjusts hyperparameters, including rank, scaling factor, and the number of experts selected, according to the feature complexity of each layer of the pre-trained large model. Furthermore, it updates the pheromone concentration of the ant colony optimization algorithm based on the model's performance metrics, achieving intelligent tuning of parameter configuration. This allows the fine-tuned large model to automatically adjust the rank configuration according to feature complexity to adapt to different task requirements. Moreover, this invention improves upon existing gating networks by constructing an ACO-MoE gating network, using the ant colony optimization algorithm for expert allocation, and further constructs a load balancing loss based on expert utilization to avoid performance degradation caused by unbalanced expert load. This invention achieves adaptive dynamic tuning of hyperparameters without manual intervention, optimizes the use of computational resources, and effectively improves the performance of the pre-trained large model after fine-tuning. This enables the fine-tuned large model to efficiently adapt to different task requirements while ensuring task completion quality.

[0173] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0175] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0176] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0177] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for fine-tuning a large model based on ant colony dynamics, characterized in that, include: After freezing all parameters of the pre-trained large model, an ACO-LoRA-A module is incorporated into the encoder of the pre-trained large model, and an ACO-LoRA-B module is incorporated into the convolutional layer before the decoder of the pre-trained large model to obtain a fine-tuned large model; the ACO-LoRA-A module adopts the standard LoRA parallel branch. The ACO-LoRA-B module is embedded in the ACO-MoE gated network; an ACO optimizer is constructed based on the ant colony optimization algorithm; the steps for each training epoch of fine-tuning the large model include: obtaining the feature complexity of the fine-tuned large model in the previous training epoch and inputting it into the ACO optimizer; using the feature complexity of the fine-tuned large model in the previous training epoch as heuristic information to solve for the optimal hyperparameter combination in the current training epoch; the hyperparameter combination includes rank, scaling factor, and number of experts selected; configuring the ACO-LoRA-A and ACO-LoRA-B modules based on the optimal hyperparameter combination in the current training epoch; inputting the training image dataset into the fine-tuned large model for forward propagation, and updating the learnable parameters of the fine-tuned large model with the total loss function; after the current training epoch ends, updating the pheromone concentration in the ACO optimizer based on the quality evaluation index of the fine-tuned large model in the current training epoch; the ACO-LoRA-B module is embedded in the ACO-MoE gated network, and the data processing process includes: based on the learnable matrix The input features of the ACO-LoRA-B module are used with learnable matrices. The intermediate image features are calculated by taking the rank of the ACO-LoRA-B module; these intermediate image features are then input into the ACO-MoE gated network to obtain gated features; based on these gated features, a learnable matrix is ​​used... Calculate the output characteristics of the ACO-LoRA-B module by taking the rank of the ACO-LoRA-B module.

2. The method for fine-tuning a large model based on ant colony dynamics as described in claim 1, characterized in that, In the ACO optimizer, the state transition probability formula for the ant colony optimization algorithm is: Where i is the index in the rank candidate set, j is the index in the scaling factor candidate set, and m is the index in the number of experts to be selected candidate set. The probability of choosing the i-th rank, the j-th scaling factor, and the m-th number of experts for the ant as the combined solution is given. Let the rank of i, the scaling factor of j, and the number of selected experts be used as the pheromone concentration of the combination. For heuristic information, For the s-th rank, Let u be the scaling factor. Let v be the number of experts selected. For an optional set of combinations, Let the s-th rank, the u-th scaling factor, and the v-th number of selected experts be used as the combined pheromone concentration. Pheromones are the heuristic factor. For inspiring information factors.

3. The method for fine-tuning a large model based on ant colony dynamics as described in claim 1, characterized in that, The intermediate image features are input into the ACO-MoE gated network to obtain gated features. The steps include: multiplying the intermediate image features and the gate weights, and adding the training noise to obtain clean logits; based on the clean logits and the gate pheromone concentration, the expert selection probability is calculated using the following formula: ;in, Let be the probability of the nth expert's choice. Let n be the gating pheromone concentration of the nth expert. To clean Logits, The Softmax activation function is used. Pheromones are the heuristic factor. To heuristically select information factors, based on expert selection probabilities, select Top-K experts and calculate the gating features using the following formula: ;in, Here, K represents the number of experts selected, which is the gated feature. Let the weight be the weight of the k-th expert. For the kth expert, These are intermediate image features.

4. The method for fine-tuning a large model based on ant colony dynamics as described in claim 3, characterized in that, The gating pheromone concentration of the ACO-MoE gating network is updated in each training batch, including: updating the gating pheromone concentration of all experts based on local gating evaporation rate; and enhancing the gating pheromone concentration of the experts selected in the current training batch based on expert quality evaluation values.

5. The method for fine-tuning a large model based on ant colony dynamics as described in claim 1, characterized in that, The total loss function includes the main task structure loss and the load balancing loss.

6. The method for fine-tuning a large model based on ant colony dynamics as described in claim 5, characterized in that, The formula for calculating the load balancing loss is: ;in, For load balancing losses, The total number of experts, Let n be the usage rate vector for the nth expert. To calculate the standard deviation, To calculate the mean.

7. The method for fine-tuning a large model based on ant colony dynamics as described in claim 1, characterized in that, Also includes: If the quality evaluation metrics of the large model do not improve after fine-tuning over multiple training rounds, adaptive scheduling is performed, including: performing a soft reset on the pheromone concentration in the ACO-MoE gated network and the ACO optimizer; and dynamically adjusting the loss weights in the total loss function.

8. The method for fine-tuning a large model based on ant colony dynamics as described in claim 1, characterized in that, Also includes: When the training rounds reach the preset number, a diversified search is performed, including steps such as generating global candidate solutions using a completely random restart, reverse information search, or optimal solution mutation strategy. A training epoch is performed using the global candidate solution, and the quality evaluation index of the fine-tuned large model in this training epoch is calculated. If the quality evaluation index of the fine-tuned large model in this training epoch is less than that of the fine-tuned large model in the previous training epoch, the pheromone concentration of the global candidate solution is reduced in the ACO optimizer. If the quality evaluation index of the fine-tuned large model in this training epoch is greater than that of the fine-tuned large model in the previous training epoch, the pheromone concentration in the ACO optimizer is updated based on the quality evaluation index of the fine-tuned large model in this training epoch.

Images