A long tail recognition adaptive data governance method based on multi-agent cooperation

Abstract

The application relates to a long-tail recognition adaptive data governance method based on multi-agent cooperation, relates to the technical field of computer vision recognition, and comprises the following steps: an adaptive sample filtering module of quality perception topology measurement (QATM) is designed, sample fidelity, head class repulsion punishment and prediction entropy training value quantization are simultaneously fused, high-fidelity and high-information synthetic samples are screened, and a compact and balanced composite training data set is constructed; secondly, a multi-agent cooperation module (MAC) containing four functional agents of classifier state estimation, target analysis, strategy selection and strategy review and one scheduling agent is constructed based on a large language model; through a forward thinking chain reasoning and a reverse feedback optimization mechanism, the classifier training state and the domain deviation degree are perceived in real time, and differentiated class perception data enhancement strategies are dynamically generated for the head class and the tail class. Finally, a closed-loop optimization mechanism is constructed through a sandbox network distribution check and a multi-dimensional composite loss function, and the enhancement strategy is adaptively iterated.

Description

Technical Field

[0001] This invention relates to the field of computer vision recognition technology, specifically to a long-tail recognition adaptive data governance method based on multi-agent collaboration. Background Technology

[0002] Real-world visual data typically exhibits a long-tail distribution, meaning that a small number of head classes have a large number of samples, while the vast majority of tail classes contain only a very small number of samples. This data imbalance causes deep learning models to be severely biased towards head classes, resulting in a sharp drop in the recognition accuracy of tail classes and severely limiting the deployment and application of these models in real-world scenarios.

[0003] Traditional long-tail identification methods are mainly divided into three categories: first, reweighting methods, which balance gradient contributions by assigning higher loss weights to tail class samples; second, resampling methods, which adjust the distribution of training data by oversampling tail classes or undersampling head classes; and third, model structure adjustment methods, which enhance the feature representation ability of tail classes by designing dedicated network modules. However, these methods generally rely on manual design, have poor generalization ability, and cannot fundamentally solve the problem of insufficient tail class data.

[0004] In recent years, data augmentation methods based on generative models have become a research hotspot for long-tail identification, supplementing tail class data with synthetic samples generated through diffusion models, GANs, and other techniques. For example, DiffLT uses a class-balanced diffusion model to generate samples for tail classes, and GIF enriches a limited dataset using Stable Diffusion. However, these methods have two core drawbacks: 1. Inconsistent quality of synthetic data: The quality of generated samples varies greatly. A large number of low-quality samples not only fail to improve model performance, but also introduce noise to interfere with training.

[0005] 2. Synthetic-real data distribution bias: The inherent randomness in the generation process leads to a significant domain difference between synthetic and real data. After mixed training, the distribution of real data for each class will be changed, further exacerbating the classification bias of the model.

[0006] With the development of Large Language Models (LLMs), some studies have attempted to utilize LLMs to assist long-tail recognition, such as LLM-AutoDA, which automatically learns data augmentation strategies through LLMs, and LTGC, which uses LLMs to generate image descriptions and external samples. However, existing LLM-driven methods still have shortcomings: the augmentation strategies are mostly fixed designs and cannot be dynamically adjusted according to the real-time training state of the classifier; the domain offset between the synthesized data and the real data is not explicitly modeled, resulting in low data augmentation efficiency; and they still rely on large-scale synthetic data, leading to high training costs. Summary of the Invention

[0007] To overcome the shortcomings of the above technologies, this invention provides a method that simultaneously solves the problems of synthetic data quality and distribution offset, achieving efficient and accurate long-tail identification.

[0008] The technical solution adopted by this invention to overcome its technical problems is: An adaptive data governance method for long-tail recognition based on multi-agent cooperation includes: S1. Establish a real training set and the synthetic sample set to be screened ; S2. Using the real training set and the synthetic sample set to be screened Obtain the feature set of real samples ,category True Sample Feature Matrix and the feature set of the synthetic samples to be screened ; S3. Utilizing Categories True Sample Feature Matrix and the feature set of the synthetic samples to be screened Calculate the synthetic sample feature set The Middle Purity fraction of each synthetic sample ; S4. Calculate the... Purity fraction of each synthetic sample Purity base score after punishment ; S5. Utilize the base purity score after penalty. Calculate the first Quality-perceived topology of each synthetic sample The synthetic sample set to be screened is determined by quality-aware topological metrics. Filtering is performed to obtain a balanced composite dataset. ; S6. Utilizing Balanced Composite Datasets The final enhancement strategy for class awareness that passes the verification is obtained. ; S7. Construct a sandbox network, and the first... A synthetic sample and the final enhancement strategy using class awareness. The transformed synthetic image is input into a sandbox network, and the output is the baseline feature set. Enhanced feature groups ; S8. Utilizing the baseline feature set Enhanced feature groups Sandbox loss Utilizing sandbox losses Construct the total loss and train the model using the total loss.

[0009] Furthermore, in step S1, the real training set The datasets used are CIFAR10-LT, CIFAR100-LT, or ImageNet-LT. The CIFAR10-LT dataset is input into a diffusion model, generating 100,000 synthetic images for each of the 10 categories. The CIFAR100-LT dataset is input into the diffusion model, generating 10,000 synthetic images for each of the 100 categories. The ImageNet-LT dataset is input into the diffusion model, generating 5,000 synthetic images for each category. The synthetic images generated from all the CIFAR10-LT, CIFAR100-LT, and ImageNet-LT datasets constitute the synthetic sample set to be selected. .

[0010] Furthermore, the above diffusion model is a pre-trained EDM model.

[0011] Furthermore, step S2 includes the following steps: S2-1. Use the real training set The Middle Zhang Image Adjust the resolution to 224×224, and then... The pixel values ​​in the image are normalized, and the Normalize function in torch.transform is used to normalize the 1st pixel value. The images are converted into normalized tensors, and the normalized tensors are input into the CLIP visual encoder to obtain the first image. Zhang Image True sample features All real sample features constitute the real sample feature set. ; S2-2. Building Categories True Sample Feature Matrix , ,in, For the real number space, For real sample feature set The middle category is The number of real sample features , For real training set The number of categories; S2-3. Combine the sample sets to be screened The Middle Zhang composite image Adjust the resolution to 224×224, and then... The pixel values ​​in the composite image are normalized, and the Normalize function in torch.transform is used to normalize the pixel values ​​of the first image. The synthesized image is converted into a normalized tensor, and the normalized tensor is input into the CLIP visual encoder to obtain the first image. Zhang composite image Features of synthetic samples to be screened All the features of the synthetic samples to be screened constitute the feature set of the synthetic samples to be screened. .

[0012] Furthermore, step S3 includes the following steps: S3-1. Feature matrix of real samples The category is obtained using the torch.mean function in PyTorch. mean vector , ; S3-2. Feature matrix of real samples The category is obtained using the torch.cov function in PyTorch. covariance matrix , ; S3-3. On the covariance matrix The category is obtained using the `torch.linalg.inv` function in PyTorch. inverse of the covariance matrix ; S3-4. Through formula Calculate the first Zhang composite image Manifold Mahalanobis distance In the formula, For transpose; S3-5. For categories... The manifold Mahalanobis distance of all synthesized images is normalized using a Min-Max method to obtain the nth... Purity fraction of each synthetic sample .

[0013] Furthermore, step S4 includes the following steps: S4-1. Combine the sample sets to be screened According to its ownership Sort the number of synthesized images in each category in descending order, and select the top... Each category serves as the header class. , To round down, select after Each category is used as a tail class. ; S4-2. Use the torch.nn.functional.cosine_similarity function to calculate the number of the header class. The mean vector of features of the synthetic samples to be screened for all synthetic images in each category. , ; S4-3. Through formula The rejection penalty value was calculated. In the formula, To find the L2 norm; S4-4. Through formula The exponential decay coefficient was calculated. In the formula, It is an adjustable hyperparameter; S4-5. Through formula The purity base score after penalty is calculated. .

[0014] Furthermore, step S5 includes the following steps: S5-1. The first Zhang composite image The data is sequentially input into a ResNet-32 model and a Softmax activation function, and the output is the class probability distribution. For category probability distribution Use the torch.sum function to get the number of... Zhang composite image Predicted entropy ; S5-2. Through formula Calculate the first Quality-perceived topology of each synthetic sample In the formula It is an adjustable hyperparameter; S5-3. Synthesize the sample set to be screened After sorting all synthetic images in descending order of quality-aware topological metric, For each category, retain the synthetic images corresponding to the top-K quality-perceived topological metrics to obtain a high-quality synthetic sample set. High-quality synthetic sample sets With the real training set By concatenating the data, a balanced composite dataset is obtained. .

[0015] Furthermore, step S6 includes the following steps: S6-1. Balance the composite dataset The input is given to the ResNet-32 model, and the output is a sequence of indicators. The ResNet-32 model was trained for 50 epochs. Output training metric sequence for each epoch , ; S6-2. Output the training metric sequence from the 40th to the 50th epoch. - Inputting into the Qwen2.5-7B-Instruct model, the output is a balanced composite dataset. The Middle Status labels for each category , , For Healthy Learning, Overfitting, Stagnation, Unstable Training; S6-3. Transform the indicator sequence - Status labels Inputting into the Qwen2.5-7B-Instruct model, the output is the first... Enhancement targets in each category , ; S6-4. Enhance the target The input is given to the Qwen2.5-7B-Instruct model, and the output is the head class enhancement strategy. and tail-type enhancement strategies , , For the first Head-class enhancement sub-strategies , , For the first Tail-class enhancement sub-strategies ; S6-5. Enhancing Head Class Strategies and tail-type enhancement strategies The input is fed into the Qwen2.5-7B-Instruct model, and the output is the final class-aware enhancement strategy that has passed the validation. .

[0016] Furthermore, step S7 includes the following steps: S7-1. Construct a sandbox network consisting of a first convolutional layer, a second convolutional layer, a first batch normalization (BN) layer, a first ReLU activation function, a third convolutional layer, a fourth convolutional layer, a second batch normalization (BN) layer, a second ReLU activation function, a first linear layer, a second linear layer, and a GELU activation function. S7-2. Balancing Composite Datasets The Middle The synthesized image is input into the sandbox network, and the output is the baseline feature set. ; S7-3. Following the final enhancement strategy of class perception Use the torchvision.transforms.Compose function in PyTorch to perform the first... The composite image is transformed to obtain the transformed composite image; S7-4. Input the transformed synthetic image into a sandbox network, and output the enhanced feature set. .

[0017] Furthermore, step S8 includes the following steps: S8-1. Through formula Sandbox loss calculated In the formula, For batch size, The value is 32. For Gaussian kernel function, To balance composite datasets The Middle The baseline feature set of the composite image, To balance composite datasets The Middle Enhanced feature sets of the composite image; S8-2. When the sandbox is lost Greater than or equal to Then, return to steps S6-S7. The baseline threshold; S8-3. Extracting the weight matrix of fully connected layers in a ResNet-32 model using the state_dict function in PyTorch. The tail category The classification weight vector for each category is: , , No. The classification weight vector for each category is: , Classification weight vector Weight matrix The Rows, classification weight vector Weight matrix The OK; S8-4. Through formula Calculate the classification weight vector With classification weight vector The minimum cosine of the included angle between ; S8-5. Through formula The multidimensional composite loss function was calculated. In the formula, , , All are weighting coefficients. This represents the decrease in cross-entropy loss between the current round and the previous round during training. For compliance reward value, This is the difference between the minimum cosine of the angle in the current round and the minimum cosine of the angle in the previous round during training. S8-6. Using the AdamW optimizer with a multi-dimensional composite loss function The Qwen2.5-7B-Instruct model is trained in step S6-4.

[0018] The beneficial effects of this invention are: 1. It solves the core defects of synthetic data: By using an adaptive sample filtering module to simultaneously consider sample quality and training value, it selects a compact and highly informative synthetic subset, effectively avoiding interference from low-quality samples; by dynamically adjusting the enhancement strategy through the model, it explicitly models and reduces the domain offset between synthetic and real data.

[0019] 2. Achieved extreme data efficiency: Only 4.8% of the synthetic data (48,000 images) was used on the CIFAR10 / 100-LT dataset, and only 20% of the synthetic data was used on ImageNet-LT to achieve optimal performance, significantly reducing the cost of data generation and training.

[0020] 3. Achieved breakthrough recognition accuracy: Top-1 accuracy reached 92.8% on CIFAR10-LT (r=100), 70.2% on CIFAR100-LT (r=100), and 72.2% on ImageNet-LT (ResNet-50backbone), comprehensively surpassing existing state-of-the-art methods, and achieving balanced performance improvements across head, middle, and tail classes.

[0021] 4. Excellent generalization and scalability: The framework is independent of the backbone network and can be adapted to various mainstream vision models such as ResNet, CLIP, and DINOv2; the enhanced operation pool can be flexibly expanded according to different tasks, and can quickly adapt to various long-tail recognition scenarios.

[0022] 5. Achieved closed-loop dynamic optimization: Through the model's forward inference and backward feedback mechanisms, as well as reinforcement learning optimization, the augmentation strategy can continuously adapt to the classifier's training state, forming a virtuous cycle of data augmentation and model training. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0024] The following is in conjunction with the appendix Figure 1 The present invention will be further described below.

[0025] An adaptive data governance method for long-tail recognition based on multi-agent cooperation includes: S1. Establish a real training set and the synthetic sample set to be screened .

[0026] S2. Using the real training set and the synthetic sample set to be screened Obtain the feature set of real samples ,category True Sample Feature Matrix and the feature set of the synthetic samples to be screened .

[0027] S3. Utilizing Categories True Sample Feature Matrix and the feature set of the synthetic samples to be screened Calculate the synthetic sample feature set The Middle Purity fraction of each synthetic sample .

[0028] S4. Calculate the... Purity fraction of each synthetic sample Purity base score after punishment .

[0029] S5. Utilize the base purity score after penalty. Calculate the first Quality-perceived topology of each synthetic sample The synthetic sample set to be screened is determined by quality-aware topological metrics. Filtering is performed to obtain a balanced composite dataset. .

[0030] S6. Utilizing Balanced Composite Datasets The final enhancement strategy for class awareness that passes the verification is obtained. .

[0031] S7. Construct a sandbox network, and the first... A synthetic sample and the final enhancement strategy using class awareness. The transformed synthetic image is input into a sandbox network, and the output is the baseline feature set. Enhanced feature groups .

[0032] S8. Utilizing the baseline feature set Enhanced feature groups Sandbox loss Utilizing sandbox losses Construct the total loss and train the model using the total loss.

[0033] We utilize a pre-trained diffusion model to generate initial synthetic datasets for each long-tail class. An adaptive sample filtering module based on Quality-Aware Topology Metric (QATM) is designed to simultaneously filter high-information synthetic samples by integrating sample fidelity and training value, thus constructing a compact and balanced composite training dataset. Secondly, based on a large language model, a multi-agent collaborative module (MAC) is constructed, comprising four functional agents (classifier state estimation, target analysis, policy selection, and policy review) and a scheduling agent. Through forward reasoning and back-feedback optimization mechanisms, it perceives the classifier training state and domain shift in real time, dynamically generating differentiated class-aware data augmentation strategies for head and tail classes. Finally, the GRPO reinforcement learning algorithm is employed to optimize the multi-agent's policy generation capabilities, forming a virtuous cycle of data augmentation and model training.

[0034] In one embodiment of the present invention, the real training set in step S1 The datasets used are CIFAR10-LT, CIFAR100-LT, or ImageNet-LT. The CIFAR10-LT dataset is input into a diffusion model, generating 100,000 synthetic images for each of the 10 categories. The CIFAR100-LT dataset is input into the diffusion model, generating 10,000 synthetic images for each of the 100 categories. The ImageNet-LT dataset is input into the diffusion model, generating 5,000 synthetic images for each category. The synthetic images generated from all the CIFAR10-LT, CIFAR100-LT, and ImageNet-LT datasets constitute the synthetic sample set to be selected. .

[0035] In one embodiment of the present invention, the diffusion model is a pre-trained EDM model.

[0036] In one embodiment of the present invention, step S2 includes the following steps: S2-1. Use the real training set The Middle Zhang Image Adjust the resolution to 224×224, and then... The pixel values ​​in the image are normalized, and the Normalize function in torch.transform is used to normalize the 1st pixel value. The first image is converted into a normalized tensor (pixel values ​​normalized to [0,1]), and the normalized tensor is input into the CLIP visual encoder to obtain the first image. Zhang Image True sample features All real sample features constitute the real sample feature set. To mitigate the impact of interference factors such as pixel noise and texture differences on quality assessment.

[0037] S2-2. Building Categories True Sample Feature Matrix , ,in, For the real number space, For real sample feature set The middle category is The number of real sample features , For real training set The number of categories.

[0038] S2-3. Combine the sample sets to be screened The Middle Zhang composite image Adjust the resolution to 224×224, and then... The pixel values ​​in the composite image are normalized, and the Normalize function in torch.transform is used to normalize the pixel values ​​of the first image. The synthesized image is converted into a normalized tensor, and the normalized tensor is input into the CLIP visual encoder to obtain the first image. Zhang composite image Features of synthetic samples to be screened All the features of the synthetic samples to be screened constitute the feature set of the synthetic samples to be screened. .

[0039] In one embodiment of the present invention, step S3 includes the following steps: S3-1. Feature matrix of real samples The category is obtained using the torch.mean function in PyTorch. mean vector , .

[0040] S3-2. Feature matrix of real samples The category is obtained using the torch.cov function in PyTorch. covariance matrix , .

[0041] S3-3. On the covariance matrix The category is obtained using the `torch.linalg.inv` function in PyTorch. inverse of the covariance matrix .

[0042] S3-4. Through formula Calculate the first Zhang composite image Manifold Mahalanobis distance In the formula, This is a transpose.

[0043] S3-5. For categories... The manifold Mahalanobis distance of all synthesized images is normalized using a Min-Max method to obtain the nth... Purity fraction of each synthetic sample A higher score indicates a better fit between the sample and the true semantic distribution of the tail class.

[0044] In one embodiment of the present invention, step S4 includes the following steps: S4-1. Combine the sample sets to be screened According to its ownership Sort the number of synthesized images in each category in descending order, and select the top... Each category serves as the header class. , To round down, select after Each category is used as a tail class. .

[0045] S4-2. Use the torch.nn.functional.cosine_similarity function to calculate the number of the header class. The mean vector of features of the synthetic samples to be screened for all synthetic images in each category. , .

[0046] S4-3. Through formula The rejection penalty value was calculated. In the formula, To find the L2 norm.

[0047] S4-4. Through formula The exponential decay coefficient was calculated. In the formula, It is an adjustable hyperparameter.

[0048] S4-5. Through formula The purity base score after penalty is calculated. Using the exponential decay coefficient purity fraction Exponential decay can reduce the overall quality score of the sample.

[0049] In one embodiment of the present invention, step S5 includes the following steps: S5-1. The first Zhang composite image The data is sequentially input into a ResNet-32 model and a Softmax activation function, and the output is the class probability distribution. For category probability distribution Use the torch.sum function to get the number of... Zhang composite image Predicted entropy .

[0050] S5-2. Through formula Calculate the first Quality-perceived topology of each synthetic sample In the formula It is an adjustable hyperparameter.

[0051] S5-3. Synthesize the sample set to be screened After sorting all synthetic images in descending order of quality-aware topological metric, For each category, retain the synthetic images corresponding to the top-K quality-perceived topological metrics to obtain a high-quality synthetic sample set. High-quality synthetic sample sets With the real training set By concatenating the data, a balanced composite dataset is obtained. Complete the filtering of low-quality, high-risk synthetic samples.

[0052] In one embodiment of the present invention, step S6 includes the following steps: S6-1. Balance the composite dataset The input is given to the ResNet-32 model, and the output is a sequence of indicators. The ResNet-32 model was trained for 50 epochs. Output training metric sequence for each epoch , .

[0053] S6-2. Output the training metric sequence from the 40th to the 50th epoch. - The input is given to the Qwen2.5-7B-Instruct model, and the output is the synthetic sample set to be screened. The Middle Status labels for each category , , The terms are Healthy Learning, Overfitting, Stagnation, and Unstable Training.

[0054] S6-3. Transform the indicator sequence - Status labels Inputting into the Qwen2.5-7B-Instruct model, the output is the first... Enhancement targets in each category , .

[0055] S6-4. Enhance the target The input is given to the Qwen2.5-7B-Instruct model, and the output is the head class enhancement strategy. and tail-type enhancement strategies , , For the first Head-class enhancement sub-strategies , , For the first Tail-class enhancement sub-strategies .

[0056] S6-5. Enhancing Head Class Strategies and tail-type enhancement strategies The input is fed into the Qwen2.5-7B-Instruct model, and the output is the final class-aware enhancement strategy that has passed the validation. .

[0057] In one embodiment of the present invention, step S7 includes the following steps: S7-1. Construct a sandbox network consisting of a first convolutional layer, a second convolutional layer, a first batch normalization (BN) layer, a first ReLU activation function, a third convolutional layer, a fourth convolutional layer, a second batch normalization (BN) layer, a second ReLU activation function, a first linear layer, a second linear layer, and a GELU activation function.

[0058] S7-2. Balancing Composite Datasets The Middle The synthesized image is input into the sandbox network, and the output is the baseline feature set. .

[0059] S7-3. Following the final enhancement strategy of class perception Use the torchvision.transforms.Compose function in PyTorch to perform the first... The composite image is transformed to obtain the transformed composite image.

[0060] S7-4. Input the transformed synthetic image into a sandbox network, and output the enhanced feature set. .

[0061] In one embodiment of the present invention, step S8 includes the following steps: S8-1. Through formula Sandbox loss calculated In the formula, For batch size, The value is 32. For Gaussian kernel function, To balance composite datasets The Middle The baseline feature set of the composite image, To balance composite datasets The Middle Enhanced feature sets of a synthesized image. Sandbox loss. A higher value indicates a more severe shift in data distribution caused by the augmentation strategy, which is more likely to disrupt training stability.

[0062] S8-2. When the sandbox is lost Greater than or equal to Then, return to steps S6-S7. This is the baseline threshold.

[0063] S8-3. Extracting the weight matrix of fully connected layers in a ResNet-32 model using the state_dict function in PyTorch. The tail category The classification weight vector for each category is: , , No. The classification weight vector for each category is: , Classification weight vector Weight matrix The Rows, classification weight vector Weight matrix The OK.

[0064] S8-4. Through formula Calculate the classification weight vector With classification weight vector The minimum cosine of the included angle between .

[0065] S8-5. Through formula The multidimensional composite loss function was calculated. In the formula, , , All are weighting coefficients. This represents the decrease in cross-entropy loss between the current round and the previous round during training. For compliance reward value, This is the difference between the minimum cosine of the angle in the current round and the minimum cosine of the angle in the previous round during training.

[0066] S8-6. Using the AdamW optimizer with a multi-dimensional composite loss function The Qwen2.5-7B-Instruct model is trained in step S6-4. Through this reinforcement learning loop, the policy selection agent not only learns the general reinforcement logic to reduce classification loss, but also implicitly masters the specific strategy of "physically widening" the decision boundary of the tail class through data augmentation. This allows for continuous adaptation to the differentiated needs of long-tail recognition and achieves adaptive iterative optimization of the data governance strategy. The AdamW optimizer learning rate is set to 5e-6, the batch size is 8, and each training stage consists of 50 steps.

[0067] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A long-tail recognition adaptive data governance method based on multi-agent collaboration, characterized in that, include: S1. Establish a real training set and the synthetic sample set to be screened ; S2. Using the real training set and the synthetic sample set to be screened Obtain the feature set of real samples ,category True Sample Feature Matrix and the feature set of the synthetic samples to be screened ; S3. Utilizing Categories True Sample Feature Matrix and the feature set of the synthetic samples to be screened Calculate the synthetic sample feature set The Middle Purity fraction of each synthetic sample ; S4. Calculate the... Purity fraction of each synthetic sample Purity base score after punishment ; S5. Utilize the base purity score after penalty. Calculate the first Quality-perceived topology of each synthetic sample The synthetic sample set to be screened is determined by quality-aware topological metrics. Filtering is performed to obtain a balanced composite dataset. ; S6. Utilizing Balanced Composite Datasets The final enhancement strategy for class awareness that passes the verification is obtained. ; S7. Construct a sandbox network, and the first... A synthetic sample and the final enhancement strategy using class awareness. The transformed synthetic image is input into a sandbox network, and the output is the baseline feature set. Enhanced feature groups ; S8. Utilizing the baseline feature set Enhanced feature groups Sandbox loss Utilizing sandbox losses Construct the total loss and train the model using the total loss.

2. The adaptive data governance method for long-tail recognition based on multi-agent collaboration as described in claim 1, characterized in that: The real training set in step S1 The datasets used are CIFAR10-LT, CIFAR100-LT, or ImageNet-LT. The CIFAR10-LT dataset is input into a diffusion model, generating 100,000 synthetic images for each of the 10 categories. The CIFAR100-LT dataset is input into the diffusion model, generating 10,000 synthetic images for each of the 100 categories. The ImageNet-LT dataset is input into the diffusion model, generating 5,000 synthetic images for each category. The synthetic images generated from all the CIFAR10-LT, CIFAR100-LT, and ImageNet-LT datasets constitute the synthetic sample set to be selected. .

3. The adaptive data governance method for long-tail recognition based on multi-agent collaboration as described in claim 2, characterized in that: The diffusion model is a pre-trained EDM model.

4. The adaptive data governance method for long-tail recognition based on multi-agent collaboration according to claim 2, characterized in that, Step S2 includes the following steps: S2-1. Use the real training set The Middle Zhang Image Adjust the resolution to 224×224, and then... The pixel values ​​in the image are normalized, and the Normalize function in torch.transform is used to normalize the 1st pixel value. The images are converted into normalized tensors, and the normalized tensors are input into the CLIP visual encoder to obtain the first image. Zhang Image True sample features All real sample features constitute the real sample feature set. ; S2-2. Building Categories True Sample Feature Matrix , ,in, For the real number space, For real sample feature set The middle category is The number of real sample features , For real training set The number of categories; S2-3. Combine the sample sets to be screened The Middle Zhang composite image Adjust the resolution to 224×224, and then... The pixel values ​​in the composite image are normalized, and the Normalize function in torch.transform is used to normalize the pixel values ​​of the first image. The synthesized image is converted into a normalized tensor, and the normalized tensor is input into the CLIP visual encoder to obtain the first image. Zhang composite image Features of synthetic samples to be screened All the features of the synthetic samples to be screened constitute the feature set of the synthetic samples to be screened. .

5. The adaptive data governance method for long-tail recognition based on multi-agent collaboration according to claim 4, characterized in that, Step S3 includes the following steps: S3-1. Feature matrix of real samples The category is obtained using the torch.mean function in PyTorch. mean vector , ; S3-2. Feature matrix of real samples The category is obtained using the torch.cov function in PyTorch. covariance matrix , ; S3-3. On the covariance matrix The category is obtained using the `torch.linalg.inv` function in PyTorch. inverse of the covariance matrix ; S3-4. Through formula Calculate the first Zhang composite image Manifold Mahalanobis distance In the formula, For transpose; S3-5. For categories... The manifold Mahalanobis distance of all synthesized images is normalized using a Min-Max method to obtain the nth... Purity fraction of each synthetic sample .

6. The adaptive data governance method for long-tail recognition based on multi-agent collaboration according to claim 4, characterized in that, Step S4 includes the following steps: S4-1. Combine the sample sets to be screened According to its ownership Sort the number of synthesized images in each category in descending order, and select the top... Each category serves as the header class. , To round down, select after Each category is used as a tail class. ; S4-2. Use the torch.nn.functional.cosine_similarity function to calculate the number of the header class. The mean vector of features of the synthetic samples to be screened for all synthetic images in each category. , ; S4-3. Through formula The rejection penalty value was calculated. In the formula, To find the L2 norm; S4-4. Through formula The exponential decay coefficient was calculated. In the formula, It is an adjustable hyperparameter; S4-5. Through formula The purity base score after penalty is calculated. .

7. The adaptive data governance method for long-tail recognition based on multi-agent cooperation according to claim 6, characterized in that, Step S5 includes the following steps: S5-1. The first Zhang composite image The data is sequentially input into a ResNet-32 model and a Softmax activation function, and the output is the class probability distribution. For category probability distribution Use the torch.sum function to get the number of... Zhang composite image Predicted entropy ; S5-2. Through formula Calculate the first Quality-perceived topology of each synthetic sample In the formula It is an adjustable hyperparameter; S5-3. Synthesize the sample set to be screened After sorting all synthetic images in descending order of quality-aware topological metric, For each category, retain the synthetic images corresponding to the top-K quality-perceived topological metrics to obtain a high-quality synthetic sample set. High-quality synthetic sample sets With the real training set By concatenating the data, a balanced composite dataset is obtained. .

8. The adaptive data governance method for long-tail recognition based on multi-agent collaboration according to claim 6, characterized in that, Step S6 includes the following steps: S6-1. Balance the composite dataset The input is given to the ResNet-32 model, and the output is a sequence of indicators. The ResNet-32 model was trained for 50 epochs. Output training metric sequence for each epoch , ; S6-2. Output the training metric sequence from the 40th to the 50th epoch. - Inputting into the Qwen2.5-7B-Instruct model, the output is a balanced composite dataset. The Middle Status labels for each category , , For Healthy Learning, Overfitting, Stagnation, Unstable Training; S6-3. Transform the indicator sequence - Status labels Inputting into the Qwen2.5-7B-Instruct model, the output is the first... Enhancement targets in each category , ; S6-4. Enhance the target The input is given to the Qwen2.5-7B-Instruct model, and the output is the head class enhancement strategy. and tail-type enhancement strategies , , For the first Head-class enhancement sub-strategies , , For the first Tail-class enhancement sub-strategies ; S6-5. Enhancing Head Class Strategies and tail-type enhancement strategies The input is fed into the Qwen2.5-7B-Instruct model, and the output is the final class-aware enhancement strategy that has passed the validation. .

9. The adaptive data governance method for long-tail recognition based on multi-agent cooperation according to claim 4, characterized in that, Step S7 includes the following steps: S7-1. Construct a sandbox network consisting of a first convolutional layer, a second convolutional layer, a first batch normalization (BN) layer, a first ReLU activation function, a third convolutional layer, a fourth convolutional layer, a second batch normalization (BN) layer, a second ReLU activation function, a first linear layer, a second linear layer, and a GELU activation function. S7-2. Balancing Composite Datasets The Middle The synthesized image is input into the sandbox network, and the output is the baseline feature set. ; S7-3. Following the final enhancement strategy of class perception Use the torchvision.transforms.Compose function in PyTorch to perform the first... The composite image is transformed to obtain the transformed composite image; S7-4. Input the transformed synthetic image into a sandbox network, and output the enhanced feature set. .

10. The adaptive data governance method for long-tail recognition based on multi-agent collaboration according to claim 8, characterized in that, Step S8 includes the following steps: S8-1. Through formula Sandbox loss calculated In the formula, For batch size, The value is 32. For Gaussian kernel function, To balance composite datasets The Middle The baseline feature set of the composite image, To balance composite datasets The Middle Enhanced feature sets of the composite image; S8-2. When the sandbox is lost Greater than or equal to Then, return to steps S6-S7. The baseline threshold; S8-3. Extracting the weight matrix of fully connected layers in a ResNet-32 model using the state_dict function in PyTorch. The tail category The classification weight vector for each category is: , , No. The classification weight vector for each category is: , Classification weight vector Weight matrix The Rows, classification weight vector Weight matrix The OK; S8-4. Through formula Calculate the classification weight vector With classification weight vector The minimum cosine of the included angle between ; S8-5. Through formula The multidimensional composite loss function was calculated. In the formula, , , All are weighting coefficients. This represents the decrease in cross-entropy loss between the current round and the previous round during training. For compliance reward value, This is the difference between the minimum cosine of the angle in the current round and the minimum cosine of the angle in the previous round during training. S8-6. Using the AdamW optimizer with a multi-dimensional composite loss function The Qwen2.5-7B-Instruct model is trained in step S6-4.

Images