A long-tail fine-grained image recognition method based on multi-expert dynamic fusion

Abstract

The application discloses a long-tail fine-grained image recognition method based on multi-expert dynamic fusion, and realizes precise feature learning and targeted optimization of head common categories, middle medium sample size categories and tail rare categories by constructing a head fine-grained expert module, a middle balanced expert module and a tail adaptive expert module; and designs a dynamic fusion mechanism of uncertainty measurement and consensus punishment, and distributes expert weights according to needs to adapt to image characteristics and realize precise classification. The application combines real scene complex background adaptation, long-tail distribution sample balance and fine-grained classification precision improvement, clearly takes long-tail fine-grained image recognition in a real scene as a core problem, breaks the limitation of laboratory data sets, solves one-sidedness of single problem research, makes research more in line with actual scene technical requirements of wide coverage and fine discrimination, and improves the generalization ability and recognition precision of the model in real scene image classification.

Description

Technical Field

[0001] This invention belongs to the technical field of computer vision and deep learning, specifically involving a long-tailed fine-grained image recognition method based on multi-expert dynamic fusion. It aims to solve the dual core challenges of long-tailed distribution and fine-grained distinction in plant image classification in real-world scenarios, simulating the comprehensive needs of covering common categories, distinguishing similar categories, and identifying rare categories in practical applications of long-tailed fine-grained image recognition. Background Technology

[0002] The performance of image recognition models is highly dependent on the distribution of their training data. Real-world data often differs greatly from meticulously constructed standard datasets in laboratories, rarely exhibiting a balanced, ideal distribution. Instead, it often displays a complex interplay of fine-grained features and long-tailed imbalanced distributions. Long-tailed visual recognition is a significant research direction in computer vision, aiming to address the common problem of extreme imbalance in the number of class samples in real-world data. Existing research has proposed various technical solutions from different perspectives. Based on their core ideas and technical approaches, these can be categorized into three types: The first type is rebalancing methods; their core is to force the model to focus on tail classes by adjusting the data or loss distribution during training. This includes resampling (e.g., oversampling tail classes / undersampling head classes), reweighted loss functions (e.g., Focal Loss), and log-odds adjustment. The second type is information augmentation methods, which aim to supplement the sparsely sampled tail classes with additional information or knowledge by enriching the learning signals of the tail classes. This includes transfer learning / knowledge distillation (transferring knowledge from head classes or large models) and data augmentation (generating new samples through image transformation or blending techniques). The third category is module improvement methods, which start from the structural level and endogenously address long-tail distributions by improving the design of the network architecture. These methods include decoupling training (performing feature learning and classifier adjustment in stages), improving classifier design, and ensemble learning (such as multi-branch networks).

[0003] In existing technologies, Zhang et al. proposed a two-stage SADE method (Yifan Zhang, Bryan Hooi, Lanqing Hong, and Jiashi Feng. “Self-supervised aggregation of diverse experts for test-agnostic long-tailed recognition”. In Proceedings of the 36th International Conference on Neural Information Processing Systems NIPS (2022), 34077–34090.): By employing diverse expert learning strategies and designing a self-supervised expert aggregation strategy, this method is the first to attempt to address unknown class distribution shifts through adaptive aggregation without prior knowledge of the actual test distribution, providing a new approach for handling dynamically changing class distributions in the real world. For the long-tail classification problem, the LFME method proposes a novel multi-expert knowledge distillation framework (Xiang, L., Ding, G., Han, J. "Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-Tailed Classification". In: ECCV (2020).): This method divides the long-tail dataset into multiple cardinally adjacent subsets according to the number of samples in each category, and trains an expert model on each subset. Then, a two-level adaptive learning mechanism is introduced. At the model level, the weights of knowledge distillation are dynamically adjusted so that the student model can gradually outperform the experts. At the instance level, a learning plan from easy to difficult is designed based on the confidence of the expert model output, and the student model is trained in a progressive manner.Hsu et al. proposed a regularization method called ABC-Norm that can simultaneously handle long-tailed distributions and fine-grained classification (Y.-C. Hsu, C.-Y. Hong “ABC-Norm Regularization for Fine-Grained and Long-Tailed ImageClassification”. in IEEE Transactions on Image Processing (2023), vol. 32, pp. 3885-3896.). This method constructs an adaptive batch prediction matrix and applies a regularization loss based on the Frobenius norm. During training, a non-uniform, adaptive classification confusion is introduced to form an adversarial learning mechanism. This design forces the model to learn more robust feature representations while pursuing prediction accuracy, thus simultaneously addressing the challenges of long-tailed imbalance and fine-grained similarity within a unified framework.

[0004] However, the current field of long-tail fine-grained image recognition has obvious shortcomings in adapting to real-world application scenarios and addressing core technical challenges. First, the most critical shortcoming of existing technologies is their one-sided problem-solving approach. Most works only address the single challenge of long-tail distribution or fine-grained classification, failing to consider the need for fusion of both. They either focus on alleviating sample imbalance through resampling and loss function weighting, without considering the extraction of subtle features required for fine-grained differentiation; or they focus on designing attention mechanisms and local feature alignment schemes to address fine-grained classification problems, but assume balanced data distribution, failing to solve the problem of insufficient learning of tail categories due to sample scarcity. This one-sided design makes it unable to cope with the dual needs of plant image recognition and medical image recognition, which require both coverage of long-tail categories and differentiation of fine-grained differences, directly affecting the actual recognition results. Secondly, for technical solutions that only focus on long-tail distributions, their drawback lies in their insufficient ability to distinguish fine-grained features, resulting in limited performance improvement. Existing mainstream classification methods under long-tail distributions are all designed and validated based on general datasets such as CIFAR-LT and ImageNet-LT. These datasets have coarse-grained categories, covering macro categories such as automobiles and furniture, with large inter-class differences but relatively consistent intra-class features. Based on this, the core optimization method that only focuses on balancing the number of samples cannot cope with the core challenge of fine-grained classification. For example, in plant images, the features of organs such as leaves and flowers of the same species change significantly with the growth stage, and closely related species have highly similar subtle features such as leaf veins and petal morphology. Simple sample balancing is difficult to capture these key local features, resulting in insufficient model adaptability to fine-grained long-tail plant data. Furthermore, the few technical solutions that address both long-tail and fine-grained issues also suffer from limitations in applicable scenarios and difficulty in transferability. The datasets used by these methods are either small in scale, with low difficulty in the long tail and small difference in the number of samples at the beginning and end of each category, making it impossible to simulate the extreme imbalance of real-world scenarios; or they focus on categories with low intra-class differences, such as birds and cars, while some categories, such as plants, are greatly affected by their growth environment and stage, resulting in more complex intra-class feature variations. Therefore, the limitations of the datasets themselves mean that the core design of these methods is not adapted to the high variability of plants, making it difficult to achieve ideal results when transferring applications. Furthermore, existing image classification research generally suffers from weak generalization ability and detachment from real-world application scenarios. Most studies rely on small laboratory datasets, with samples collected under controlled environments featuring uniform backgrounds, even lighting, and complete plant morphology. These datasets do not include common interference factors in real-world scenes, such as complex backgrounds, extreme lighting, and incomplete morphology. Consequently, model training lacks robustness against interference and exhibits poor generalization performance. Moreover, these datasets often achieve category balance through manual selection, completely violating the natural long-tail distribution pattern of real-world samples. Model training is based on the assumption of balanced data and does not optimize for rare plant categories, resulting in extremely low accuracy in recognizing rare categories in real-world scenarios, failing to meet practical needs. Summary of the Invention

[0005] The main objective of this invention is to overcome the shortcomings and deficiencies of existing technologies and provide a long-tail fine-grained image recognition method based on multi-expert dynamic fusion. By constructing diverse expert modules to adapt to the learning needs of different sample distribution categories, the method intelligently allocates weights based on expert prediction confidence and consensus through a dynamic fusion mechanism, and specifically integrates the advantages of each expert to achieve accurate classification.

[0006] To achieve the above objectives, the present invention provides a long-tailed fine-grained image recognition method based on multi-expert dynamic fusion, comprising the following steps:

[0007] A long-tail fine-grained image recognition model is constructed, including an expert-shared feature extraction module, a diversified expert learning module, and a dynamic fusion module. The diversified expert learning module includes a head fine-grained expert module, a middle balancing expert module, and a tail adaptation expert module. The expert-shared feature extraction module, the diversified expert learning module, and the dynamic fusion module are connected sequentially. The head fine-grained expert module, the middle balancing expert module, and the tail adaptation expert module are connected in parallel.

[0008] Obtain a long-tailed, fine-grained image set and input it into the expert-shared feature extraction module to extract image features;

[0009] The image features are input into the fine-grained head expert module to obtain the head classification results;

[0010] Input the image features into the central balance expert module to obtain the central classification results;

[0011] Input the image features into the tail adaptation expert module to obtain the tail classification results;

[0012] The head classification results, middle classification results, and tail classification results are input into the dynamic fusion module to calculate the prediction confidence and consensus of each expert module and assign basic weights and penalty coefficients to each expert module. Then, the results are fused with the head classification results, middle classification results, and tail classification results to obtain the final classification result.

[0013] As a preferred technical solution, the expert shared feature extraction module is composed of the first two layers of a ResNet-50 network.

[0014] As a preferred technical solution, the head fine-grained expert module introduces center loss and Top-k negative sample loss for constraint;

[0015] The center loss reduces intra-class discrepancies by constraining features of samples of the same class to aggregate towards the class center, and is expressed as:

[0016] ,

[0017] Among them, Lc The center loss value is denoted by m, where m is the number of sample classes, and x is the center loss value. i Let be the feature vector of the i-th sample. For the yth i The center vector of each category, y i Let i be the category of the i-th sample;

[0018] The Top-k negative sample loss enhances the inter-class feature discrimination by selecting the top k negative classes with the highest confidence scores from all negative classes. This is expressed as:

[0019] ,

[0020] Among them, L k Let s be the loss value for the top-k negative samples, s be the confidence score vector, and l be the true label of the sample. l J represents the confidence score corresponding to the true label of the sample. ' For the set of all negative classes, Let s be the set of the top k negative classes with the highest confidence scores. i Let be the confidence score of the i-th negative category.

[0021] As a preferred technical solution, the central balancing expert module uses a balancing Softmax loss function for constraints; the balancing Softmax loss function is expressed as:

[0022] ,

[0023] Among them, L bal To balance the softmax loss value, D s ={x i ,y i} i=1 To input the training set of the central balance expert module, n s x is the number of samples in the training set. i For the i-th sample, y i Let be the true label of the i-th sample, σ() be the Softmax function, v2() be the unnormalized log probability value output by the middle balance expert module, and π be the prior distribution of the training label frequencies. , k is the kth category, π k Let n be the prior probability of the k-th class. k This represents the total number of samples in the k-th category in the training set.

[0024] As a preferred technical solution, the tail adaptation expert module is constrained by the inverse Softmax loss function; the inverse Softmax loss function is expressed as:

[0025] ,

[0026] Among them, L inv D is the inverse softmax loss value. s ={x i ,y i} i=1 For the training set of the input tail adaptation expert module, n s x is the number of samples in the training set. i For the i-th sample, y i Let be the true label of the i-th sample, σ() be the Softmax function, v3() be the unnormalized log probability value output by the tail-adaptation expert module, and λ be a hyperparameter. It is obtained by reversing the order of the training label frequencies π.

[0027] As a preferred technical solution, the steps for obtaining the basic weights are as follows:

[0028] The Top-k entropy is used to calculate the prediction uncertainty of each expert module for each sample in the long-tailed fine-grained image set. The calculation formula is as follows:

[0029] ,

[0030] in, This represents the uncertainty in the prediction of sample x by expert module e. This represents the probability value obtained by taking the k1 categories with the highest probability values ​​from the prediction probability distribution of sample x by the e-th expert module;

[0031] Z-score normalization based on global entropy distribution and Sigmoid mapping are used to assign basic weights to each expert module.

[0032] The Z-score normalization formula for the global entropy distribution is:

[0033] ,

[0034] Among them, z e μ is the normalized value of the e-th expert module. H Let σ be the mean of the global entropy. H The standard deviation of global entropy;

[0035] The Sigmoid mapping formula is: Conf e = α·Sigmoid(γ·z e )+β,Conf e Let α be the confidence score of the e-th expert module, β be the amplitude coefficient, γ be the offset constant, and γ be the scaling factor.

[0036] The formula for calculating the basic weights is as follows:

[0037] ,

[0038] in, Here, represents the base weights for the e-th expert module of sample x, where N is the total number of experts. e (x) represents the confidence score of the e-th expert module for sample x.

[0039] As a preferred technical solution, the step of obtaining the penalty coefficient is as follows:

[0040] First, calculate the prediction entropy value of each expert module and the consensus degree between any two expert modules, described as follows:

[0041] ,

[0042] ,

[0043] in, Let p be the topk2 probability distribution of the e-th expert module. e Let S be the original probability distribution output by the e-th expert module for sample x, k2 be the number of top k2 categories considered in the calculation, and S be the probability distribution of the sample x. ij (x) represents the consensus degree between expert modules i and j for sample x, where i ≠ j, and II() is the indicator function. It is the m-th topk2 category index of expert module i. Let n be the index of the nth topk2 category of expert module j. Let be the topk2 probability distribution of the i-th expert module. Let ||||2| be the topk2 probability distribution of the j-th expert module, and ||||2| be the L2 norm.

[0044] The average consensus and relative consensus are calculated based on the consensus among all expert modules. The calculation formula is as follows:

[0045] ,

[0046] ,

[0047] in, Let r be the average consensus score of the i-th expert module, N be the number of expert modules, and r be the average consensus score of the i-th expert module. i (x) represents the relative consensus degree of expert module i. Let J be the average consensus score of the j-th expert module;

[0048] Furthermore, the penalty coefficient is dynamically adjusted based on the Top-k weighted cosine overlap of the multi-expert module's predicted distribution. The calculation formula is as follows:

[0049] ,

[0050] Among them, P i (x) is the penalty coefficient for expert module i, Conf i Let be the confidence score of expert module i. Let be the sample average entropy, 'a' be the basic bias constant, and 'b' be the amplitude scaling constant.

[0051] As a preferred technical solution, the final result obtained is specifically as follows:

[0052] The final weight of each expert module is calculated by combining the base weight and penalty coefficient of each expert module.

[0053] The final predicted value is obtained by weighting and summing the final weights of each expert module with the results of each expert module.

[0054] Another aspect of the present invention provides a long-tail fine-grained image recognition system based on multi-expert dynamic fusion, which is applied to the above-mentioned long-tail fine-grained image recognition method based on multi-expert dynamic fusion. The system includes a model building module, a feature extraction module, a head classification module, a middle classification module, a tail classification module, and a fusion classification module.

[0055] The model building module is used to construct a long-tailed fine-grained image recognition model, including an expert shared feature extraction module, a diversified expert learning module, and a dynamic fusion module; the diversified expert learning module includes a head fine-grained expert module, a middle balancing expert module, and a tail adaptation expert module; the expert shared feature extraction module, the diversified expert learning module, and the dynamic fusion module are connected in sequence; the head fine-grained expert module, the middle balancing expert module, and the tail adaptation expert module are connected in parallel;

[0056] The feature extraction module is used to acquire a long-tailed fine-grained image set and input it into the expert-shared feature extraction module to extract image features;

[0057] The head classification module is used to input image features into the head fine-grained expert module to obtain head classification results;

[0058] The middle classification module is used to input image features into the middle balance expert module to obtain the middle classification results;

[0059] The tail classification module is used to input image features into the tail adaptation expert module to obtain tail classification results;

[0060] The fusion classification module is used to input the head classification results, middle classification results, and tail classification results into the dynamic fusion module to calculate the prediction confidence and consensus of each expert module and assign basic weights and penalty coefficients to each expert module. Then, it is fused with the head classification results, middle classification results, and tail classification results to obtain the final classification result.

[0061] In another aspect, the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned long-tail fine-grained image recognition method based on multi-expert dynamic fusion.

[0062] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0063] This invention achieves synergistic optimization of long-tail distribution adaptation and fine-grained classification, and the fusion mechanism can dynamically adapt to image characteristics, effectively avoiding the problems of expert preference misjudgment and opinion conflict. This is due to two differentiated technical means: First, a long-tail fine-grained image recognition model is constructed. Unlike the existing multi-expert model, which simply divides experts into head, middle and tail experts without considering the fine-grained requirements, this model extracts basic features using an expert shared feature extraction module. At the same time, it sets up three independent expert modules: head fine-grained, middle balanced, and tail adaptation. For common categories in plant images (large intra-class differences and high inter-class similarity), middle medium sample categories (insufficient sample balance), and tail rare categories (scarce samples), it designs exclusive optimization schemes for center loss + Top-k negative sample loss, balanced softmax loss, and inverse softmax loss. This achieves accurate feature learning through divide and conquer, and strengthens the fine-grained discrimination ability and long-tail category adaptation.

[0064] Secondly, a dynamic fusion mechanism based on uncertainty measurement and consensus penalty is proposed, which overcomes the limitations of existing fusion strategies such as fixed weights, voting methods, and gating methods in adapting to the complex characteristics of long-tailed, fine-grained images. This mechanism first quantifies the uncertainty of expert predictions using Top-k entropy, and then allocates basic weights by combining Z-score normalization and Sigmoid mapping to achieve accurate matching between expert advantages and sample characteristics. Next, it quantifies the differences in expert opinions based on Top-k weighted cosine overlap, determines the sample difficulty by combining the sample average entropy, and dynamically calculates the penalty coefficient to calibrate the basic weights. For simple samples, the penalty for dissenting experts is increased to suppress erroneous opinions, while for complex samples, the penalty is moderate to retain the correct judgments of the minority. Finally, the fusion prediction result is obtained by weighted summation of the logit values ​​of each expert, effectively solving the problems of expert preference misjudgment and multiple opinion conflicts, and improving the model's generalization ability and recognition accuracy in real-world image classification. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a flowchart of a long-tailed fine-grained image recognition method based on multi-expert dynamic fusion in an embodiment of the present invention.

[0067] Figure 2 This is a framework diagram of the long-tailed fine-grained image recognition method based on multi-expert dynamic fusion in an embodiment of the present invention.

[0068] Figure 3 This is a structural diagram of the long-tailed fine-grained image recognition system based on multi-expert dynamic fusion in an embodiment of the present invention.

[0069] Figure 4 This is a structural diagram of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation

[0070] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0071] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0072] This invention proposes a multi-expert dynamic fusion method to address the dual core challenges of long-tail distribution and fine-grained differentiation in plant image classification in real-world scenarios. It simulates the comprehensive needs of covering common categories, distinguishing similar categories, and identifying rare categories in practical applications of long-tail, fine-grained image recognition. This method addresses the limitations of existing research that relies on ideally balanced datasets and tackles single problems individually. It organically combines adaptation to complex real-world backgrounds, balancing long-tail distribution samples, and improving fine-grained classification accuracy. Through a divide-and-conquer approach combined with dynamic fusion, it solves the problem by adapting different experts to different categories and dynamically balancing expert contributions with weights. This method constructs three diverse expert modules to adapt to the learning needs of different sample distribution categories. The dynamic fusion module intelligently allocates weights based on expert prediction confidence and consensus, specifically integrating the advantages of each expert module to achieve accurate classification. Figure 1 As shown, this embodiment of the long-tailed fine-grained image recognition method based on multi-expert dynamic fusion includes the following steps:

[0073] S1. Construct a long-tailed fine-grained image recognition model, including an expert-shared feature extraction module, a diversified expert learning module, and a dynamic fusion module; the diversified expert learning module includes a head-level fine-grained expert module, a middle-level balancing expert module, and a tail-level adaptation expert module. Based on the long-tailed distribution characteristics of long-tailed fine-grained images and the difficulties in fine-grained classification, this invention constructs three types of expert modules with clearly defined functions, each adept at handling different distributions, achieving precise feature learning tailored to specific tasks. The structure of the long-tailed fine-grained image recognition model is as follows: Figure 2 As shown, the expert-shared feature extraction module f θ The system consists of a multi-expert learning module and a dynamic fusion module, connected sequentially. Within the multi-expert learning module, the head fine-grained expert module E1, the middle balanced expert module E2, and the tail adapting expert module E3 are connected in parallel. Multi-expert architecture is a commonly used model approach in computer vision and deep learning. This architecture, often called a hybrid expert system, utilizes multiple experts to learn different content in a targeted manner to solve a complex task. Common methods include adding a gating network after the multi-expert model to select the most suitable expert for different inputs, or using simple strategies such as averaging or selecting the highest value to choose the expert's output. This method employs dynamic fusion, connected after the multi-expert module, dynamically fusing the outputs of the expert modules. Specifically, it assigns weights to the classification results of the three expert modules based on the input features and dynamically adjusts them to obtain the final classification result.

[0074] S2. Obtain a long-tailed fine-grained image set and input it into the expert shared feature extraction module to extract image features.

[0075] Furthermore, this invention uses the first two layers of the ResNet-50 network as the expert-shared feature extraction module f.θ Extract the basic features of the image.

[0076] S3. Input the image features into the head fine-grained expert module to obtain the head classification results.

[0077] Furthermore, the head-level fine-grained expert module E1 specializes in common categories with sufficient samples, focusing on solving the fine-grained challenge of large intra-class differences and high inter-class similarity. This expert module introduces center loss and top-k negative sample loss for constraint. Specifically, center loss constrains the aggregation of features from samples of the same class towards the class center, reducing the interference from intra-class differences.

[0078] ,

[0079] Among them, L c For the center loss value, x i Let be the feature vector of the i-th sample. For the yth i The center vectors of each category, where m is the number of samples.

[0080] Unlike traditional cross-entropy loss, which considers all negative classes equally, Top-k negative sample loss strengthens contrastive learning by selecting the k negative classes with the highest confidence scores (mostly closely related species), specifically focusing on easily confused classes and improving the discriminative power of inter-class features.

[0081] ,

[0082] Among them, L k Let s be the loss value for the top-k negative samples, s be the confidence score vector, and l be the true label of the sample. l J represents the confidence score corresponding to the true label of the sample. ' For the set of all negative classes, Let s be the set of the top k negative classes with the highest confidence scores. i Let J be the confidence score for the i-th negative class. ' Let s be the set of all negative classes. ' = {s i ,i∈J '} is a set containing all negative class confidence scores; obtain s ' The k-th highest value in the middle is denoted as t. k ; use t k As a threshold pair s ' Divide J into segments. ' Divide into J ' >and J ' <:

[0083] ,

[0084] in It includes the negative categories whose confidence scores rank in the top k among all negative categories, while J ' < is the set of negative categories whose confidence scores rank below the top k. In this invention, negative categories refer to all other categories besides the true label of the sample.

[0085] S4. Input the image features into the central balance expert module to obtain the central classification results.

[0086] Furthermore, the E2 balanced expert module is adapted for classes with a medium sample size (20-100 samples per class). Its core objective is to correct classifier bias caused by uneven sample frequency. These classes lack the sufficient samples of the head classes to support complex fine-grained learning, and also avoid the extreme sample scarcity problem of the tail classes. Therefore, a balanced Softmax loss function is used to achieve balanced class prediction. This loss function uses the label frequency of each class in the training set as a prior and adjusts the unnormalized logit value (logit) of the model output. Specifically, it increases the logit of classes with fewer samples and decreases the logit of classes with more samples, thereby offsetting the model's natural bias towards high-frequency classes. The balanced Softmax loss function is expressed as:

[0087] ,

[0088] Among them, L bal To balance the softmax loss values, define D s ={x i ,y i} i=1 To input the training set of the central balance expert module, n s x is the number of samples in the training set. i For the i-th sample, y i Let v1 be the true label of the i-th sample, v2() be the logit value output by the middle balance expert module E2, σ() be the softmax function, and π be the prior distribution of the training label frequencies. , k is the kth category, π k Let n be the prior probability of the k-th class. k is the total number of samples in the k-th class in the training set; the training label frequency is the proportion of samples of each true class in the training set to the total number of samples. Through this adjustment, the prediction results output by E2 can simulate the effect of uniform class distribution, avoid the middle classes being dominated by the head classes during training, and ensure the sufficiency of feature learning for middle sample classes.

[0089] S5. Input the image features into the tail adaptation expert module to obtain the tail classification results.

[0090] Furthermore, the tail-adaptation expert module E3 specifically overcomes the bottleneck of scarce samples in rare categories. Its core is to force the model to focus on tail-category features through the inverse Softmax loss function. The inverse Softmax loss function is expressed as:

[0091] ,

[0092] Among them, L inv Let v3() be the logit value output by the tail adaptation expert E3, and λ be a hyperparameter. This represents the inverse softmax loss value. This is obtained by reversing the order of the training label frequencies π. For example, π corresponds to categories 12345, and it is reversibly assigned to categories 54321. Category 5 then obtains the category label frequencies π of category 1 as its inverse training prior. The key to this loss function lies in the inverse adjustment, which during training gives a huge reward to the head class (significantly increasing its effective logit) and punishes the tail class (reducing its effective logit), forcing the model to devote more effort to learning the features of these rare classes, thereby alleviating the feature underfitting problem caused by insufficient samples in the tail class.

[0093] S6. Input the head classification results, middle classification results, and tail classification results into the dynamic fusion module to calculate the prediction confidence and consensus of each expert module and assign basic weights and penalty coefficients to each expert module. Then, fuse them with the head classification results, middle classification results, and tail classification results to obtain the final classification result.

[0094] Furthermore, existing multi-expert model fusion strategies (fixed weights, voting, gating) have significant limitations and fail to adapt to the core characteristics of plant images, such as significant long-tail distribution, high inter-class similarity, and large intra-class differences. Specifically, in plant image classification tasks, on the one hand, some experts have inherent biases. For example, the tail-adapting expert (E3, due to training with an inverse long-tail distribution, may overconfidently misclassify non-tail samples as tail categories, leading to highly confident incorrect answers. On the other hand, when faced with rare tail samples, the head expert (E1), due to limited sample exposure, may habitually misclassify them as common head categories. Relying solely on a single fusion logic would directly ignore the correct opinions of tail experts. Therefore, the fusion mechanism must simultaneously address expert bias misjudgment and effective trade-offs when multiple opinions conflict, requiring dynamic adaptation to plant image characteristics. To address this issue, this invention proposes a dynamic weight fusion mechanism based on uncertainty measurement and consensus penalty. The core is to achieve intelligent fusion by using a dual-module collaboration that respects expert strengths while avoiding expert bias, enabling on-demand weight allocation. This includes:

[0095] S61. Uncertainty Measurement: Based on the fundamental weight allocation of Top-k entropy, it matches expert advantage with sample characteristics. Specifically:

[0096] First, Top-k entropy is used to calculate the prediction uncertainty of each expert module for each sample in the long-tailed fine-grained image set (the lower the entropy value, the higher the confidence score, and the lower the uncertainty). Based on this, basic weights are allocated to ensure that the expert advantages are accurately matched with the sample characteristics. The core calculation logic is to quantify the confidence level of the expert module for the sample by calculating the Top-k entropy value of the expert prediction results.

[0097] ,

[0098] in, This represents the uncertainty in the prediction of sample x by expert module e. This represents the probability value obtained by the e-th expert module after taking the k1 categories with the highest probability values ​​in the predicted probability distribution of sample x. The lower the entropy value, the greater the expert module's confidence in classifying the sample. Therefore, the expert module is more confident in its classification results and is assigned a higher weight.

[0099] Next, based on Z-score normalization and Sigmoid mapping according to the global entropy distribution, basic weights are assigned to each expert module, described as follows:

[0100] ,

[0101] Conf e = α·Sigmoid(γ·z e )+β,

[0102] ,

[0103] Among them, z e μ is the normalized value of the e-th expert module. H Let σ be the mean of the global entropy. H Conf is the standard deviation of global entropy. e Let be the confidence level of the e-th expert module, α be the amplitude coefficient, β be the offset constant, and γ be the scaling factor. Here, represents the base weights for the e-th expert module of sample x, where N is the total number of experts. e(x) represents the confidence score of the e-th expert module for sample x. The specific application scenarios for uncertainty measurement are as follows: If the test sample is a rare tail category, the tail expert modules, due to their focus on rare category features during training, have low prediction entropy (high confidence score), while the head and middle expert modules, due to less sample exposure, have high entropy (low confidence score). The mechanism automatically assigns higher weights to the tail expert modules, while reducing the weights of the head and middle expert modules, prioritizing the judgments of the tail expert modules. When testing fine-grained, easily confused samples (such as roses vs. rose bushes), the head expert module has the lowest entropy, and its weight is increased to ensure fine-grained discrimination accuracy. In this embodiment, the global entropy mean μ... H =0.6963, global entropy standard deviation σ H =0.5732 is an empirical value. The amplitude coefficient α = 0.95, the offset constant β = 0.05, and the scale factor γ = -0.8.

[0104] S62. Consensus Penalty: Based on Top-k weighted cosine overlap calibration, it avoids expert bias misjudgment.

[0105] To address the issues of expert overconfidence-driven misjudgments and conflicting opinions among multiple experts, this invention introduces a consensus penalty module that integrates Top-k weighted cosine overlap (opinion differences) and sample difficulty (overall uncertainty). This module calibrates weight allocation to dynamically adjust the penalty intensity, preventing erroneous opinions from dominating while maintaining consistent decision-making for simple samples. The core logic is as follows: First, the sample difficulty is determined by calculating the mean of the prediction entropy values ​​of all expert modules—a lower mean indicates higher confidence scores from all expert modules, suggesting a simpler sample; a higher mean indicates greater ambiguity in the overall understanding of the sample by the expert modules, suggesting a more complex sample. Then, the penalty coefficient is dynamically adjusted by combining the Top-k weighted cosine overlap of the multi-expert prediction distribution (lower overlap indicates greater opinion differences). The specific calculation steps are as follows:

[0106] First, calculate the prediction entropy value of each expert module and the consensus degree between any two expert modules, described as follows:

[0107] ,

[0108] ,

[0109] in, Let p be the topk2 probability distribution of the e-th expert module. e Let S be the original probability distribution of the output of the e-th expert module for sample x, k2 be the number of top k2 categories considered in the calculation, and S be the probability distribution of the output of the e-th expert module for sample x. ij (x) represents the consensus degree between expert modules i and j for sample x, where i ≠ j, and II() is an indicator function that, when the condition within its parentheses is true (i.e., ..., II()...), ... The function evaluates to 1 if the two expert modules focus on the same category at positions m and n; otherwise, it evaluates to 0, indicating inconsistent categories. This function ensures that the product of the probabilities of the two expert modules is included in the consensus calculation only when their topk2 predictions are of the same category. It is the m-th topk2 category index of expert module i. Let n be the index of the nth topk2 category of expert module j. Let be the topk2 probability distribution of the i-th expert module. Let ||j|| be the topk2 probability distribution of the j-th expert module, and ||j|| be the L2 norm.

[0110] Then, based on the consensus among all expert modules, the average consensus and relative consensus are calculated using the following formula:

[0111] ,

[0112] ,

[0113] in, Let r be the average consensus score of the i-th expert module, N be the number of expert modules, and r be the average consensus score of the i-th expert module. i (x) represents the relative consensus degree of expert module i. Let be the average consensus degree of the j-th expert module.

[0114] Furthermore, the penalty coefficient is dynamically adjusted based on the Top-k weighted cosine overlap of the multi-expert module's predicted distribution. The calculation formula is as follows:

[0115] ,

[0116] Among them, P i (x) is the penalty coefficient for expert module i, Conf i Let be the confidence score of expert module i. Let be the sample average entropy, 'a' be the basic bias constant, and 'b' be the amplitude scaling constant.

[0117] S63. Fusion Prediction: Calculate the final weight of each expert module by combining the basic weights and penalty coefficients of each expert module, expressed as: Next, the final predicted value is obtained by weighting and summing the final weights of each expert module with the results of each expert module, and is described as follows: In this embodiment, a=0.2 and b=0.6.

[0118] When the sample is a simple sample (such as common wheat leaves, where the mean entropy value of all experts is low and the uncertainty is low), if an expert's opinion differs greatly from that of other experts, it indicates that the expert has obvious biased misjudgment (such as the tail expert module misclassifying wheat as a rare plant). In this case, the penalty coefficient P is reduced, the penalty intensity is increased, the weight of the expert is reduced, the minority erroneous opinion is suppressed, and the high accuracy of simple samples is ensured. When the sample is a complex sample (such as closely related roses vs. rose bushes, or the rare plant Davidia involucrata, where the mean entropy value of all experts is high and the uncertainty is high), if there are differences in the opinions of multiple experts, it indicates that the sample itself is difficult to distinguish. In this case, the penalty coefficient P is reduced, only slightly reducing the weight of biased experts, to avoid missing the minority correct opinion due to excessive penalty (such as the tail expert module's correct judgment of rare categories).

[0119] The diverse expert modules designed in this invention have differentiated domain adaptability. Different expert modules have their own advantages for the head, middle and tail categories in the long tail distribution. Their performance differences and expertise are shown in Table 1.

[0120] Table 1. Top-1 accuracy of each expert module on the Pl@ntNet-300K dataset.

[0121]

[0122] As shown in Table 1, the three expert modules in this model can learn different key points in a targeted manner. The fine-grained head expert module performs excellently in the head category, achieving an accuracy of 74.52%. The middle balancing expert module focuses more on the balance between categories, thus its performance is relatively more balanced, with accuracy rates of 70.14%, 58.16%, and 41.27% for the head, middle, and tail categories, respectively. The tail adaptation expert module focuses on feature learning for the tail category, achieving an accuracy of 47.71%. The above demonstrates that the multi-expert dynamic fusion model proposed in this method can learn three experts with distinct specializations on the same dataset, effectively improving the model's overall performance across different category distributions.

[0123] Tables 2 and 3 show the comparison of the effects of this invention with existing technologies. All experiments were conducted on a deep learning server equipped with four NVIDIA Titan XP computing graphics cards. The model used ResNet50 as the backbone network, the batch size was set to 128, and the SGD optimizer was used for training. The initial learning rate was 1*10. -3The model was trained for a total of 100 epochs, with the first 5 epochs using a learning rate warm-up strategy. Consistent with existing work, this invention employs the following commonly used performance metrics for quantitative evaluation and analysis: Overall Accuracy (ACC) calculates the classification correctness of all samples, used to evaluate the overall classification performance of the model; Many-shot Accuracy / Medium-shot Accuracy / Few-shot Accuracy is divided according to the number of samples, evaluating the model's classification accuracy on head, middle, and tail categories respectively. This can intuitively reveal whether the model favors head categories and neglects tail categories. The specific division criteria are consistent with other works: categories with more than 100 training samples are defined as head categories, those with less than 20 are defined as tail categories, and the rest are middle categories; Macro Accuracy is calculated by first calculating the classification accuracy for each category separately, and then taking the arithmetic mean of the accuracy for all categories. This is used to fairly evaluate the model's average performance across all categories, unaffected by imbalanced data distribution. In the comparative experiments of this invention, four existing methods were considered as baselines: ResNet50, SADE, LFME, and ABC-Norm. SADE employs a multi-expert learning architecture, which aggregates multi-expert prediction results through a self-supervised learning mechanism during testing. LFME achieves expert diversification learning by having three experts learn from disjoint subsets of the dataset, and uses knowledge distillation to train student models to aggregate expert knowledge. ABC-Norm is a method specifically proposed for long-tailed fine-grained recognition problems. Its core idea is to introduce a constraint term into the loss function that can simultaneously model the long-tailed distribution (through the class adaptive matrix) and fine-grained features (by minimizing the rank of the prediction matrix to enhance the discriminative power between classes), thereby achieving joint optimization of the long-tailed distribution and fine-grained class confusion problems.

[0124] Table 2. Top-1 accuracy of this invention on the Pl@ntNet-300K dataset

[0125]

[0126] Table 3. Top-1 accuracy of the present invention on the plant subset of the iNaturalist2017 dataset.

[0127]

[0128] Experimental results show that the present invention achieves superior performance on both datasets: on the Pl@nt-300K dataset, the method achieves an overall accuracy of 80.18%, a macro accuracy of 52.96%, and head, middle, and tail accuracies of 74.05%, 53.49%, and 41.1%, respectively; on the iNaturalist2017 subset, it achieves an overall accuracy of 72.72%, a macro accuracy of 71.79%, and head, middle, and tail accuracies of 81.88%, 68.54%, and 70.51%, respectively.

[0129] Compared to baseline methods, this invention significantly improves tail category accuracy and macro accuracy while maintaining head category and overall classification performance, demonstrating its advantage in handling long-tail distributions. A detailed comparative analysis is as follows:

[0130] Compared with ResNet50, as a basic backbone network, ResNet50 focuses too much on the head class with a large sample size. Although it can achieve a high overall accuracy, its tail accuracy is low due to the neglect of tail class feature learning. Furthermore, because the number of tail classes is large, its macro accuracy is also very low. This indicates that the feature representation it learns has obvious class bias and tends to favor the head class with a larger sample size.

[0131] A comparison with SADE and LFME. SADE and LFME are representative methods designed for image recognition tasks with long-tailed distributions. However, these two methods are mainly designed and validated on coarse-grained general datasets such as CIFAR-LT and ImageNet-LT. Their characteristics of large inter-class differences and relatively consistent intra-class features are fundamentally different from the task of plant image recognition. Therefore, when these two methods are directly transferred to plant image recognition tasks, although they can alleviate the impact of long-tailed distributions to some extent, their ability to recognize mid-tailed categories is still insufficient, further leading to low macro-accuracy. The multi-expert dynamic fusion method proposed in this invention, because it also considers fine-grained issues, performs better on plant images. It can balance the long-tailed distribution and further improve performance by solving its fine-grained problems.

[0132] Comparison with ABC-Norm. As one of the few dedicated methods that simultaneously considers long-tail distribution and fine-grained problems, it performs well in terms of mid-class accuracy and macro accuracy. On the Pl@nt-300K dataset, its mid-class accuracy is 0.1% higher than that of this invention. However, because it also suffers from the problem of using a small dataset and low long-tail difficulty, it fails to fully adapt to the characteristics of plant image recognition. Therefore, its tail accuracy and macro accuracy are still 14.08% and 5.83% lower than those of this study, respectively.

[0133] In summary, by jointly modeling long-tail distribution and fine-grained features, this invention significantly improves the recognition ability of mid-tail categories while maintaining the performance of head categories, achieving a more balanced and fair learning effect among categories.

[0134] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0135] Based on the same idea as the long-tailed fine-grained image recognition method based on multi-expert dynamic fusion in the above embodiments, the present invention also provides a long-tailed fine-grained image recognition system based on multi-expert dynamic fusion. This system can be used to execute the above-described long-tailed fine-grained image recognition method based on multi-expert dynamic fusion. For ease of explanation, the structural schematic diagram of the embodiment of the long-tailed fine-grained image recognition system based on multi-expert dynamic fusion only shows the parts related to the embodiments of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0136] like Figure 3 As shown, another embodiment of the present invention provides a long-tail fine-grained image recognition system based on multi-expert dynamic fusion, including a model building module, a feature extraction module, a head classification module, a middle classification module, a tail classification module, and a fusion classification module;

[0137] The model building module is used to construct a long-tailed fine-grained image recognition model, including an expert shared feature extraction module, a diversified expert learning module, and a dynamic fusion module. The diversified expert learning module includes a head fine-grained expert module, a middle balancing expert module, and a tail adaptation expert module. The expert shared feature extraction module, the diversified expert learning module, and the dynamic fusion module are connected in sequence. The head fine-grained expert module, the middle balancing expert module, and the tail adaptation expert module are connected in parallel.

[0138] The feature extraction module is used to acquire a long-tailed, fine-grained image set and input it into the expert-shared feature extraction module to extract image features;

[0139] The head classification module is used to input image features into the head fine-grained expert module to obtain head classification results;

[0140] The middle classification module is used to input image features into the middle balance expert module to obtain the middle classification results;

[0141] The tail classification module is used to input image features into the tail adaptation expert module to obtain tail classification results;

[0142] The fusion classification module is used to input the head classification results, middle classification results, and tail classification results into the dynamic fusion module to calculate the prediction confidence and consensus of each expert module and assign basic weights and penalty coefficients to each expert module. Then, it is fused with the head classification results, middle classification results, and tail classification results to obtain the final classification result.

[0143] It should be noted that the long-tailed fine-grained image recognition system based on multi-expert dynamic fusion of the present invention corresponds one-to-one with the long-tailed fine-grained image recognition method based on multi-expert dynamic fusion of the present invention. The technical features and beneficial effects described in the embodiments of the long-tailed fine-grained image recognition method based on multi-expert dynamic fusion are applicable to the embodiments of the long-tailed fine-grained image recognition system based on multi-expert dynamic fusion. For details, please refer to the description in the embodiments of the method of the present invention, which will not be repeated here.

[0144] Furthermore, in the above embodiments of the long-tailed fine-grained image recognition system based on multi-expert dynamic fusion, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or software implementation convenience. That is, the internal structure of the long-tailed fine-grained image recognition system based on multi-expert dynamic fusion is divided into different program modules to complete all or part of the functions described above.

[0145] like Figure 4 As shown, in one embodiment, a computer-readable storage medium is provided, storing a program in a memory. When the program is executed by a processor, it implements the long-tail fine-grained image recognition method based on multi-expert dynamic fusion, specifically:

[0146] A long-tail fine-grained image recognition model is constructed, including an expert-shared feature extraction module, a diversified expert learning module, and a dynamic fusion module. The diversified expert learning module includes a head fine-grained expert module, a middle balancing expert module, and a tail adaptation expert module. The expert-shared feature extraction module, the diversified expert learning module, and the dynamic fusion module are connected in sequence. The head fine-grained expert module, the middle balancing expert module, and the tail adaptation expert module are connected in parallel.

[0147] Obtain a long-tailed, fine-grained image set and input it into the expert-shared feature extraction module to extract image features;

[0148] The image features are input into the fine-grained head expert module to obtain the head classification results;

[0149] Input the image features into the central balance expert module to obtain the central classification results;

[0150] Input the image features into the tail adaptation expert module to obtain the tail classification results;

[0151] The head classification results, middle classification results, and tail classification results are input into the dynamic fusion module to calculate the prediction confidence and consensus of each expert module and assign basic weights and penalty coefficients to each expert module. Then, the results are fused with the head classification results, middle classification results, and tail classification results to obtain the final classification result.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0153] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A long-tailed fine-grained image recognition method based on multi-expert dynamic fusion, characterized in that, Includes the following steps: A long-tail fine-grained image recognition model is constructed, including an expert-shared feature extraction module, a diversified expert learning module, and a dynamic fusion module. The diversified expert learning module includes a head fine-grained expert module, a middle balancing expert module, and a tail adaptation expert module. The expert-shared feature extraction module, the diversified expert learning module, and the dynamic fusion module are connected sequentially. The head fine-grained expert module, the middle balancing expert module, and the tail adaptation expert module are connected in parallel. Obtain a long-tailed, fine-grained image set and input it into the expert-shared feature extraction module to extract image features; The image features are input into the fine-grained head expert module to obtain the head classification results; Input the image features into the central balance expert module to obtain the central classification results; Input the image features into the tail adaptation expert module to obtain the tail classification results; The head classification results, middle classification results, and tail classification results are input into the dynamic fusion module to calculate the prediction confidence and consensus of each expert module and assign basic weights and penalty coefficients to each expert module. Then, the results are fused with the head classification results, middle classification results, and tail classification results to obtain the final classification result. The steps for obtaining the penalty coefficient are as follows: First, calculate the prediction entropy value of each expert module and the consensus degree between any two expert modules, described as follows: ， ， in, Let p be the topk2 probability distribution of the e-th expert module. e Let S be the original probability distribution output by the e-th expert module for sample x, k2 be the number of top k2 categories considered in the calculation, and S be the probability distribution of the sample x. ij (x) represents the consensus degree between expert modules i and j for sample x, where i ≠ j, and II() is the indicator function. It is the m-th topk2 category index of expert module i. Let n be the index of the nth topk2 category of expert module j. Let be the topk2 probability distribution of the i-th expert module. Let ||||2| be the topk2 probability distribution of the j-th expert module, and ||||2| be the L2 norm. The average consensus and relative consensus are calculated based on the consensus among all expert modules. The calculation formula is as follows: ， ， in, Let r be the average consensus score of the i-th expert module, N be the number of expert modules, and r be the average consensus score of the i-th expert module. i (x) represents the relative consensus degree of expert module i. Let J be the average consensus score of the j-th expert module; Furthermore, the penalty coefficient is dynamically adjusted based on the Top-k weighted cosine overlap of the multi-expert module's predicted distribution. The calculation formula is as follows: ， Among them, P i (x) is the penalty coefficient for expert module i, Conf i Let be the confidence score of expert module i. Let be the sample average entropy, 'a' be the basic bias constant, and 'b' be the amplitude scaling constant.

2. The long-tailed fine-grained image recognition method based on multi-expert dynamic fusion according to claim 1, characterized in that, The expert-shared feature extraction module is composed of the first two layers of a ResNet-50 network.

3. The long-tailed fine-grained image recognition method based on multi-expert dynamic fusion according to claim 1, characterized in that, The head-level fine-grained expert module introduces center loss and Top-k negative sample loss as constraints. The center loss reduces intra-class discrepancies by constraining features of samples of the same class to aggregate towards the class center, and is expressed as: ， Among them, L c The center loss value is denoted by m, where m is the number of sample classes, and x is the center loss value. i Let be the feature vector of the i-th sample. For the yth i The center vector of each category, y i Let i be the category of the i-th sample; The Top-k negative sample loss enhances the inter-class feature discrimination by selecting the top k negative classes with the highest confidence scores from all negative classes. This is expressed as: ， Among them, L k Let s be the loss value for the top-k negative samples, s be the confidence score vector, and l be the true label of the sample. l J represents the confidence score corresponding to the true label of the sample. ' For the set of all negative classes, Let s be the set of the top k negative classes with the highest confidence scores. i Let be the confidence score of the i-th negative category.

4. The long-tailed fine-grained image recognition method based on multi-expert dynamic fusion according to claim 1, characterized in that, The central balancing expert module uses a balancing Softmax loss function for constraints; the balancing Softmax loss function is expressed as: ， Among them, L bal To balance the softmax loss value, D s ={x i ,y i } i=1 To input the training set of the central balance expert module, n s x is the number of samples in the training set. i For the i-th sample, y i Let be the true label of the i-th sample, σ() be the Softmax function, v2() be the unnormalized log probability value output by the middle balance expert module, and π be the prior distribution of the training label frequencies. k is the kth category, π k Let n be the prior probability of the k-th class. k This represents the total number of samples in the k-th category in the training set.

5. The long-tailed fine-grained image recognition method based on multi-expert dynamic fusion according to claim 1, characterized in that, The tail adaptation expert module is constrained by the inverse Softmax loss function; the inverse Softmax loss function is expressed as: ， Among them, L inv D is the inverse softmax loss value. s ={x i ,y i } i=1 For the training set of the input tail adaptation expert module, n s x is the number of samples in the training set. i For the i-th sample, y i Let be the true label of the i-th sample, σ() be the Softmax function, v3() be the unnormalized log probability value output by the tail-adaptation expert module, and λ be a hyperparameter. It is obtained by reversing the order of the training label frequencies π.

6. The long-tailed fine-grained image recognition method based on multi-expert dynamic fusion according to claim 1, characterized in that, The steps for obtaining the basic weights are as follows: The Top-k entropy is used to calculate the prediction uncertainty of each expert module for each sample in the long-tailed fine-grained image set. The calculation formula is as follows: ， in, This represents the uncertainty in the prediction of sample x by expert module e. This represents the probability value obtained by taking the k1 categories with the highest probability values ​​from the prediction probability distribution of sample x by the e-th expert module; Z-score normalization based on global entropy distribution and Sigmoid mapping are used to assign basic weights to each expert module. The Z-score normalization formula for the global entropy distribution is: ， Among them, z e μ is the normalized value of the e-th expert module. H Let σ be the mean of the global entropy. H The standard deviation of global entropy; The Sigmoid mapping formula is: Conf e = α·Sigmoid(γ·z e )+β,Conf e Let α be the confidence score of the e-th expert module, β be the amplitude coefficient, γ be the offset constant, and γ be the scaling factor. The formula for calculating the basic weights is as follows: ， in, Here, represents the base weights for the e-th expert module of sample x, where N is the total number of experts. e (x) represents the confidence score of the e-th expert module for sample x.

7. The long-tailed fine-grained image recognition method based on multi-expert dynamic fusion according to claim 1, characterized in that, The final result obtained is as follows: The final weight of each expert module is calculated by combining the base weight and penalty coefficient of each expert module. The final predicted value is obtained by weighting and summing the final weights of each expert module with the results of each expert module.

8. A long-tailed fine-grained image recognition system based on multi-expert dynamic fusion, characterized in that, The long-tail fine-grained image recognition method based on multi-expert dynamic fusion as described in any one of claims 1-7, the system includes a model building module, a feature extraction module, a head classification module, a middle classification module, a tail classification module, and a fusion classification module; The model building module is used to construct a long-tailed fine-grained image recognition model, including an expert shared feature extraction module, a diversified expert learning module, and a dynamic fusion module; the diversified expert learning module includes a head fine-grained expert module, a middle balancing expert module, and a tail adaptation expert module; the expert shared feature extraction module, the diversified expert learning module, and the dynamic fusion module are connected in sequence; the head fine-grained expert module, the middle balancing expert module, and the tail adaptation expert module are connected in parallel; The feature extraction module is used to acquire a long-tailed fine-grained image set and input it into the expert-shared feature extraction module to extract image features; The head classification module is used to input image features into the head fine-grained expert module to obtain head classification results; The middle classification module is used to input image features into the middle balance expert module to obtain the middle classification results; The tail classification module is used to input image features into the tail adaptation expert module to obtain tail classification results; The fusion classification module is used to input the head classification results, middle classification results, and tail classification results into the dynamic fusion module to calculate the prediction confidence and consensus of each expert module and assign basic weights and penalty coefficients to each expert module. Then, it is fused with the head classification results, middle classification results, and tail classification results to obtain the final classification result.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program, which, when executed by a processor, implements the long-tailed fine-grained image recognition method based on multi-expert dynamic fusion as described in any one of claims 1-7.

Images