Fine-grained image classification method and system based on ConFeat network

Abstract

The application discloses a fine-grained image classification method and system based on a ConFeat network. The method comprises the following steps: calculating the confusion probability of each training batch sample, generating a confusion table based on the confusion probability, and guiding the generation of the next training batch sample based on the confusion table of the current training batch; inputting the image sample to be classified into a backbone network to obtain original features; constructing a learnable class center matrix, inputting the original features into an adaptive feature space module, measuring the distance between the original features and the corresponding class centers, and updating the class centers through gradient descent; inputting the original features and the corresponding center features of the class centers into a multi-branch feature fusion module to obtain enhanced feature representation, and obtaining a predicted classification result based on the enhanced feature representation. The application can reduce the amount of calculation, continuously optimize the discrimination boundary on the premise of retaining the complete sample semantics, and improve the image classification effect.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a fine-grained image classification method, system, storage medium, and electronic device based on ConFeat network. Background Technology

[0002] Fine-grained visual classification aims to identify subcategories within a basic category. The core challenge lies in the small inter-class distances and significant intra-class variations. To address these challenges, existing techniques mainly fall into two categories: detection-based methods and attention-based methods. Detection-based methods employ a two-stage strategy: explicit object detection or region proposal mechanisms to locate the most discriminative local key regions, followed by fine-grained feature extraction and classification of these regions. Examples include using R-CNN for semantic region localization and feature fusion, designing SPDA-CNN combined with Faster R-CNN for part localization prediction, and recurrent visual attention models such as RA-CNN to recursively learn multi-scale discriminative regions. While these methods effectively focus on distinguishing regions, the additional computational overhead and the complexity of the multi-stage process make it difficult to achieve robust feature learning with limited training data in extremely sparse, fine-grained scenarios.

[0003] In contrast, attention-based methods are end-to-end learning approaches that automatically and implicitly learn and highlight important regions by integrating attention mechanisms. Among these, Vision Transformer (ViT)-based methods are particularly prominent. For example, technologies like TransFG apply ViT to single-stage fine-grained classification, integrating attention weights to locate discriminative regions; or ACC-ViT incorporates innovative components such as data augmentation, decision fusion, and discriminative region selection. ViT-based methods demonstrate superior global dependency modeling capabilities, which are crucial for handling extremely fine-grained classification tasks. Extremely fine-grained visual classification is an extreme development of FGVC, aiming to identify subcategories with minimal visual differences, even requiring expert knowledge to distinguish. Currently, extremely fine-grained classification methods can be mainly divided into data-centric and feature-centric methods. Data-centric approaches typically leverage data augmentation techniques to create diverse samples, including block-level methods that segment, randomly shuffle, and mask images, such as MaskCOV, which drives the network to predict spatial covariance context through covariance, or CLE-ViT, which generates positive samples through self-shuffling and masking; and component-level methods that focus on semantic components, such as SPARE, which forces the network to pay attention to subtle differences that are not masked by randomly erasing learned components, or Mix-ViT, which enhances contextual understanding by predicting token replacements.

[0004] Feature-centric approaches focus on extracting more discriminative features using techniques such as contrastive learning or knowledge distillation. For example, SSFE-Net combines self-supervised learning, knowledge distillation, and class mean increment adaptation to leverage features with smaller inter-class variance; FFVT introduces a mutual attention weight selection mechanism to fuse important tokens to enhance discriminative power; CSDNet utilizes contrastive learning and self-distillation to mine subclass-specific differential knowledge; and FDCL-DA integrates foreground-background feature distillation and contrastive feature learning. However, existing data-centric approaches often rely on destructive data augmentation, which compromises the semantic integrity of images; while feature-centric approaches are limited by the rigidity of static feature representations and, lacking dynamic feedback mechanisms, cannot optimize the feature space in real-time and specifically based on actual classification results during training. Summary of the Invention

[0005] This invention provides a fine-grained image classification method, system, storage medium, and electronic device based on the ConFeat network, which can reduce the amount of computation, continuously optimize the discrimination boundary while preserving the complete semantics of the samples, and improve the image classification effect.

[0006] This invention provides a fine-grained image classification method based on the ConFeat network, comprising: Obtain the training set, which includes the image samples to be classified and their corresponding ground truth labels; The image samples to be classified are input into the ConFeat network for classification, and the predicted classification results are obtained. The total loss function is constructed based on the predicted classification results and the corresponding ground truth labels. The ConFeat network is trained using the total loss function. The ConFeat network includes a dynamic obfuscation-guided sampling module, a backbone network, an adaptive feature space module, and a multi-branch feature fusion module; the processing procedure of the ConFeat network includes: Calculate the confusion probability of each training batch of samples, generate a confusion table based on the confusion probability, and guide the generation of the next training batch of samples based on the confusion table of the current training batch. The image samples to be classified are input into the backbone network to obtain the original features; Construct a learnable class center matrix, input the original features into the adaptive feature space module, measure the distance between the original features and the corresponding class centers, and update the class centers through gradient descent; The original features and the center features corresponding to the class centers are input into the multi-branch feature fusion module to obtain the enhanced feature representation, and the predicted classification result is obtained based on the enhanced feature representation; Fine-grained classification is performed using a pre-trained ConFeat network.

[0007] Furthermore, according to the above-described fine-grained image classification method based on the ConFeat network, the calculation of the confusion probability of each training batch of samples includes: For training batches ,in Indicates batch size. It is the input sample. These are the corresponding real labels, This indicates that it belongs to the current batch. A sample set of classes; Calculate the confusion probability of each training batch of samples:

[0008] in, To obfuscate probabilities, This indicates that the ConFeat network responds to input samples The original output score, Representation Class The probability, This is the activation function.

[0009] Furthermore, according to the above-described fine-grained image classification method based on the ConFeat network, the generation of samples for the next training batch is guided by the confusion table of the current training batch, including: The next training batch of samples is constructed as follows:

[0010] in, This represents a randomly selected target class sample. This represents the confused class samples identified in the confusion table, where , This represents a sample randomly drawn from the remaining dataset. For hyperparameters, .

[0011] Furthermore, according to the above fine-grained image classification method based on the ConFeat network, the class centers are updated by gradient descent as expressed by the following formula:

[0012] in, As the class center, The learning rate is updated around the center. To exclude losses at the center, Feature attraction loss; The center repulsion loss is calculated using the following formula:

[0013]

[0014]

[0015] in, The correlation matrix between class center pairs The probability distribution of class center pairs; Feature attraction loss is calculated using the following formula:

[0016] in, These are the original features extracted from the samples via the backbone network. To be consistent with real labels The corresponding class center.

[0017] Furthermore, according to the above-described fine-grained image classification method based on the ConFeat network, the multi-branch feature fusion module includes a global feature generation branch, a central feature generation branch, and a feature integration branch; the processing procedure of the multi-branch feature fusion module includes: In the global feature generation branch, attention-weighted global features are generated using a learnable class center matrix; In the central feature generation branch, the class-specific center is connected to the original feature and a projection transformation is performed to obtain the central feature; The original features, global features, and central features are input into the feature integration branch for weighted fusion to obtain the enhanced feature representation.

[0018] Furthermore, according to the above-described fine-grained image classification method based on the ConFeat network, the class center feature is calculated using the following formula:

[0019] in, For a class-specific center, This is a real label.

[0020] Furthermore, according to the above-described fine-grained image classification method based on the ConFeat network, the total loss function is:

[0021] in, It is the standard cross-entropy classification loss. It is a center-repulsion loss weight. It is the feature attraction loss weight. To exclude losses at the center, The feature attraction loss is used.

[0022] This invention also provides a fine-grained image classification system based on the ConFeat network, comprising: The acquisition module is used to acquire the training set, including the image samples to be classified and their corresponding ground truth labels; The training module is used to input the image samples to be classified into the ConFeat network for classification, obtain the predicted classification results, construct the total loss function based on the predicted classification results and the corresponding ground truth labels, and train the ConFeat network using the total loss function. The ConFeat network includes a dynamic obfuscation-guided sampling module, a backbone network, an adaptive feature space module, and a multi-branch feature fusion module; the processing procedure of the ConFeat network includes: Calculate the confusion probability of each training batch of samples, generate a confusion table based on the confusion probability, and guide the generation of the next training batch of samples based on the confusion table of the current training batch. The image samples to be classified are input into the backbone network to obtain the original features; Construct a learnable class center matrix, input the original features into the adaptive feature space module, measure the distance between the original features and the corresponding class centers, and update the class centers through gradient descent; The original features and the center features corresponding to the class centers are input into the multi-branch feature fusion module to obtain the enhanced feature representation, and the predicted classification result is obtained based on the enhanced feature representation; The application module is used for fine-grained classification using the trained ConFeat network.

[0023] The present invention also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the above-described fine-grained image classification methods based on the ConFeat network.

[0024] The present invention also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in the steps of the fine-grained image classification method based on the ConFeat network described in any of the preceding claims.

[0025] This invention provides a fine-grained image classification method, system, storage medium, and electronic device based on the ConFeat network. Targeting computer vision classification tasks, this invention proposes a simple yet effective method—obfuscation sampling—which identifies and locates the most easily confused class pairs by modeling the relationships between categories. It can be easily inserted into many neural networks without changing the neural network structure, solving the problem of misclassification and model performance degradation caused by obfuscation. Furthermore, to process the extracted raw features, an adaptive feature space module and a multi-branch feature fusion module are proposed. These two modules coordinate with each other to generate enhanced features and produce prediction results. A new network, ConFeat, is constructed based on these three modules. Finally, training verification demonstrates its significant performance improvement on small-sample, extremely fine-grained image classification tasks. Attached Figure Description

[0026] The technical solution and other beneficial effects of the present invention will become apparent from the following detailed description of specific embodiments of the invention, in conjunction with the accompanying drawings.

[0027] Figure 1 A flowchart illustrating a fine-grained image classification method based on the ConFeat network provided in an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram illustrating the sample differences between fine-grained and ultra-fine-grained image classification provided in an embodiment of the present invention.

[0029] Figure 3 The flowchart of the main dynamic obfuscation guidance sampling module provided in the embodiment of the present invention is shown.

[0030] Figure 4 A flowchart illustrating the processing of the intelligent adaptive feature space module provided in this embodiment of the invention.

[0031] Figure 5 The flowchart illustrates the processing of the multi-branch feature fusion module provided in this embodiment of the invention.

[0032] Figure 6 This is a schematic diagram of the structure of a fine-grained image classification system based on the ConFeat network provided in an embodiment of the present invention.

[0033] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] This invention provides a fine-grained image classification method, system, storage medium, and electronic device based on the ConFeat network. The fine-grained image classification system based on the ConFeat network provided in this invention can be integrated into an electronic device, such as a terminal or server. The terminal can include tablet computers, laptops, personal computers (PCs), miniature processing boxes, or other devices.

[0036] Please see Figure 1 , Figure 1 The flowchart illustrates a fine-grained image classification method based on the ConFeat network provided in this embodiment of the invention. This method, applied in electronic devices, includes the following steps: S1, Obtain the training set, which includes the image samples to be classified and their corresponding ground truth labels.

[0037] Figure 2 This is a schematic diagram illustrating the sample differences between fine-grained and ultra-fine-grained image classification provided in an embodiment of the present invention. The three columns from left to right represent samples of coarse-grained image classification, samples of fine-grained image classification, and samples of ultra-fine-grained image classification, respectively.

[0038] S2, input the image sample to be classified into the ConFeat network for classification, obtain the predicted classification result, construct the total loss function based on the predicted classification result and the corresponding ground truth label, and train the ConFeat network through the total loss function.

[0039] like Figure 1 As shown, the ConFeat (Confusion Features) network includes a dynamic confusion-guided sampling module, a backbone network, an adaptive feature space module, and a multi-branch feature fusion module; the processing steps of the ConFeat network include: S21, calculate the confusion probability of each training batch of samples, generate a confusion table based on the confusion probability, and guide the generation of the next training batch of samples based on the confusion table of the current training batch.

[0040] Figure 3 This is a flowchart of the main dynamic obfuscation guidance sampling module provided in an embodiment of the present invention. Specifically, it is set as follows: Let N represent the set of all N classes. The confusion probability is... Defined as a comprehensive metric that quantifies misclassification among all class pairs, for a given training batch. ,in Indicates the batch size, where It is the input sample. These are the corresponding real labels, This indicates that it belongs to the current batch. A sample set of classes.

[0041] The confusion probability of each training batch of samples is calculated as follows:

[0042] in, To obfuscate probabilities, This indicates that the ConFeat network responds to input samples The original output score, Representation Class The probability is updated for each batch. To adapt to the limited number of samples for each class in an extremely fine-grained dataset, ensuring timely capture of evolving inter-class relationships.

[0043] For each class Extract the top-k most easily confused classes. ,in k = 3, It stores the three most easily confused ones. kind. For the next training batch, the primary target class is first randomly selected. Then based on the current obfuscation table Use obfuscated guided sampling to fill the remaining capacity.

[0044] Next batch Strategically constructed as:

[0045] in, This represents a randomly selected target class sample. Indicates in the confusion table The confused samples identified in the middle, among which , Represents a sample randomly drawn from the remaining dataset, hyperparameters Control the aliasing sampling ratio.

[0046] S22, input the image samples to be classified into the backbone network to obtain the original features.

[0047] Specifically, the backbone network performs multi-scale feature extraction on the image samples to be classified in order to capture fine-grained details from different spatial regions. In specific embodiments, the backbone network can be mainstream visual feature extraction networks such as Swin Transformer and ResNet-50.

[0048] S23. Construct a learnable class center matrix, input the original features into the adaptive feature space module, measure the distance between the original features and the corresponding class centers, and update the class centers through gradient descent.

[0049] Figure 4 This is a flowchart of the intelligent adaptive feature space module provided in this embodiment of the invention, which constructs a learnable class center matrix. ,in N The number of classes. D The dimension of the original features extracted from the corresponding backbone network. For class The class centers are initialized using a small-variance Gaussian distribution. Small variance It provides numerically stable initialization and provides an equal starting point for all classes to learn different representations.

[0050] The adaptive feature space module receives the raw features extracted by the backbone network. And treat it as a spatial constraint object; in this module, the original features... With the corresponding class center Perform distance measurement and update class centers using gradient descent:

[0051] in The learning rate is updated around the center. To exclude losses at the center, The feature attraction loss is used.

[0052] The center repulsion loss aims to maximize the separability between different class centers in the feature space. The calculation process of the center repulsion loss is as follows: First, calculate the correlation matrix among all class center pairs:

[0053] To better highlight the significant relationships between class centers, the softmax activation function is applied to normalize these correlations and transform them into meaningful probability distributions.

[0054] The repulsion loss is calculated as follows:

[0055] in, The correlation matrix between class center pairs Let be the probability distribution of class center pairs, where the numerator represents the average correlation between different class centers (which we want to minimize), and the denominator represents the average autocorrelation of each class center (which we want to maximize).

[0056] The feature attraction loss aims to ensure that sample features are tightly clustered around their corresponding class centers. The calculation process for the feature attraction loss is as follows: For training batches The attraction loss is defined as:

[0057] in, These are the original features extracted from the samples via the backbone network. To be consistent with real labels The corresponding class centers. This mean squared error formula provides stable optimization dynamics with smooth gradients, achieving a stronger distance-scale attraction as samples move further away from their class centers.

[0058] S24. Input the original features and the center features corresponding to the class centers into the multi-branch feature fusion module to obtain the enhanced feature representation, and obtain the predicted classification result based on the enhanced feature representation.

[0059] Figure 5 The flowchart below illustrates the processing of a multi-branch feature fusion module provided in this embodiment of the invention. The multi-branch feature fusion module includes a global feature generation branch, a central feature generation branch, and a feature integration branch. The processing steps of the multi-branch feature fusion module include: S241, in the global feature generation branch, attention-weighted global features are generated using a learnable class center matrix.

[0060] Utilizing a learnable class center matrix By capturing comprehensive class-specific information, the attention-weighted global features obtained are:

[0061] in This indicates the quantization of each class center relative to the original features. Attention weights for relevance, Attention weights encapsulate comprehensive class-centric information.

[0062] S242, in the central feature generation branch, the class-specific center is connected to the original feature and a projection transformation is performed to obtain the central feature.

[0063] Class-specific center Different computations were performed during the training and inference phases (i.e., testing and application after model training) to optimize learning efficiency and classification performance:

[0064] During the training phase, the real labels are used. Corresponding class center During the reasoning process, attention weights are used to determine the weights. The average value of the top three most relevant class centers is taken, where the reasoning process refers to the testing and application process after the model training is completed.

[0065] The central features are constructed by concatenating class-specific centers with the original input features and applying a projection transformation, as expressed by the following formula:

[0066] in, This indicates a splicing operation. This represents the projection function.

[0067] Through extensive experimentation, a simple two-layer projection with ReLU activation was used to connect 2 D 3D feature transformation back to original D 3D space can achieve excellent performance and provides better results than complex dimensionality reduction schemes.

[0068] S243 inputs the original features, global features, and central features into the feature integration branch for weighted fusion to obtain the enhanced feature representation.

[0069] Final Enhanced Feature Representation It is calculated as a learnable weighted combination of all three branches:

[0070] in, Indicates original features, Represents global features. Indicates central features, These are learnable parameters.

[0071] Furthermore, the enhanced feature representation is input into a fully connected layer, a multilayer perceptron, or another classifier to obtain the predicted classification result, i.e., the predicted probability of each category.

[0072] A joint loss strategy is employed, using the ConFeat network for end-to-end joint training of the training data. During training, the adaptive feature space module updates its parameters synchronously with the backbone network and other modules, eliminating the need for staged pre-training. The overall training objective combines the standard classification loss with the adaptive feature space optimization loss to achieve comprehensive discriminative learning. The total loss can be expressed as:

[0073] in, It is the standard cross-entropy classification loss. It is a center-repulsion loss weight. It is the feature attraction loss weight.

[0074] The specific training employed a dual optimization strategy, using the AdamW optimization algorithm with different learning rates: a learning rate of 6e-5 for model parameters and 5e-5 for class centers, with adjustment rates of 1e-4 and 7e-5 for the SoyGene dataset, respectively; training was conducted in 16 batches (12 batches for SoyGene) for 200 epochs; automatic mixed precision and gradient clipping were used to train stability; and Top-1 accuracy was used as the evaluation metric for all experiments.

[0075] S3 performs fine-grained classification using a pre-trained ConFeat network.

[0076] To verify and illustrate the technical effects of the method of the present invention, since this method is the first to propose a confusion-guided sampling module, this embodiment will be compared with other state-of-the-art methods in a classification task.

[0077] The performance of the ConFeat network was evaluated on eight benchmark datasets, including five ultra-fine-grained image datasets and three fine-grained image datasets. The ultra-fine-grained datasets included Cotton80, SoyLocal, SoyGene, SoyAgeing, and SoyGlobal; the fine-grained datasets included CUB-200-2011 for bird species classification, the Apple Leaf Disease AFD dataset for apple leaf disease detection, and the IP102 dataset for pest identification. ConFeat was comprehensively evaluated by comparing it with 19 state-of-the-art classification methods on ultra-fine-grained image datasets. These baselines were divided into four groups. The first group included three CNN-based methods: AlexNet, VGG-16, and ResNet-50. The second group focused on self-supervised learning methods, including SimCLR, MoCo v2, and BYOL. The third group encompassed weakly supervised and fine-grained classification methods, including Cutout, ADL, Cutmix, DCL, and MaskCOV. The fourth group deals with Visual Transformers-based methods, including ViT, DeiT, TransFG, SIM-Trans, Hybrid ViT, Mix-ViT, CLE-ViT, and CSDNet. Here, ADL uses two different dropout rate configurations (0.25 and 0.5) for a thorough comparison. Furthermore, Cutout experiments were conducted in conjunction with the size of the shielding region under two optimal parameter settings, specifically shielding region lengths of 8 and 16. Meanwhile, self-supervised methods are evaluated in two cases: linear evaluation (only the classifier is optimized) and fine-tuning evaluation (all model parameters are fine-tuned), where linear evaluation is denoted by L and fine-tuning evaluation by FT.

[0078] During the training phase, a single NVIDIA A5000 GPU was used for training. During the validation phase, the validation set was pruned, and the top-1 classification accuracy was recorded.

[0079] The statistical results of the comparative experiments of the ConFeat network are shown in Tables 1, 2, 3, and 4. Table 1 presents the experimental evaluation of ConFeat and state-of-the-art methods on extremely fine-grained benchmark datasets, demonstrating the superior classification ability of the method provided in this invention. ConFeat, with its Swin-B backbone network, exhibits significant advantages on all test datasets, outperforming dedicated extremely fine-grained classification methods. On extremely limited sample datasets, such as Cotton80 and SoyLocal, where each subclass contains an average of only 3 images, ConFeat achieved remarkable accuracies of 72.08% and 63.17%, respectively, significantly outperforming competing extremely fine-grained classification methods. Even on datasets with a large number of classes, ConFeat maintains this advantage; for example, on Soy-Gene with over 1000 subclasses, ConFeat achieves an accuracy of 87.67%. Table 2 presents the experimental evaluation of ConFeat and state-of-the-art methods on the soybean aging dataset. Compared to other competing methods, ConFeat achieved superior performance on all five subsets, with improvements ranging from 1.72% to 2.47% compared to the second-best method. Furthermore, Table 3 shows that the method provided in this invention achieves good performance on the CUB-200-2011 fine-grained bird classification dataset. Most notably, Table 4 shows the comparison results on the IP102 dataset, which is characterized by its large data size (containing 75,222 images across 102 categories) and imbalanced data distribution. Despite this, the ConFeat network still achieved the highest accuracy. The consistency across these different datasets indicates that the method provided in this invention remains advantageous and generalizable across various classification scenarios.

[0080] Table 1. Experimental Results of Model Comparison - Fine-grained Dataset

[0081] Table 2. Experimental Results of Model Comparison - Soybean Aging Dataset

[0082] Table 3. Experimental Results of Model Comparison - CUB-200-2011 Dataset

[0083] Table 4. Experimental Results of Model Comparison - IP102 Dataset

[0084] Based on the method described in the above embodiments, this embodiment will further describe it from the perspective of a fine-grained image classification system based on the ConFeat network. The fine-grained image classification system based on the ConFeat network can be implemented as an independent entity or integrated into an electronic device. The electronic device can be a terminal, server, or other device. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessor box, or other devices.

[0085] Please see Figure 6 , Figure 6 This invention specifically describes a fine-grained image classification system based on the ConFeat network, which is applied in electronic devices. The fine-grained image classification system based on the ConFeat network may include: The acquisition module is used to acquire the training set, including the image samples to be classified and their corresponding ground truth labels; The training module is used to input the image samples to be classified into the ConFeat network for classification, obtain the predicted classification results, construct the total loss function based on the predicted classification results and the corresponding ground truth labels, and train the ConFeat network using the total loss function. The ConFeat network includes a dynamic obfuscation-guided sampling module, a backbone network, an adaptive feature space module, and a multi-branch feature fusion module; the processing procedure of the ConFeat network includes: Calculate the confusion probability of each training batch of samples, generate a confusion table based on the confusion probability, and guide the generation of the next training batch of samples based on the confusion table of the current training batch. The image samples to be classified are input into the backbone network to obtain the original features; Construct a learnable class center matrix, input the original features into the adaptive feature space module, measure the distance between the original features and the corresponding class centers, and update the class centers through gradient descent; The original features and the center features corresponding to the class centers are input into the multi-branch feature fusion module to obtain the enhanced feature representation, and the predicted classification result is obtained based on the enhanced feature representation; The application module is used for fine-grained classification using the trained ConFeat network.

[0086] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0087] In addition, this embodiment of the invention also provides an electronic device, which may be a computer, tablet computer, or other similar device. This electronic device can implement the steps of any embodiment of the fine-grained image classification method based on the ConFeat network provided in this embodiment of the invention. Therefore, it can achieve the beneficial effects achievable by any fine-grained image classification method based on the ConFeat network provided in this embodiment of the invention, as detailed in the preceding embodiments, and will not be repeated here.

[0088] Figure 7 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the fine-grained image classification method based on the ConFeat network provided in the above embodiments. The electronic device 500 can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0089] The memory 520 can be used to store software programs and modules, such as the program instructions / modules corresponding to those in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520. The memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 520 may further include memory remotely located relative to the processor 580, and these remote memories can be connected to the electronic device 500 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0090] The input unit 530 can be used to receive input numeric or character information, and to generate a keyboard and mouse related to user settings and function control. Display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof. Display unit 540 may include display panel 541, which may optionally be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms.

[0091] Electronic device 500, through transmission module 570 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 570 is shown in the figure, it is understood that it is not an essential component of electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0092] The processor 580 is the control center of the electronic device 500. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 500 by running or executing software programs and / or modules stored in the memory 520, and by calling data stored in the memory 520, thereby providing overall monitoring of the electronic device. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 580.

[0093] Electronic device 500 also includes a power supply 590 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to processor 580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 590 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0094] Although not shown, the electronic device 500 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations: Obtain the training set, which includes the image samples to be classified and their corresponding ground truth labels; The image samples to be classified are input into the ConFeat network for classification, and the predicted classification results are obtained. The total loss function is constructed based on the predicted classification results and the corresponding ground truth labels. The ConFeat network is trained using the total loss function. The ConFeat network includes a dynamic obfuscation-guided sampling module, a backbone network, an adaptive feature space module, and a multi-branch feature fusion module; the processing procedure of the ConFeat network includes: Calculate the confusion probability of each training batch of samples, generate a confusion table based on the confusion probability, and guide the generation of the next training batch of samples based on the confusion table of the current training batch. The image samples to be classified are input into the backbone network to obtain the original features; Construct a learnable class center matrix, input the original features into the adaptive feature space module, measure the distance between the original features and the corresponding class centers, and update the class centers through gradient descent; The original features and the center features corresponding to the class centers are input into the multi-branch feature fusion module to obtain the enhanced feature representation, and the predicted classification result is obtained based on the enhanced feature representation; Fine-grained classification is performed using a pre-trained ConFeat network.

[0095] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0096] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any embodiment of the fine-grained image classification method based on the ConFeat network provided by the present invention.

[0097] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0098] Since the instructions stored in the storage medium can execute the steps in any embodiment of the fine-grained image classification method based on the ConFeat network provided in the embodiments of the present invention, the beneficial effects that any fine-grained image classification method based on the ConFeat network provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0099] The foregoing has provided a detailed description of a fine-grained image classification method, system, storage medium, and electronic device based on the ConFeat network provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fine-grained image classification method based on ConFeat network, characterized in that, The method includes: Obtain the training set, which includes the image samples to be classified and their corresponding ground truth labels; The image samples to be classified are input into the ConFeat network for classification, and the predicted classification results are obtained. The total loss function is constructed based on the predicted classification results and the corresponding ground truth labels. The ConFeat network is trained using the total loss function. The ConFeat network includes a dynamic obfuscation-guided sampling module, a backbone network, an adaptive feature space module, and a multi-branch feature fusion module; the processing procedure of the ConFeat network includes: Calculate the confusion probability of each training batch of samples, generate a confusion table based on the confusion probability, and guide the generation of the next training batch of samples based on the confusion table of the current training batch. The image samples to be classified are input into the backbone network to obtain the original features; Construct a learnable class center matrix, input the original features into the adaptive feature space module, measure the distance between the original features and the corresponding class centers, and update the class centers through gradient descent; The original features and the center features corresponding to the class centers are input into the multi-branch feature fusion module to obtain the enhanced feature representation, and the predicted classification result is obtained based on the enhanced feature representation; Fine-grained classification is performed using a pre-trained ConFeat network.

2. The fine-grained image classification method based on ConFeat network according to claim 1, characterized in that, Calculate the confusion probability of each training batch of samples, including: For training batches ,in Indicates batch size. It is the input sample. These are the corresponding real labels, This indicates that it belongs to the current batch. A sample set of classes; Calculate the confusion probability of each training batch of samples: in, To obfuscate probabilities, This indicates that the ConFeat network responds to input samples The original output score, Representation Class The probability, This is the activation function.

3. The fine-grained image classification method based on ConFeat network according to claim 2, characterized in that, The generation of samples for the next training batch is guided by the obfuscation table of the current training batch, including: The next training batch of samples is constructed as follows: in, This represents a randomly selected target class sample. This represents the confused class samples identified in the confusion table, where , This represents a sample randomly drawn from the remaining dataset. For hyperparameters, .

4. The fine-grained image classification method based on ConFeat network according to claim 1, characterized in that, The gradient descent update of class centers is expressed by the following formula: in, As the class center, The learning rate is updated around the center. To exclude losses at the center, Feature attraction loss; The center repulsion loss is calculated using the following formula: in, The correlation matrix between class center pairs The probability distribution of class center pairs; Feature attraction loss is calculated using the following formula: in, These are the original features extracted from the samples via the backbone network. To be consistent with real labels The corresponding class center.

5. The fine-grained image classification method based on ConFeat network according to claim 1, characterized in that, The multi-branch feature fusion module includes a global feature generation branch, a central feature generation branch, and a feature integration branch; The processing procedure of the multi-branch feature fusion module includes: In the global feature generation branch, attention-weighted global features are generated using a learnable class center matrix; In the central feature generation branch, the class-specific center is connected to the original feature and a projection transformation is performed to obtain the central feature; The original features, global features, and central features are input into the feature integration branch for weighted fusion to obtain the enhanced feature representation.

6. The fine-grained image classification method based on ConFeat network according to claim 5, characterized in that, The class center feature is calculated using the following formula: in, For a class-specific center, This is a real label.

7. The fine-grained image classification method based on ConFeat network according to claim 1, characterized in that, The total loss function is: in, It is the standard cross-entropy classification loss. It is a center-repulsion loss weight. It is the feature attraction loss weight. To exclude losses at the center, The feature attraction loss is used.

8. A fine-grained image classification system based on ConFeat network, characterized in that, include: The acquisition module is used to acquire the training set, including the image samples to be classified and their corresponding ground truth labels; The training module is used to input the image samples to be classified into the ConFeat network for classification, obtain the predicted classification results, construct the total loss function based on the predicted classification results and the corresponding ground truth labels, and train the ConFeat network using the total loss function. The ConFeat network includes a dynamic confusion-guided sampling module, a backbone network, an adaptive feature space module, and a multi-branch feature fusion module. The processing steps of the ConFeat network include: Calculate the confusion probability of each training batch of samples, generate a confusion table based on the confusion probability, and guide the generation of the next training batch of samples based on the confusion table of the current training batch. The image samples to be classified are input into the backbone network to obtain the original features; Construct a learnable class center matrix, input the original features into the adaptive feature space module, measure the distance between the original features and the corresponding class centers, and update the class centers through gradient descent; The original features and the center features corresponding to the class centers are input into the multi-branch feature fusion module to obtain the enhanced feature representation, and the predicted classification result is obtained based on the enhanced feature representation; The application module is used for fine-grained classification using the trained ConFeat network.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the fine-grained image classification method based on the ConFeat network as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The method includes a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used to execute the steps of the fine-grained image classification method based on the ConFeat network as described in any one of claims 1 to 7.

Images