Fine-grained image classification method and device based on intercommunication channel loss

Abstract

The invention discloses a fine-grained image classification method and device based on intercommunication channel loss. The method comprises the following steps: calculating the similarity between channels from an initial feature map with a plurality of feature channels in an image to be classified; and forcing each channel to learn features with enough discrimination power for the obtained initial feature map through discriminative components including a channel attention mechanism and cross-channel maximum pooling. And then bilinear convergence is carried out on the discriminative features and the original features, and richer second-order feature information is captured. And a classification layer is constructed after the trunk model of the convolutional neural network, and the cross entropy loss is calculated. And then a joint loss function is constructed through discriminative loss corresponding to the discriminative component, the diversity loss, the bilinear loss and the cross entropy loss. According to the method, the discriminative features of the fine-grained target can be effectively extracted, the classification accuracy is improved, meanwhile, a relatively lightweight network structure is kept, and the requirements of fine-grained image classification application are met.

Description

Technical Field

[0001] This application relates to the field of fine-grained visual classification technology, and more specifically, to a fine-grained image classification method and apparatus based on cross-channel loss. Background Technology

[0002] Currently, efficient classification algorithms in artificial intelligence can perform preliminary screening of large amounts of information collected by drones or satellite systems. This preliminary screening allows for the transmission of relatively effective information, significantly reducing the amount of information transmitted, improving transmission efficiency, reducing the occupation of communication channels, and enhancing information warfare and counter-terrorism capabilities. Simultaneously, it can quickly and effectively identify targets among camouflaged objects with subtle differences. Fine-grained visual classification, as an important research direction in computer vision, aims to address the problem of traditional object classification methods performing poorly when dealing with objects with subtle differences within the same category. With the rapid development of deep learning technology, especially the widespread application of convolutional neural networks (CNNs), fine-grained classification has made significant progress.

[0003] However, fine-grained visual classification still faces challenges, such as high data annotation costs, the need for large amounts of training data, and computational resources. To address these challenges, multi-branch and multi-scale attention learning methods have become a research hotspot. These methods help analysts capture more details from images, identify subtle differences in targets, and provide more accurate intelligence analysis results. This has significant guiding implications for decision-making and project planning. Furthermore, it can achieve better performance in large-scale data processing, helping to efficiently and accurately identify and classify targets. Summary of the Invention

[0004] To address at least one deficiency or improvement requirement of the existing technology, this invention provides a fine-grained image classification method and apparatus based on cross-channel loss. By constructing a unique three-branch network structure and cleverly integrating the attention object module and the attention point localization module, it achieves efficient and accurate processing of fine-grained image classification tasks based on cross-channel loss.

[0005] This invention provides a fine-grained image classification method based on cross-channel loss, comprising: An image to be classified is acquired and input into a pre-trained convolutional neural network backbone model to extract an initial feature map, wherein the initial feature map has multiple feature channels; The initial feature map is grouped by channel according to the number of target categories to obtain multiple feature channel groups, and the target categories and feature channels correspond one-to-one. For each of the feature channel groups, channel-level processing is performed based on a discriminative component, which includes a channel attention mechanism and a cross-channel max pooling operation, to select key feature channels with discriminative power from each feature channel group, generate a discriminative feature map, and calculate cross-channel loss, which includes discriminative loss and polymorphism loss. The discriminative feature map and the initial feature map are subjected to bilinear pooling, and the bilinear loss is calculated. A classification layer is constructed after the convolutional neural network backbone model, and the cross-entropy loss is calculated. Based on the cross-channel loss, the bilinear loss, and the cross-entropy loss corresponding to the discriminative component, a joint loss function is constructed. With the goal of minimizing the joint loss of the joint loss function, all parameters of the classification layer and the convolutional neural network backbone model are updated simultaneously to obtain the updated backbone model and the updated classification layer, so as to achieve fine-grained classification of the image to be classified.

[0006] Furthermore, the channel attention mechanism encourages each feature channel to learn discriminative information by assigning random binary weights to the feature channels in each training iteration and randomly selecting some feature channels to participate in training.

[0007] Furthermore, the cross-channel max pooling operation takes the maximum value of the spatial response of each channel within each feature channel group to highlight the discriminative features in that group and suppress redundant and noise information.

[0008] Furthermore, the diversity loss is achieved by calculating the similarity between different feature channels and minimizing that similarity, so that feature channels of the same category focus on different local regions of the target.

[0009] Furthermore, the bilinear pooling operation captures the second-order interaction features between the discriminative feature map and the initial feature map by calculating the outer product between them.

[0010] Furthermore, the step of constructing a classification layer after the convolutional neural network backbone model and calculating the cross-entropy loss includes: The discriminative feature map and the initial feature map are subjected to bilinear pooling to obtain an enhanced feature representation containing second-order interaction information; A classification layer is constructed after the convolutional neural network backbone model. The enhanced feature representation is input into the classification layer to obtain the class probability distribution. The cross-entropy loss is calculated based on the category probability distribution and the true label.

[0011] Furthermore, the step of simultaneously updating all parameters of the classification layer and the convolutional neural network backbone model with the goal of minimizing the joint loss function, to obtain the updated backbone model and the updated classification layer, in order to achieve fine-grained classification of the image to be classified, includes: Obtain the backbone model parameters of the convolutional neural network backbone model and the classification layer parameters of the classification layer that minimize the joint loss; The convolutional neural network backbone model is updated based on the backbone model parameters to obtain the updated backbone model, and the classification layer is updated based on the classification layer parameters to obtain the updated classification layer. The image to be classified is input into the updated backbone model to obtain the target enhanced feature representation; Based on the updated classification layer, the target enhancement feature representation is mapped to a fine-grained category and output, thereby achieving fine-grained image classification.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the fine-grained image classification method based on cross-channel loss as described above.

[0013] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the fine-grained image classification method based on cross-channel loss as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the fine-grained image classification method based on cross-channel loss as described above.

[0015] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: This application uses a pre-trained convolutional neural network backbone model to calculate inter-channel similarity from initial feature maps with multiple feature channels in an image to be classified. This guides different channels to focus on different discriminative regions of the image, increasing feature diversity and reducing redundant information. The acquired initial feature maps are processed using discriminative components, including channel attention and cross-channel max pooling, to force each channel to learn features with sufficient discriminative power, improving the model's ability to recognize subtle differences. Then, the discriminative features are bilinearly merged with the original features, capturing richer second-order feature information. Bilinear pooling enhances the model's sensitivity to image details, helping to distinguish subtle differences between fine-grained targets. Furthermore, a classification layer is constructed after the convolutional neural network backbone model, and a cross-entropy loss is calculated to ensure basic classification performance. Subsequently, a joint loss function is constructed using the discriminative loss corresponding to the discriminative components, the diversity loss, the bilinear loss, and the cross-entropy loss. Through joint optimization of these three losses, the model can more comprehensively learn the features of fine-grained targets. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of an inter-channel loss provided in an embodiment of this application; Figure 2 A flowchart illustrating an optional fine-grained image classification method based on cross-channel loss provided in this application embodiment; Figure 3 A schematic diagram of an optional channel attention mechanism provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of an optional diversity component provided in an embodiment of this application; Figure 5 A schematic diagram of an optional bilinear neural network architecture provided in an embodiment of this application; Figure 6 A schematic diagram illustrating the principle of an optional bilinear neural network gradient calculation provided in this application embodiment; Figure 7 A schematic diagram of an optional joint loss and network model provided for an embodiment of this application; Figure 8 This is a schematic diagram of an optional electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0019] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0020] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0021] Mutual Channel Loss (MC-Loss) guides the model to learn local discriminative regions by applying a loss on the feature channels. It can better learn the local discriminative regions of the classification target without relying on complex network structures and training mechanisms. Figure 1 The figure illustrates a schematic diagram of an optional cross-channel loss according to an embodiment of this application; as shown... Figure 1 As shown, the cross-channel loss consists of a discriminative component and a diversity component. Its main idea is to group the extracted features at the channel level. The channel attention mechanism of the discriminative component forces each channel to learn features with sufficient discriminative power, and the diversity component enables channels of the same category to focus on the most discriminative region, reducing the redundancy of the discriminative region.

[0022] Figure 2 The illustration shows a flowchart of an optional fine-grained image classification method based on cross-channel loss according to an embodiment of this application, as follows: Figure 2 As shown, the fine-grained image classification method based on cross-channel loss according to embodiments of this application includes: Step S102: Obtain the image to be classified and input the image to be classified into the pre-trained convolutional neural network backbone model to extract the initial feature map.

[0023] The initial feature map has multiple feature channels.

[0024] Step S104: The initial feature map is grouped by channel according to the number of target categories to obtain multiple feature channel groups, and the target categories and feature channels correspond one-to-one; Step S106: For each of the feature channel groups, channel-level processing is performed based on the discriminative component, which includes a channel attention mechanism and a cross-channel max pooling operation, to select key feature channels with discriminative power from each feature channel group, generate a discriminative feature map, and calculate the cross-channel loss, which includes discriminative loss and polymorphism loss. Step S108: Perform bilinear pooling on the discriminative feature map and the initial feature map, and calculate the bilinear loss; Step S110: Construct a classification layer after the convolutional neural network backbone model and calculate the cross-entropy loss; Step S112: Construct a joint loss function based on the cross-channel loss corresponding to the discriminative component, the bilinear loss, and the cross-entropy loss; Step S114: With the goal of minimizing the joint loss of the joint loss function, all parameters of the classification layer and the convolutional neural network backbone model are updated simultaneously to obtain the updated backbone model and the updated classification layer, so as to achieve fine-grained classification of the image to be classified.

[0025] The following example, using fine-grained classification of an aircraft image dataset, illustrates the implementation process of this technical solution.

[0026] For data preparation, the images to be classified were first obtained, for example, using the self-constructed AirF-KM aircraft image dataset, which contains 6749 images of 55 types of aircraft targets. The training and test sets were divided in a 7:3 ratio, and the images to be classified were input into a pre-trained convolutional neural network backbone model to extract initial feature maps. The pre-trained convolutional neural network backbone model can use ResNet50 as the backbone network, incorporating pre-trained parameters from the ImageNet dataset. The initial feature maps have multiple feature channels. This application introduces a cross-channel loss function, comprising a discriminative component and a diversity component. The discriminative component guides each channel to learn discriminative features through channel attention and cross-channel max pooling. The diversity component reduces redundancy by calculating the total similarity of all channels, allowing channels of the same category to focus on different discriminative regions.

[0027] Based on cross-channel loss, a bilinear pooling mechanism is introduced to improve the classification model structure. The grouped attention map calculated by the discriminative component is bilinearly merged with the original feature map to form a bilinear flow. Bilinear pooling enriches the feature representation of the image. A bilinear enhanced cross-channel loss model is proposed, combining bilinear loss with cross-entropy loss and cross-channel loss to form BEMC-Loss. For example, based on ResNet50, MC-Loss and bilinear pooling modules are added to construct a fine-grained image classification model based on cross-channel loss, with a training epoch set to, for example, 60 epochs. During training, cross-entropy loss, cross-channel loss, and bilinear loss are calculated simultaneously and jointly optimized. The model calculates three different loss sub-items separately, and optimizes the network parameters through gradient backpropagation. A pre-trained backbone network is used for feature extraction, and task-specific fine-tuning is performed.

[0028] Optionally, for neural networks trained using the traditional cross-entropy loss objective, the discriminative component is a subset (feature group) of feature channels containing discriminative information about the image. In the cross-channel loss model, each class is represented by a certain number of grouped feature channels; that is, a class is distinguished using the discriminative features possessed by a set of feature channels. The discriminative component Ldis is defined, for example: (1) in, It is the cross-entropy loss between the true class label y and the GAP output. Defined as: (2)

[0029] Wherein, GAP, CCMP, and CWA represent global average pooling, cross-channel max pooling, and channel attention, respectively. A random mask between 0 and 1.

[0030] Global Average Pooling (GAP) is used to calculate the average response for each feature channel, producing a c-dimensional vector where each element corresponds to a single class. A classification layer is built after the convolutional neural network backbone model, and cross-entropy loss is calculated. Finally, the cross-entropy loss function LCE is used to calculate the difference between the true label and the predicted probability given by the softmax function behind the global average pooling operation. In summary, the discriminative component can be understood as each set of features representing a class, and the classifier used for classification is obtained for each class through GAP-CCMP-CWA, then the cross-entropy classification loss is calculated with the true label. The smaller the loss, the more discriminative the extracted feature map, leading to better classification results.

[0031] Cross-Channel Max Pooling (CCMP) is used to compute the maximum response of each feature channel in the initial feature map for each element of a specific class, and then transform it into a size corresponding to that specific class. It is a one-dimensional vector. It should be noted that the cross-channel max pooling here is performed when the feature channels are grouped according to the category, so the cross-channel loss MC-Loss can also be called grouped cross-channel max pooling.

[0032] In contrast, the alternative Cross-Channel Average Pooling (CCAP) tends to average each element within a group, which may suppress the peaks in feature channels, i.e., attention in local regions. Therefore, to preserve these key local attentions and highlight discriminative features in applications, CCMP can yield better training results.

[0033] Channel-Wise Attention (CWA) explicitly encourages each feature channel to contain sufficient discriminative information by assigning random binary weights to channels and randomly selecting a few feature channels from each feature group in each iteration. This process can be visualized as a random channel dropping operation, i.e., channel-level dropout. Figure 3 The figure illustrates a flowchart of an optional channel attention mechanism according to an embodiment of this application, such as... Figure 3 As shown.

[0034] Using channel attention operations to force the network to capture discriminative information equally across all ξ channels corresponding to a specific class allows the model to learn discriminative features more comprehensively during training, thereby effectively improving the model's generalization ability. It's important to note that CWA is only used during training; the entire MC-Loss branch does not exist during testing. Therefore, the distribution of input features received by the classification layer is the same during training and testing.

[0035] Regarding diversity components, while discriminative components can focus on extracting more discriminative regions for feature extraction, they cannot guarantee that the distribution of the extracted discriminative regions is diverse. Figure 4 The figure illustrates a structural schematic diagram of an optional diverse component according to an embodiment of this application, such as... Figure 4 As shown, during training, solely focusing on extracting the most discriminative region leads to feature concentration in a single, prominent area. However, in reality, features of a class of objects are often distributed across different locations. Therefore, different feature channels for a single class should focus on different regions of the image, rather than all channels focusing on the most discriminative region.

[0036] To address this issue, this application introduces a diversity component for calculating the total similarity across all channels. The diversity loss Ldiv can be defined as: (3) in, Defined as: (4) In this approach, supervision is applied directly to the convolutional filters after softmax, and the degree of intersection is measured by summing the spatial dimensions using a CCMP algorithm. In practical applications, the diversity component reduces redundant information by diversifying the feature channels of each group, helping to discover different discriminative regions for each class in the image and thus achieving better classification results.

[0037] Regarding bilinear pooling, aircraft targets often exhibit high variability due to environmental factors (such as lighting, occlusion, and distance), requiring classification models to have high discriminative power and robustness. Therefore, it is necessary to guide model training by highlighting the role of discriminative regions in the background. Figure 5 The figure illustrates a schematic diagram of an optional bilinear neural network architecture according to an embodiment of this application, such as... Figure 5 As shown, inspired by the two-stream hypothesis of visual processing in the human brain—that "in visual cognition, the ventral stream is involved in object recognition and identification, while the dorsal stream is involved in processing the spatial position of objects relative to the observer"—the Bilinear Neural Network (B-CNN) model for image classification can significantly improve classification performance by utilizing feature fusion based on spatial position. It can be widely generalized based on neural networks and fine-tuned for specific domains. The core method in this neural network is bilinear pooling, which obtains richer feature representations from the attention map, thereby improving the model's sensitivity to image details. The Bilinear Neural Network (B-CNN) consists of two branches based on convolutional neural networks, modeling part positions and two-factor interactions; for example, if one network is a part detector, the other is a local feature extractor. The B-CNN architecture can be represented as follows: (5) Where fA and fB represent two feature functions in the neural network, P represents the bilinear pooling function, and C is the classification function of the neural network. The feature function can be represented as a mapping. That is, to take an image I and its location Output a value of size The characteristics of fA and fB are considered. Taking into account the pairwise interaction of feature extractors fA and fB as output conditions, the feature outputs at each position are combined using an outer product. The bilinear feature combination of fA and fB at position l can be expressed as: (6) Next, pooling operations are performed at different positions. The sum of the outer product of the feature maps at each position can be represented as:

[0038] Right now (7) Where the feature sizes extracted by fA and fB are respectively and ,but The size is .Will Remodeling bilinear vector As a feature description of the image, through the normalization step: (8) and (9) Finally, image descriptors are obtained for further classification, which are mapped to the labeled sample space using the classification function C.

[0039] It should be noted that the two neural networks M-Net and D-Net used for extracting features in a bilinear model, if the outputs of the two networks are of size... and Given matrices A and B, their bilinear pooling yields the characteristic matrix A. of The gradients of the classification and normalization layers are direct, while the gradients of layers below the pooling layer can be calculated using the chain rule, such as... Figure 6 As shown.

[0040] According to the chain rule, the gradient can be calculated as follows: (10) In a bilinear convolutional neural network, the two branches can be either symmetric or asymmetric. If the features x and y extracted from the same sample come from two different feature extractors, it's called a multimodal bilinear neural network; if x = y, it's called a homologous bilinear neural network. For symmetric networks, only one feature extractor is actually needed. While this results in slightly lower classification performance compared to asymmetric networks, it significantly reduces computational cost and can still achieve considerable improvements.

[0041] In classification tasks, the input image is first processed through a base network (e.g., VGG16 or ResNet50) to extract feature maps. Let the extracted feature maps be denoted as... Where the height is H, the width is W, and the number of channels is N. Set the value of N to... ,in and These represent the number of categories in the dataset and the number of feature channels used to represent each category, respectively. It is a scalar hyperparameter, typically with a value greater than 2. The nth feature channel of F is represented as... , Then each of F The dimensional channel matrix is ​​reconstructed into a vector W multiplied by H, i.e., WH. The grouping feature channels corresponding to the i-th class are... express, Mathematically, this can be represented as (11) After grouping, the feature map can be represented as follows: (12) Next, the feature map is fed into two streams of the network. One stream, the LCE stream, takes F as input to a fully connected layer with LCE cross-entropy loss, allowing the network to extract information features mainly focused on the global discriminative region. The other stream, the LMC stream, is used to supervise the network to focus on different local discriminative regions. A branch is extended from the LMC stream, bilinearly merging the grouped attention maps calculated by the discriminative component with the original feature map to form a third stream, defined as the bilinear stream, which is used to enrich the feature representation of the image.

[0042] In bilinear flow, the enhanced discriminative feature Fdis, calculated using the Channel Attention (CWA) and Cross-Channel Max Pooling (CCMP) components in the LMC's Ldis component, is bilinearly pooled with the original extracted feature F. The resulting bilinear feature is defined as follows: (13) Wherein, the size of the original feature F is It consists of N channels, and the size of the discriminative feature Fdis is also . Because after grouping by feature channels, they are integrated into c channels. Further, F and Fdis are transformed into... and The two matrices. The final bilinear feature FB has a size of... .

[0043] Next, the bilinear feature is normalized, and the bilinear loss (LB) is obtained by calculating the cross-entropy loss using the bilinear feature and the class label. Figure 7 This is a schematic diagram of an optional joint loss and network model provided in an embodiment of this application. The model utilizes hyperparameters weight µ and... Combining the bilinear loss (LB) with the cross-entropy loss (LCE) and the interchannel loss (LMC) to form the joint loss function—the Bilinear Enhanced Mutual Channel Loss (BEMC-Loss)—is defined as follows: (14) Optionally, the step of constructing a classification layer after the convolutional neural network backbone model and calculating the cross-entropy loss includes: The discriminative feature map and the initial feature map are subjected to bilinear pooling to obtain an enhanced feature representation containing second-order interaction information; A classification layer is constructed after the convolutional neural network backbone model. The enhanced feature representation is input into the classification layer to obtain the class probability distribution. The cross-entropy loss is calculated based on the category probability distribution and the true label.

[0044] Optionally, the step of simultaneously updating all parameters of the classification layer and the convolutional neural network backbone model with the objective of minimizing the joint loss function to obtain the updated backbone model and the updated classification layer, in order to achieve fine-grained classification of the image to be classified, includes: Obtain the backbone model parameters of the convolutional neural network backbone model and the classification layer parameters of the classification layer that minimize the joint loss; The convolutional neural network backbone model is updated based on the backbone model parameters to obtain the updated backbone model, and the classification layer is updated based on the classification layer parameters to obtain the updated classification layer. The image to be classified is input into the updated backbone model to obtain the target enhanced feature representation; Based on the updated classification layer, the target enhancement feature representation is mapped to a fine-grained category and output, thereby achieving fine-grained image classification.

[0045] During model training, the network computes different loss terms for three different objectives and optimizes network parameters through gradient backpropagation. The network model uses a base model as the feature extraction layer and an LCE flow that computes the global loss as the classification layer. The classification layer can use either linear or non-linear connection layers. It's important to note that during training, all three flows in the joint loss are used to compute the loss and participate in model optimization, but during testing, only the LCE flow is used for classification.

[0046] In summary, this application calculates inter-channel similarity from initial feature maps with multiple feature channels in an image to be classified using a pre-trained convolutional neural network backbone model. This guides different channels to focus on different discriminative regions of the image, increasing feature diversity and reducing redundant information. The acquired initial feature maps are processed using discriminative components, including channel attention and cross-channel max pooling, forcing each channel to learn features with sufficient discriminative power, thus improving the model's ability to recognize subtle differences. Then, the discriminative features are bilinearly merged with the original features, capturing richer second-order feature information. Bilinear pooling enhances the model's sensitivity to image details, helping to distinguish subtle differences between fine-grained targets. Furthermore, a classification layer is constructed after the convolutional neural network backbone model, and a cross-entropy loss is calculated to ensure basic classification performance. Subsequently, a joint loss function is constructed using the discriminative loss corresponding to the discriminative components, the diversity loss, the bilinear loss, and the cross-entropy loss. Through joint optimization of these three losses, the model can more comprehensively learn the features of fine-grained targets. This application can effectively extract discriminative features of fine-grained targets, improve classification accuracy, and maintain a relatively lightweight network structure, achieving the dual goals of "lightweight + high precision" and meeting the needs of fine-grained image classification applications.

[0047] Exemplary electronic devices Figure 8 This is a schematic diagram of an optional electronic target device according to an embodiment of this application, such as... Figure 8 As shown, it includes a processor 802, a communication interface 804, a memory 806, and a communication bus 408. The processor 802, communication interface 804, and memory 806 communicate with each other via the communication bus 808. Memory 806 is used to store computer programs; When processor 802 executes a computer program stored in memory 806, it performs the following steps: S1, acquire the image to be classified, and input the image to be classified into a pre-trained convolutional neural network backbone model to extract an initial feature map, wherein the initial feature map has multiple feature channels; S2, the initial feature map is grouped into channels according to the number of target categories to obtain multiple feature channel groups, and the target categories and feature channels correspond one-to-one; S3, for each of the feature channel groups, channel-level processing is performed based on the discriminative component, which includes a channel attention mechanism and a cross-channel max pooling operation, to select key feature channels with discriminative power from each feature channel group, generate a discriminative feature map, and calculate the cross-channel loss, which includes discriminative loss and polymorphism loss. S4, Perform bilinear pooling on the discriminative feature map and the initial feature map, and calculate the bilinear loss; S5, after the convolutional neural network backbone model, construct a classification layer and calculate the cross-entropy loss; S6. Construct a joint loss function based on the cross-channel loss, the bilinear loss, and the cross-entropy loss corresponding to the discriminative component; S7. With the goal of minimizing the joint loss of the joint loss function, all parameters of the classification layer and the convolutional neural network backbone model are updated simultaneously to obtain the updated backbone model and the updated classification layer, so as to achieve fine-grained classification of the image to be classified.

[0048] Optionally, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic target device and other devices.

[0049] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage target device located remotely from the aforementioned processor.

[0050] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0051] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this application may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the fine-grained image classification method based on interchannel loss according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0052] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0053] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the moving object tracking methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.

[0054] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, target device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0055] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0056] The block diagrams of devices, target apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, target apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0057] It should also be noted that in the target apparatus, equipment, and method of this application, each component or step can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0058] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0059] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A fine-grained image classification method based on cross-channel loss, characterized in that, include: An image to be classified is acquired and input into a pre-trained convolutional neural network backbone model to extract an initial feature map, wherein the initial feature map has multiple feature channels; The initial feature map is grouped by channel according to the number of target categories to obtain multiple feature channel groups, and the target categories and feature channels correspond one-to-one. For each of the feature channel groups, channel-level processing is performed based on a discriminative component, which includes a channel attention mechanism and a cross-channel max pooling operation, to select key feature channels with discriminative power from each feature channel group, generate a discriminative feature map, and calculate cross-channel loss, which includes discriminative loss and polymorphism loss. The discriminative feature map and the initial feature map are subjected to bilinear pooling, and the bilinear loss is calculated. A classification layer is constructed after the convolutional neural network backbone model, and the cross-entropy loss is calculated. A joint loss function is constructed based on the cross-channel loss, the bilinear loss, and the cross-entropy loss corresponding to the discriminative component. With the goal of minimizing the joint loss of the joint loss function, all parameters of the classification layer and the convolutional neural network backbone model are updated simultaneously to obtain the updated backbone model and the updated classification layer, so as to achieve fine-grained classification of the image to be classified.

2. The fine-grained image classification method based on cross-channel loss as described in claim 1, characterized in that, The channel attention mechanism assigns random binary weights to feature channels in each training iteration and randomly selects some feature channels to participate in training, thereby encouraging each feature channel to learn discriminative information.

3. The fine-grained image classification method based on cross-channel loss as described in claim 1, characterized in that, The cross-channel max pooling operation takes the maximum value of the spatial response of each channel within each feature channel group to highlight the discriminative features in that group and suppress redundant and noise information.

4. The fine-grained image classification method based on cross-channel loss as described in claim 1, characterized in that, The diversity loss is achieved by calculating the similarity between different feature channels and minimizing that similarity, so that feature channels of the same category focus on different local regions of the target.

5. The fine-grained image classification method based on cross-channel loss as described in claim 1, characterized in that, The bilinear pooling operation captures the second-order interaction features between the discriminative feature map and the initial feature map by calculating the outer product between them.

6. The fine-grained image classification method based on cross-channel loss as described in claim 1, characterized in that, The step of constructing a classification layer after the convolutional neural network backbone model and calculating the cross-entropy loss includes: The discriminative feature map and the initial feature map are subjected to bilinear pooling to obtain an enhanced feature representation containing second-order interaction information; A classification layer is constructed after the convolutional neural network backbone model. The enhanced feature representation is input into the classification layer to obtain the class probability distribution. The cross-entropy loss is calculated based on the category probability distribution and the true label.

7. The fine-grained image classification method based on cross-channel loss as described in claim 1, characterized in that, The step involves simultaneously updating all parameters of the classification layer and the convolutional neural network backbone model, with the goal of minimizing the joint loss function, to obtain the updated backbone model and the updated classification layer, thereby achieving fine-grained classification of the image to be classified. This includes: Obtain the backbone model parameters of the convolutional neural network backbone model and the classification layer parameters of the classification layer that minimize the joint loss; The convolutional neural network backbone model is updated based on the backbone model parameters to obtain the updated backbone model, and the classification layer is updated based on the classification layer parameters to obtain the updated classification layer. The image to be classified is input into the updated backbone model to obtain the target enhanced feature representation; Based on the updated classification layer, the target enhancement feature representation is mapped to a fine-grained category and output, thereby achieving fine-grained image classification.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the fine-grained image classification method based on cross-channel loss as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the fine-grained image classification method based on cross-channel loss as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the fine-grained image classification method based on cross-channel loss as described in any one of claims 1 to 7.

Images