Target fine-grained fusion classification method and device
By combining multi-source image feature-level fusion and fine-grained loss functions with depth feature extraction from visible light and SAR images, the problem of unstable target classification in complex environments is solved, achieving higher accuracy and stable target recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 10TH RES INST OF CETC
- Filing Date
- 2026-04-24
- Publication Date
- 2026-05-26
AI Technical Summary
In complex environments, aerial remote sensing images from a single sensor are easily affected by adverse weather conditions and complex geographical environments, resulting in blurred target features, weakened edges, loss of texture information, reduced signal-to-noise ratio, and impact on the accuracy and stability of target classification.
A multi-source image feature-level fusion method is adopted, which combines visible light and SAR images. Depth features are extracted through the ResNet-34 residual network, and a fine-grained loss function and weighted cross-entropy loss are introduced to achieve complementary enhancement and stability improvement of features.
It significantly improves the accuracy and stability of target classification in complex scenarios, overcomes the shortcomings of unstable performance of single sensors, and enhances environmental adaptability and recognition effect.
Smart Images
Figure CN122090185A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and more specifically, to a method and apparatus for fine-grained target fusion classification. Background Technology
[0002] Target classification, as one of the core fundamental research problems in the field of airborne remote sensing, has long been a key technical bottleneck restricting the intelligence level of reconnaissance systems. In recent years, with the rapid development of artificial intelligence, especially deep learning technology, target recognition technology has achieved revolutionary breakthroughs. From early machine learning methods based on handcrafted features to today's end-to-end deep learning models represented by convolutional neural networks, recognition accuracy and efficiency have achieved a qualitative leap and are now widely used. The core task of airborne image target classification is to accurately determine the category attributes of image slices containing potential targets extracted from wide-area, large-scene airborne remote sensing images through specific algorithms or manual methods. However, since airborne remote sensing platforms typically operate in complex and variable tropospheric environments, their imaging process is easily affected by adverse factors such as severe weather conditions and complex geographical environments. This directly leads to problems such as blurred target features, weakened edges, loss of texture information, and reduced signal-to-noise ratio in the acquired single-source remote sensing images. These image degradation phenomena severely weaken the target detection capability and reconnaissance reliability of sensor systems and pose a serious challenge to subsequent target classification models.
[0003] Considering the difficulty of ensuring the stability of detection results by a single sensor in complex environments, multi-source sensor information fusion technology is currently widely adopted. By combining the advantages of different sensors, complementary enhancement of target information can be achieved, thereby improving classification and discrimination capabilities. In practical applications, commonly used data sources mainly include visible light images and SAR (Synthetic Aperture Radar) images. SAR, as an active high-precision microwave imaging radar, has unique advantages: it is sensitive to metallic targets and can penetrate natural covering layers such as clouds, fog, rain, and snow, supporting all-day, all-weather observation. However, due to the characteristics of microwave radiation, SAR images generally have significant speckle noise, which reduces image resolution and affects target observability, posing challenges to classification tasks.
[0004] In contrast, visible light images offer rich visual features such as color, texture, shape, and contrast, making them one of the most commonly used remote sensing data sources. However, their imaging quality is easily affected by weather conditions: rain and fog can degrade images, and imaging performance significantly decreases at night or in low-light environments. Therefore, in complex scenes, relying solely on a single data source often leads to low or unstable classification accuracy for typical targets, necessitating the development of novel target classification methods that can effectively fuse information from multiple sources. Summary of the Invention
[0005] The purpose of this application is to provide a fine-grained target fusion classification method and apparatus to overcome the shortcomings of existing technologies. By fusing features from multiple sources, the method effectively complements the information of visible light and SAR images, overcoming the performance instability of a single sensor in complex environments. By introducing a fine-grained feature learning mechanism and a comprehensive loss function, the method significantly improves the model's ability to distinguish subtle feature differences, thereby achieving higher accuracy and more stable classification of typical targets in complex scenes.
[0006] The objective of this application is achieved through the following technical solution: Firstly, this application proposes a fine-grained target fusion classification method, including: The acquired visible light target slice images and SAR target slice images are input into the visible light image branch network and SAR image branch network, respectively; The depth features of visible light images and SAR images are extracted by feature extraction layers of visible light image branch network and SAR image branch network, respectively. The feature extraction layer adopts the residual network ResNet-34 structure, and the final convolutional layer Conv5_x is modified to associate the number of output channels with the number of target categories M. The number of output channels is η×M, where η is the number of channels allocated to each category, so as to achieve fine-grained feature learning. The depth features of the extracted visible light image and the depth features of the SAR image are concatenated along the feature dimension to generate a fused feature vector. The fused feature vector is input into the classification layer, and the features are mapped to the category space through a fully connected layer to obtain the target fusion classification result; During the model training phase, both the visible light image branch network and the SAR image branch network are constrained by a fine-grained loss function, while the classification layer uses a weighted cross-entropy loss. The total loss function of the model is defined as: , This represents the fine-grained loss of a constrained visible light image network. This represents the fine-grained loss of the constrained SAR image network. This represents the weighted cross-entropy loss, with parameters... and It is a hyperparameter that balances the impact of different losses on the overall loss function.
[0007] In one possible embodiment, the feature extraction layer processes the input data through a convolutional layer, a nonlinear activation function layer, and a batch normalization layer, and performs feature learning through multiple residual units; The final convolutional layer Conv5_x is modified by learning the output features of Conv5_x under the constraints of discriminability loss of channel features and redundancy reduction loss of local features.
[0008] In one possible embodiment, the fine-grained loss function is composed of channel feature distinguishability loss. Reduce loss by local feature redundancy constitute.
[0009] In one possible embodiment, channel feature distinguishability loss The calculation formula is: ,in Represents cross-entropy loss, This represents the ground truth label of the sample. Representation and Category Corresponding features This represents the channel attention, cross-channel max pooling, and global average pooling operations based on the output of the Conv5_x convolutional layer. This indicates the total number of categories.
[0010] In one possible embodiment, the The calculation formula is: ,in and These represent the width and height of the output feature map of the Conv5_x convolutional layer, respectively. It is a scalar value. It is a related to the first A binary weight vector associated with each feature channel.
[0011] In one possible implementation, local feature redundancy reduces loss. The calculation formula is: ,in Indicates the total number of categories. For eigenvalues, This refers to the softmax function.
[0012] In one possible implementation, weighted cross-entropy loss The calculation formula is:
[0013] in Represents cross-entropy loss, Indicates the first in the student network The logit value of each sample. Is with The corresponding weights This represents the ground truth label of the sample. That is the number of samples.
[0014] Secondly, this application proposes a target fine-grained fusion classification device, comprising: The input module is used to input the acquired visible light target slice image and SAR target slice image into the visible light image branch network and SAR image branch network, respectively; The extraction module is used to extract depth features of visible light images and SAR images through the feature extraction layers of the visible light image branch network and the SAR image branch network, respectively. The feature extraction layer adopts the ResNet-34 residual network structure, and the final convolutional layer Conv5_x is modified to associate the number of output channels with the number of target categories M. The number of output channels is η×M, where η is the number of channels allocated to each category, so as to achieve fine-grained feature learning. The connection module is used to connect the depth features of the extracted visible light image with the depth features of the SAR image in the feature dimension to generate a fused feature vector. The generation module is used to input the fused feature vector into the classification layer, and then map the features to the category space through a fully connected layer to obtain the target fused classification result.
[0015] In one possible implementation, the feature extraction layer processes the input data through a convolutional layer, a nonlinear activation function layer, and a batch normalization layer, and performs feature learning through multiple residual units. The final convolutional layer Conv5_x is modified by learning the output features of Conv5_x under the constraints of discriminability loss of channel features and redundancy reduction loss of local features.
[0016] In one possible implementation, the fine-grained loss function is composed of channel feature distinguishability loss. Reduce loss by local feature redundancy constitute.
[0017] The main solution and its various further alternatives described above can be freely combined to form multiple solutions, all of which are solutions that can be adopted and are claimed in this application; furthermore, the (non-conflicting alternatives) can also be freely combined with each other and with other alternatives. Those skilled in the art, after understanding the solution of this application, will realize from the prior art and common general knowledge that there are many combinations, all of which are technical solutions to be protected in this application, and will not be exhaustively listed here.
[0018] This application discloses a fine-grained target fusion classification method and apparatus. Visible light target slice images and SAR target slice images are input into visible light image branch networks and SAR image branch networks, respectively, to extract depth features from the visible light image and the SAR image. The extracted depth features from the visible light image and the SAR image are concatenated along the feature dimension to generate a fused feature vector. This fused feature vector is input into a classification layer, where a fully connected layer maps the features to the class space to obtain the target fusion classification result. An improved ResNet-34 is used as the feature extraction backbone network. Fine-grained feature learning is achieved by adjusting the number of output channels of the final convolutional layer Conv5_x in relation to the number of classes. During training, a multi-task loss mechanism combining a fine-grained loss function and weighted cross-entropy loss is introduced to effectively improve the classification accuracy and stability of the model in complex scenes. Attached Figure Description
[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 The diagram shows a flowchart of a fine-grained fusion classification method for targets proposed in an embodiment of this application.
[0021] Figure 2 The overall framework of the target classification network is shown. Detailed Implementation
[0022] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.
[0023] Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0024] In existing technologies, commonly used data sources for image target classification include visible light images and SAR images. SAR images refer to images captured by synthetic aperture radar (SAR), an active high-precision microwave imaging radar sensor. SAR imaging is increasingly widely used, especially for its strong ability to identify metallic targets. It easily penetrates natural coverings such as light, clouds, fog, rain, and snow, supporting all-weather observation. However, in practical applications, the characteristics of microwave radiation cause SAR images to easily generate a large amount of speckle noise, affecting image resolution and target observability, significantly complicating target classification. Visible light images are currently a commonly used image data source. Visible light imaging can obtain rich information such as color, contrast, shape, and texture, but it is easily affected by weather conditions such as rain and fog. Its effective range is affected by visibility, and at night, visible light images are affected by illumination, resulting in a significant decrease in image quality.
[0025] To address the issue of low or unstable accuracy in classifying typical targets in complex scenarios, this application proposes a fine-grained target fusion classification method and apparatus based on the imaging characteristics of various sensors. This method fuses and combines images from multiple sensors to highlight the description and characterization of the target. Since visible light sensors and SAR sensors have different sensitivities to the spectral and electromagnetic spectrum, employing image feature-level fusion technology is essentially an effective means of achieving wide-band coverage detection using multiple sensors. By comprehensively utilizing the rich target detail information in visible light images and the strong all-weather penetration of SAR images, the environmental adaptability and recognition effect of target classification are improved.
[0026] Please refer to Figure 1 , Figure 1 This paper presents a flowchart illustrating a fine-grained fusion classification method for targets proposed in an embodiment of this application. The method includes: Step S1: Input the acquired visible light target slice image and SAR target slice image into the visible light image branch network and SAR image branch network, respectively.
[0027] The input data consists of paired visible light target slice images and SAR target slice images, which are input into the visible light image network branch and the SAR image network branch, respectively.
[0028] Step S2: Extract depth features of visible light images and SAR images respectively through the feature extraction layers of the visible light image branch network and the SAR image branch network. The feature extraction layer adopts the ResNet-34 residual network structure, and the final convolutional layer Conv5_x is modified to associate the number of output channels with the number of target categories M. The number of output channels is η×M, where η is the number of channels allocated to each category, so as to achieve fine-grained feature learning.
[0029] The feature extraction layer of the dual-branch network extracts highly discriminative fine-grained depth features from visible light and SAR images respectively. The feature extraction layer is based on the mainstream residual network ResNet-34 architecture. This network, by introducing a fast connection mechanism, not only effectively accelerates the training process of ultra-deep neural networks and significantly improves model accuracy, but also fundamentally alleviates the gradient vanishing or gradient exploding problems caused by the increase of network layers.
[0030] Given input data, the ResNet-34 feature extraction layer first feeds the data into convolutional layers, nonlinear activation function layers and batch normalization layers in sequence to extract basic features, and then inputs the processing results into multiple residual units for deep feature transformation.
[0031] The feature extraction layer processes the input data through convolutional layers, non-linear activation function layers, and batch normalization layers, and performs feature learning through multiple residual units. The final convolutional layer Conv5_x is modified by learning the output features of Conv5_x under the constraints of discriminability loss of channel features and redundancy reduction loss of local features.
[0032] To achieve more refined and discriminative feature learning, this application makes key modifications to the final Conv5_x convolutional layer architecture in the ResNet-34 network. In the original ResNet-34 network, the number of convolutional kernels in the Conv5_x layer is fixed at 512, and its feature map channels contain mixed features from all categories. Assuming the total number of target categories is M, and a specific number of channels η is assigned to each category, the number of output channels of the Conv5_x layer is modified to η×M. This design allows the network to learn a specific set of channel features for each category, thereby explicitly modeling unique feature representations corresponding to different categories. To ensure that the network truly focuses on discriminative features related to different categories, the output features of the Conv5_x layer are strictly constrained by a fine-grained loss function during training. This loss function consists of two parts: channel feature discriminability loss (aimed at maximizing the difference between features of different categories) and local feature redundancy reduction loss (aimed at avoiding representational redundancy of feature channels within the same category). Through this joint constraint, the deep features output from the feature extraction layer ultimately possess powerful fine-grained extraction capabilities, enabling them to capture subtle local differences between similar targets.
[0033] Step S3: Connect the depth features of the extracted visible light image with the depth features of the SAR image in the feature dimension to generate a fused feature vector.
[0034] This method effectively fuses two depth features extracted from the visible light image branch network and the SAR image branch network, respectively. A joint representation approach is used in the feature fusion layer to connect the depth features from both branches along the feature dimension, thereby generating a fused feature vector containing multi-source information. Essentially, this method constructs a shared feature subspace. The feature vector fusion layer semantically combines the transformed visible light image feature vector and SAR image feature vector, achieving feature-level fusion. The constructed shared subspace typically possesses semantic invariance, effectively removing noise and interference information from single-modal data while retaining the most crucial common semantic features for target identification.
[0035] Step S4: Input the fused feature vector into the classification layer, and map the features to the category space through the fully connected layer to obtain the target fused classification result.
[0036] The generated fused feature vectors are then converted into specific category determinations to obtain the final target fused classification result. This step is performed by the classification layer, which maps the high-dimensional fused features to the category space. The classification layer is specifically implemented using a fully connected layer. The fully connected layer can integrate all the information from the fused feature vectors. Through the learned weight matrix, it transforms the feature distribution into a space with each target category as its dimension. Finally, the activation function of the output layer (such as Softmax) is used to obtain the probability that the input sample belongs to each category, and the category with the highest probability is the final classification result.
[0037] During the model training phase, both the visible light image branch network and the SAR image branch network are constrained by a fine-grained loss function, while the classification layer uses a weighted cross-entropy loss. The total loss function of the model is defined as: , This represents the fine-grained loss of a constrained visible light image network. This represents the fine-grained loss of the constrained SAR image network. This represents the weighted cross-entropy loss, which balances the impact of potential imbalance in the number of samples from different classes on the cross-entropy loss. The parameters... and It is a hyperparameter that balances the impact of different losses on the overall loss function.
[0038] To ensure robustness and high accuracy during model training, this application imposes multiple constraints on the classification layer during the training phase. First, addressing the common problem of imbalanced sample sizes across different classes in real-world datasets, the classification layer employs a weighted cross-entropy loss function based on sample balancing. Constraints are applied. This loss function effectively mitigates the negative impact of class imbalance on model generalization performance and classification accuracy by assigning higher weights to classes with fewer samples. Secondly, to address the possibility of insufficient training data samples and further improve the model's generalization ability, an automatic augmentation algorithm is introduced during training to augment the input images, thereby enriching the diversity of training samples without actually increasing data acquisition costs. Finally, the loss of the classification layer... Fine-grained loss between the two feature extraction branches , The total loss function that constitutes the model ( The parameters of the entire network are optimized collaboratively by balancing the hyperparameters α and β. In summary, step S4 not only completes the decision mapping from features to categories, but also ensures the high reliability and robustness of the final output of the fusion classification system through a carefully designed loss function and training strategy.
[0039] Fine-grained loss function is derived from channel feature distinguishability loss. Reduce loss by local feature redundancy constitute.
[0040] and This represents fine-grained loss, which consists of two parts. It is the loss of discriminability of channel features. It can use the corresponding features of multiple channels to limit the category-specific description, making the features more discriminative and distinguishing different categories of objects as much as possible. The loss is used to reduce feature redundancy between corresponding channel features of the same category, with the aim of enabling different channel features to reflect different local discriminative features of the image.
[0041] Channel feature distinguishability loss The calculation formula is: ,in Represents cross-entropy loss, This represents the ground truth label of the sample. Representation and Category Corresponding features This represents the channel attention, cross-channel max pooling, and global average pooling operations based on the output of the Conv5_x convolutional layer. This indicates the total number of categories.
[0042] The calculation formula is: ,in and These represent the width and height of the output feature map of the Conv5_x convolutional layer, respectively. It is a scalar value, a feature block. A specific element in the vector represents the activation value at spatial location z in the j-th feature channel of the i-th category. middle, Elements are set to 0, the rest Set to 1. The main diagonal elements of a diagonal matrix are Element.
[0043] Local feature redundancy reduces loss The calculation formula is: ,in Indicates the total number of categories. The feature value is a scalar representing the activation value at the g-th spatial location in the j-th channel of the feature block corresponding to the i-th category. This is the softmax function. It applies softmax operation, cross-channel max pooling, summation, and average calculation. The softmax function... Defined as: .
[0044] Weighted cross-entropy loss The calculation formula is: ,in Represents cross-entropy loss, Indicates the first in the student network The logit value of each sample. Is with The corresponding weights This represents the ground truth label of the sample. That is the number of samples.
[0045] Please refer to Figure 2 , Figure 2 The overall framework of the target classification network is shown, which mainly consists of two parts: the network architecture and the loss function. The network architecture employs a dual-branch design to achieve effective processing and fusion of multi-source information. One branch is dedicated to feature extraction and fine-grained feature learning for targets in visible light images, while the other branch specifically handles target classification in SAR images. These two branches work in parallel, extracting highly discriminative deep features from images of different modalities. Subsequently, the network includes a feature vector fusion layer that fuses the feature vectors extracted from visible light and SAR images to form a joint feature representation containing complementary information. Finally, the fused feature vector is fed into the classification layer, which... Figure 1The gray box represents the network responsible for mapping the fused features to the category space, outputting the final target classification result. In terms of model training, the training process of this network is constrained by a comprehensive loss function, which consists of three parts: a fine-grained loss constraining the visible light image branch. Fine-grained loss of constrained SAR image branches And the weighted cross-entropy loss used in the classification layer. This design ensures that each branch learns discriminative, fine-grained features, while also guaranteeing the classifier's robustness when faced with imbalanced data.
[0046] Compared with the prior art, the embodiments of this application have the following beneficial effects: First, by integrating the rich detail information of visible light images with the all-weather operation advantage of SAR images, the problem of unstable performance of a single sensor in harsh weather or low light conditions is effectively overcome, significantly improving the accuracy of target classification and environmental adaptability.
[0047] Second, by making targeted modifications to the Conv5_x layer of the feature extraction network and introducing a fine-grained loss function for constraint, the model is able to learn more discriminative subtle features, thereby significantly improving its ability to distinguish between similar targets.
[0048] Third, the design of the comprehensive loss function and the application of weighted cross-entropy loss L3 effectively alleviated the negative impact of class imbalance on model performance. Combined with automatic data augmentation technology, this improved the model's generalization ability and stability.
[0049] Fourth, a fusion strategy that connects features along the feature dimension is adopted to construct a shared feature subspace, which effectively complements and integrates feature information from different modalities, providing a more comprehensive and robust feature representation for the final classification decision.
[0050] The following provides a possible implementation of a target fine-grained fusion classification device, which performs the various execution steps and corresponding technical effects of the target fine-grained fusion classification method shown in the above embodiments and possible implementations, including: The input module is used to input the acquired visible light target slice image and SAR target slice image into the visible light image branch network and SAR image branch network, respectively; The extraction module is used to extract depth features of visible light images and SAR images through the feature extraction layers of the visible light image branch network and the SAR image branch network, respectively. The feature extraction layer adopts the ResNet-34 residual network structure, and the final convolutional layer Conv5_x is modified to associate the number of output channels with the number of target categories M. The number of output channels is η×M, where η is the number of channels allocated to each category, so as to achieve fine-grained feature learning. The connection module is used to connect the depth features of the extracted visible light image with the depth features of the SAR image in the feature dimension to generate a fused feature vector. The generation module is used to input the fused feature vector into the classification layer, and then map the features to the category space through a fully connected layer to obtain the target fused classification result.
[0051] In one possible implementation, the feature extraction layer processes the input data through a convolutional layer, a nonlinear activation function layer, and a batch normalization layer, and performs feature learning through multiple residual units. The final convolutional layer Conv5_x is modified by learning the output features of Conv5_x under the constraints of discriminability loss of channel features and redundancy reduction loss of local features.
[0052] In one possible implementation, the fine-grained loss function is composed of channel feature distinguishability loss. Reduce loss by local feature redundancy constitute.
[0053] In summary, this application discloses a fine-grained target fusion classification method and apparatus. Acquired visible light target slice images and SAR target slice images are input into a visible light image branch network and a SAR image branch network, respectively, to extract depth features from the visible light image and the SAR image. The extracted depth features from the visible light image and the SAR image are concatenated along the feature dimension to generate a fused feature vector. This fused feature vector is input into a classification layer, and the features are mapped to the category space through a fully connected layer to obtain the target fusion classification result. An improved ResNet-34 is used as the feature extraction backbone network. Fine-grained feature learning is achieved by adjusting the number of output channels of the final convolutional layer Conv5_x in relation to the number of categories. During training, a multi-task loss mechanism combining a fine-grained loss function and weighted cross-entropy loss is introduced to effectively improve the classification accuracy and stability of the model in complex scenes.
[0054] This preferred embodiment provides a computer device that can implement the steps in any embodiment of the target fine-grained fusion classification method provided in this application. Therefore, it can achieve the beneficial effects of the target fine-grained fusion classification method provided in this application. For details, please refer to the previous embodiments, which will not be repeated here.
[0055] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of this application provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any embodiment of the target fine-grained fusion classification method provided in this application.
[0056] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0057] Since the instructions stored in the storage medium can execute the steps in any of the target fine-grained fusion classification method embodiments provided in this application, the beneficial effects that any of the target fine-grained fusion classification methods provided in this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.
[0058] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A fine-grained fusion classification method for targets, characterized in that, include: The acquired visible light target slice images and SAR target slice images are input into the visible light image branch network and SAR image branch network, respectively; The depth features of visible light images and SAR images are extracted by feature extraction layers of visible light image branch network and SAR image branch network, respectively. The feature extraction layer adopts the residual network ResNet-34 structure, and the final convolutional layer Conv5_x is modified to associate the number of output channels with the number of target categories M. The number of output channels is η×M, where η is the number of channels allocated to each category, so as to achieve fine-grained feature learning. The depth features of the extracted visible light image and the depth features of the SAR image are concatenated along the feature dimension to generate a fused feature vector. The fused feature vector is input into the classification layer, and the features are mapped to the category space through a fully connected layer to obtain the target fusion classification result; During the model training phase, both the visible light image branch network and the SAR image branch network are constrained by a fine-grained loss function, while the classification layer uses a weighted cross-entropy loss. The total loss function of the model is defined as: , This represents the fine-grained loss of a constrained visible light image network. This represents the fine-grained loss of the constrained SAR image network. This represents the weighted cross-entropy loss, with parameters... and It is a hyperparameter that balances the impact of different losses on the overall loss function.
2. The target fine-grained fusion classification method as described in claim 1, characterized in that, The feature extraction layer processes the input data through convolutional layers, non-linear activation function layers, and batch normalization layers, and performs feature learning through multiple residual units. The final convolutional layer Conv5_x is modified by learning the output features of Conv5_x under the constraints of discriminability loss of channel features and redundancy reduction loss of local features.
3. The target fine-grained fusion classification method as described in claim 1, characterized in that, Fine-grained loss function is derived from channel feature distinguishability loss. Reduce loss by local feature redundancy constitute.
4. The target fine-grained fusion classification method as described in claim 3, characterized in that, Channel feature distinguishability loss The calculation formula is: ,in Represents cross-entropy loss, This represents the ground truth label of the sample. Representation and Category Corresponding features This represents the channel attention, cross-channel max pooling, and global average pooling operations based on the output of the Conv5_x convolutional layer. This indicates the total number of categories.
5. The target fine-grained fusion classification method as described in claim 4, characterized in that, The The calculation formula is: ,in and These represent the width and height of the output feature map of the Conv5_x convolutional layer, respectively. It is a scalar value. It is a related to the first A binary weight vector associated with each feature channel.
6. The target fine-grained fusion classification method as described in claim 3, characterized in that, Local feature redundancy reduces loss The calculation formula is: ,in Indicates the total number of categories. For eigenvalues, This refers to the softmax function.
7. The target fine-grained fusion classification method as described in claim 1, characterized in that, Weighted cross-entropy loss The calculation formula is: in Represents cross-entropy loss, Indicates the first in the student network The logit value of each sample. Is with The corresponding weights This represents the ground truth label of the sample. That is the number of samples.
8. A target fine-grained fusion classification device, characterized in that, include: The input module is used to input the acquired visible light target slice image and SAR target slice image into the visible light image branch network and SAR image branch network, respectively; The extraction module is used to extract depth features of visible light images and SAR images through the feature extraction layers of the visible light image branch network and the SAR image branch network, respectively. The feature extraction layer adopts the ResNet-34 residual network structure, and the final convolutional layer Conv5_x is modified to associate the number of output channels with the number of target categories M. The number of output channels is η×M, where η is the number of channels allocated to each category, so as to achieve fine-grained feature learning. The connection module is used to connect the depth features of the extracted visible light image with the depth features of the SAR image in the feature dimension to generate a fused feature vector. The generation module is used to input the fused feature vector into the classification layer, and then map the features to the category space through a fully connected layer to obtain the target fused classification result.
9. The target fine-grained fusion classification device as described in claim 8, characterized in that, The feature extraction layer processes the input data through convolutional layers, non-linear activation function layers, and batch normalization layers, and performs feature learning through multiple residual units. The final convolutional layer Conv5_x is modified by learning the output features of Conv5_x under the constraints of discriminability loss of channel features and redundancy reduction loss of local features.
10. The target fine-grained fusion classification device as described in claim 8, characterized in that, Fine-grained loss function is derived from channel feature distinguishability loss. Reduce loss by local feature redundancy constitute.