Small sample target recognition method and device based on adaptive relationship distillation

Abstract

The application discloses a small sample target identification method and device based on adaptive relationship distillation, and belongs to the field of deep learning. The method comprises the following steps: acquiring a query image and a support image set; inputting the query image and the support image set into an identification model, so as to identify and infer a small sample target of the query image based on the support image set by using the identification model; wherein the identification model comprises a backbone network used for extracting image features of the query image and the support image set respectively, an adaptive relationship distillation module used for deep interaction and fusion of the image features extracted by the backbone network, and an identification module; the adaptive relationship distillation module comprises a channel attention module and a spatial attention module. The scheme can retain local details of the support image set, and through adaptive feature fusion, the support features and the query features can be adaptively fused, so as to ensure that the rich local detail information in the support image can guide the query image to realize high-precision identification.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method and apparatus for small-sample target recognition based on adaptive relation distillation. Background Technology

[0002] With the development of deep learning technology, object recognition algorithms have achieved remarkable results when there is sufficient data. However, in practical applications, there is often a situation where training samples for certain specific categories are extremely scarce. To solve this problem, few-sample object recognition has emerged.

[0003] Current few-shot object recognition methods typically involve two inputs: a query image and a support set. The model needs to refer to a small number of labeled samples in the support set to identify similar objects in the query image. In existing solutions, the relationship between support features and query features is often handled by performing global pooling on the support features, compressing the features of the support image into a single global vector to adjust the feature response of the query branch. While this simplifies computation, it easily loses rich local details in the support image.

[0004] Therefore, when there are significant differences in appearance between objects in the query image and those in the supporting images, or when objects are partially occluded, the model often struggles to extract sufficient discriminative features without guidance from local detail information. Furthermore, traditional feature fusion strategies often employ fixed or simple weighting methods, lacking the ability to automatically adjust the fusion strategy based on image content.

[0005] Therefore, there is an urgent need to provide a small-sample target recognition method and apparatus based on adaptive relation distillation. Summary of the Invention

[0006] To address the problem that traditional techniques often lose rich local details in supporting images, leading to low recognition accuracy, this invention provides a small-sample target recognition method and apparatus based on adaptive relational distillation.

[0007] On the one hand, a few-sample target recognition method based on adaptive relation distillation is provided, the method comprising: Obtain the query image and a support image set; the support image set contains several types of small sample targets; The query image and the support image set are input into a recognition model to identify and infer small sample targets in the query image based on the support image set. The recognition model includes a backbone network for extracting image features from the query image and the support image set, an adaptive relation distillation module for deep interaction and fusion of the image features extracted by the backbone network, and a recognition module. The adaptive relation distillation module contains a channel attention module and a spatial attention module. The target category and bounding box coordinates of the query image are obtained.

[0008] On the other hand, a small-sample target recognition device based on adaptive relation distillation, based on the steps described in any method embodiment of the specification, is provided. The device includes: The acquisition unit is used to acquire the query image and the support image set; the support image set contains several types of small sample targets. A recognition unit is configured to input the query image and the support image set into a recognition model, so as to use the recognition model to recognize and infer small sample targets in the query image based on the support image set; wherein, the recognition model includes a backbone network for extracting image features from the query image and the support image set respectively, an adaptive relation distillation module for deep interaction and fusion of the image features extracted by the backbone network, and a recognition module; the adaptive relation distillation module includes a channel attention module and a spatial attention module; The output unit is used to obtain the target category and target bounding box coordinates of the query image.

[0009] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for executing the computer program stored in the memory to implement the steps of the method described above.

[0010] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the method described above.

[0011] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0012] The technical solution provided by this invention can bring at least the following beneficial effects: By preserving the spatial structure of supporting features and introducing spatial attention, local details within the supporting image set can be retained, which helps the model perform feature matching when faced with changes in object appearance or occlusion. By introducing a hybrid attention mechanism, the network can automatically learn feature weights, achieving adaptive feature utilization and improving the efficiency of feature fusion. Therefore, this scheme can preserve the local details of the supporting image set, and through adaptive feature fusion, the supporting and query features can be adaptively fused, ensuring that the rich local details in the supporting images guide the query images to achieve high-precision recognition. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a small-sample target recognition method based on adaptive relation distillation provided by an embodiment of the present invention; Figure 2 This is a structural diagram of a small-sample target recognition device based on adaptive relation distillation according to an embodiment of the present invention; Figure 3 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0016] The specific implementation of the above concept is described below.

[0017] Please refer to Figure 1 This invention provides a small-sample target recognition method based on adaptive relation distillation, the method comprising: Step 100: Obtain the query image and the supporting image set; the supporting image set contains several types of small sample targets; Step 102: Input the query image and the support image set into the recognition model to use the recognition model to identify and infer the small sample target of the query image based on the support image set; wherein, the recognition model includes a backbone network for extracting image features of the query image and the support image set respectively, an adaptive relation distillation module for deep interaction and fusion of the image features extracted by the backbone network, and a recognition module; the adaptive relation distillation module contains a channel attention module and a spatial attention module; Step 104: Obtain the target category and bounding box coordinates of the query image.

[0018] In this embodiment of the invention, by preserving the spatial structure of supporting features and introducing spatial attention, local detail information within the supporting image set can be retained, which helps the model perform feature matching when faced with changes in object appearance or occlusion. By introducing a hybrid attention mechanism, the network can automatically learn feature weights, achieving adaptive feature utilization and improving the efficiency of feature fusion. Therefore, this scheme can preserve the local details of the supporting image set, and through adaptive feature fusion, the supporting features and query features can be adaptively fused, ensuring that the rich local detail information in the supporting images guides the query images to achieve high-precision recognition.

[0019] The following description Figure 1 The execution method for each step is shown.

[0020] For step 100: In this embodiment, a standard data input for a few-sample target recognition task is constructed using a query image to be identified and a support image set containing several labeled samples. The query image is the picture for which target localization is required, while the support image set provides visual reference information for specific categories.

[0021] For steps 102 and 104: In some implementations, the recognition model performs recognition and reasoning in the following manner: By utilizing two feature extraction networks with shared weights in the backbone network, features are extracted from the query image and the support image set, respectively, to obtain query features and support features. The adaptive relation distillation module is used to optimize the supporting features and then concatenate them with the processed query features to obtain the fused features. The fused features are input into the recognition module to generate candidate regions of the query image. Local features are extracted by combining the region of interest features of the fused features, so as to use classifiers and regressors to identify and infer small sample targets in the query image.

[0022] In this embodiment, the query image and support image sets are input into a feature extraction network with shared weights. This embodiment uses a ResNet-101 pre-trained on a general dataset as the backbone network. This network extracts deep features from the query image and support images respectively, generating query features and support features that reflect the semantic information of the images, serving as the basis for subsequent processing. An adaptive relation distillation module is constructed to perform deep interaction and fusion of the extracted features. This stage first uses a hybrid attention mechanism to weight and optimize the support features to adaptively enhance key local details and suppress noise. Subsequently, the optimized support features are spatially aligned and concatenated with the query features along the channel dimension, ultimately generating a fused feature containing rich contextual information. The generated fused feature is input into the recognition head. First, a region proposal network generates candidate regions, and local features are obtained by combining region of interest feature extraction operations. Finally, a classifier and regressor composed of fully connected layers complete the final inference, outputting the target's category prediction result and precise bounding box coordinates.

[0023] In this embodiment of the invention, the adaptive relation distillation module includes: a cascaded channel attention module and a spatial attention module, as well as a support encoder and a query encoder.

[0024] In some implementations, an adaptive relation distillation module is used to optimize supporting features and concatenate them with processed query features to obtain fused features, including: Obtain the support features of the support image set output by the backbone network, average the support features in the category dimension, merge feature maps belonging to the same category, and generate a single-category category average feature. After weighting the class average features by the channel attention module in terms of channel dimension, the spatial attention module is used to weight the weighted class average features in terms of spatial dimension to obtain attention-optimized features. The attention-optimized features are convolved using a support encoder to reduce the number of channels by half. Then, the attention-optimized features with reduced channel dimension are scaled to the same width and height as the query features using a bilinear interpolation algorithm to obtain spatially aligned support features. The query features are convolved using a query encoder to reduce the number of channels by half while keeping the spatial size unchanged, resulting in dimensionality-reduced query features. The dimensionality-reduced query features and spatially aligned support features are concatenated along the channel dimension to obtain fused features that contain semantic information of the query image and fine-grained details of the support image; the number of channels of the fused features is consistent with that of the query features output by the backbone network.

[0025] In this embodiment, the adaptive relation distillation module employs a five-step sequential processing flow. By progressively optimizing supporting features and adapting to query features, it ultimately achieves effective fusion of the two. The specific processing steps are as follows: Step 1: The module first receives the supporting features output by the feature extraction network. To eliminate redundant differences between samples of the same type, the module performs an averaging operation on the supporting features along the category dimension, merging feature maps belonging to the same category to generate a single-category average feature.

[0026] Step 2: Input the categorical average features obtained in the previous step into the hybrid attention unit. This unit contains a cascaded channel attention module and a spatial attention module. The features first pass through the channel attention module, where a multilayer perceptron learns the dependencies between channels; then they pass through the spatial attention module, where convolutional layers extract the importance of spatial locations. Through this process, the feature responses of key object regions are adaptively enhanced while suppressing background noise, outputting attention-optimized features.

[0027] Step 3: To ensure the supporting features match the query features, the attention-optimized features are first convolved using a support encoder, reducing the number of channels by half. Then, addressing the issue that the supporting feature size is typically smaller than the query feature, bilinear interpolation is used to enlarge the width and height of the feature map, making its spatial dimensions identical to the query feature, thus generating spatially aligned supporting features.

[0028] Step 4: After preparing the supporting features, the module processes the input query features. A query encoder is used to perform a convolution operation on the original query features, reducing their channel count by half to obtain dimensionality-reduced query features. This step aims to adjust the channel dimensions of the query features to maintain dimensionality balance with the processed supporting features after fusion.

[0029] Step 5: Finally, the dimensionality-reduced query features obtained in Step 4 and the spatially aligned support features obtained in Step 3 are concatenated along the channel dimension. The concatenated fused feature has the total number of channels restored to the original query features, while maintaining the same spatial size. This fused feature contains the semantic information of the query image and the fine-grained details of the support image, and is then output to the recognition head for subsequent target classification and localization.

[0030] Through the above implementation methods, the present invention can effectively integrate information from support sets and query images, and adaptively focus on key features using a hybrid attention mechanism, providing an effective solution for small sample target recognition tasks.

[0031] In this embodiment, the encoder is supported by a single 3×3 convolutional layer.

[0032] Please refer to Figure 2This invention provides a small-sample target recognition device based on adaptive relation distillation, used to implement the steps of any method embodiment in the specification. The device includes: The acquisition unit 201 is used to acquire the query image and the supporting image set; the supporting image set contains several types of small sample targets; The recognition unit 202 is used to input the query image and the support image set into the recognition model, so as to use the recognition model to recognize and infer the small sample target of the query image based on the support image set; wherein, the recognition model includes a backbone network for extracting image features of the query image and the support image set respectively, an adaptive relation distillation module for deep interaction and fusion of the image features extracted by the backbone network, and a recognition module; the adaptive relation distillation module contains a channel attention module and a spatial attention module; Output unit 203 is used to obtain the target category and target bounding box coordinates of the query image.

[0033] It should be noted that the above device embodiments and method embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0034] Embodiments of this application also provide a computer device, please refer to... Figure 3 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the small sample target recognition method based on adaptive relation distillation provided in the above method embodiments.

[0035] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the small sample target recognition method based on adaptive relation distillation provided in the above-described method embodiments.

[0036] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the small sample target recognition methods based on adaptive relation distillation described in the above embodiments.

[0037] For ease of description, the above devices or apparatuses are described separately according to their functions, divided into various modules or units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0038] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0039] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0040] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A small-sample target recognition method based on adaptive relation distillation, characterized in that, include: Obtain the query image and a support image set; the support image set contains several types of small sample targets; The query image and the support image set are input into a recognition model to identify and infer small sample targets in the query image based on the support image set. The recognition model includes a backbone network for extracting image features from the query image and the support image set, an adaptive relation distillation module for deep interaction and fusion of the image features extracted by the backbone network, and a recognition module. The adaptive relation distillation module contains a channel attention module and a spatial attention module. The target category and bounding box coordinates of the query image are obtained.

2. The method as described in claim 1, characterized in that, The recognition model performs recognition and reasoning in the following manner: By utilizing two shared-weight feature extraction networks in the backbone network, feature extraction is performed on the query image and the support image set respectively to obtain query features and support features; The adaptive relation distillation module is used to optimize the supporting features and then concatenate them with the processed query features to obtain fused features. The fused features are input into the recognition module to generate candidate regions of the query image. Local features are extracted by combining the region of interest features of the fused features, so as to use classifiers and regressors to identify and infer small sample targets of the query image.

3. The method as described in claim 2, characterized in that, The adaptive relation distillation module includes: a cascaded channel attention module and a spatial attention module, as well as a support encoder and a query encoder.

4. The method as described in claim 2, characterized in that, The adaptive relation distillation module is used to optimize the supporting features and concatenate them with the processed query features to obtain fused features, including: Obtain the support features of the support image set output by the backbone network, perform an averaging operation on the support features in the category dimension, merge feature maps belonging to the same category, and generate a single-category category average feature. After weighting the category average features by the channel attention module in terms of channel dimension, the weighted category average features are then weighted by the spatial attention module in terms of spatial dimension to obtain attention-optimized features. The attention-optimized features are convolved using a support encoder to reduce the number of channels by half. Then, the attention-optimized features with reduced channel dimension are scaled using a bilinear interpolation algorithm to the same width and height as the query features, resulting in spatially aligned support features. The query features are convolved using a query encoder to reduce the number of channels by half while keeping the spatial size unchanged, resulting in dimensionality-reduced query features. The dimensionality-reduced query features and spatially aligned support features are concatenated along the channel dimension to obtain a fused feature that contains semantic information of the query image and fine-grained details of the support image; the number of channels of the fused feature is consistent with that of the query features output by the backbone network.

5. The method as described in claim 3, characterized in that, The supporting encoder consists of a single 3×3 convolutional layer.

6. The method as described in claim 5, characterized in that, The feature extraction network of the backbone network employs a ResNet-101 network pre-trained on a general image dataset.

7. A small-sample target recognition device based on adaptive relational distillation, used to implement the steps of the method according to any one of claims 1-6, characterized in that, include: The acquisition unit is used to acquire the query image and the supporting image set; The supporting image set contains several types of small sample targets; A recognition unit is configured to input the query image and the support image set into a recognition model, so as to use the recognition model to recognize and infer small sample targets in the query image based on the support image set; wherein, the recognition model includes a backbone network for extracting image features from the query image and the support image set respectively, an adaptive relation distillation module for deep interaction and fusion of the image features extracted by the backbone network, and a recognition module; the adaptive relation distillation module includes a channel attention module and a spatial attention module; The output unit is used to obtain the target category and target bounding box coordinates of the query image.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images