Small sample fine-grained image classification method based on space-frequency domain hierarchical learning

By using space-frequency domain hierarchical learning technology in the small sample fine-grained image classification method, combined with dual frequency domain interactive enhancement and support set-query set interactive learning modules, the problem of feature distribution deviation and subtle feature identification when data is insufficient is solved, and the problem is achieved It has higher recognition accuracy and generalization ability, and is suitable for plant disease identification.

CN119339156AActive Publication Date: 2025-01-21GUIZHOU UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411635304.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-15
Publication Date
2025-01-21
Estimated Expiration
2044-11-15

AI Technical Summary

Technical Problem

The existing small sample fine-grained image classification method can easily lead to deviations from the original data distribution when the number of data is insufficient, affecting the generalization ability of the model. Especially in plant disease recognition, subtle and scattered discriminant features are difficult to be accurate. Identify.

Method used

Adopt a small sample fine-granular image classification method based on air-frequency domain hierarchy, through dual-frequency domain interaction modules, airspace-frequency domain hierarchical learning modules and support set-query set interactive learning modules Level fusion and interactive learning to optimize the classification model to improve recognition accuracy.

Benefits of technology

It significantly improves the generalization capacity of the model under a small amount of data, can more accurately identify the slightly differential characteristics of judgment, suitable for plant disease recognition, and reduce the recognition cost.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119339156A_ABST
    Figure CN119339156A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image processing, in particular to a small-sample fine-grained image classification method based on space-frequency domain hierarchical learning, and the method comprises the steps: obtaining a to-be-recognized plant disease image; the to-be-recognized plant disease image is input into a preset disease classification model, a plant disease recognition result is obtained, and the disease classification model is used for hierarchically fusing spatial domain features and frequency domain features in the plant disease image to recognize plant diseases. According to the method, the problem of insufficient generalization ability caused by excessive fine-grained learning of an existing method can be relieved, the method is particularly more targeted to disease data represented by fine, different-size and dispersed discriminative features, the disease identification problem of plants can be detected in actual production activities, and the cost is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a small sample fine-grained image classification method based on space-frequency domain hierarchical learning. Background Art

[0002] Few-shot fine-grained image classification (FS-FGIC) refers to image classification tasks in which the number of samples for each category is very limited and the differences between categories are very subtle, which makes the classification task more challenging. FS-FGIC is often modeled using meta-learning, which uses category prototypes in the embedding space to achieve rapid adaptation. This approach is particularly suitable for scenarios where samples are scarce because it can leverage existing knowledge and experience to accelerate the learning process of new tasks, thereby achieving higher accuracy and generalization in fine-grained image classification.

[0003] In fine-grained image classification tasks, since there is a sufficient amount of data for learning, there is usually no problem for the model to focus on learning high-level fine-grained features. However, in the FS-FGIC task, due to the insufficient amount of data, excessive focus on learning fine-grained features may cause the feature space to deviate from the original data distribution, thereby affecting the generalization ability of the model in the test phase. It is also worth noting that the FS-FGIC task is usually trained based on the CUB dataset. In the images in the CUB dataset, the super-class discriminative regions of most pictures are usually large and concentrated, while there are large inter-class differences between sub-classes. In contrast, in actual plant disease classification applications, the discriminative features of different types of leaf diseases are often of different sizes and scattered, and the inter-class variance between sub-categories will be further reduced. These characteristics make the current plant disease recognition more difficult.

[0004] The shortcomings of the current FS-FGIC model are summarized as follows: (1) Due to insufficient data, excessive focus on learning fine-grained features may lead to deviations between the feature space and the original data distribution, thereby affecting the generalization ability of the model in the test phase; (2) It is not possible to identify and extract subtle, uneven, and scattered discriminative features; (3) These discriminative features that are difficult to identify often have smaller inter-class variances and larger intra-class variances, increasing the difficulty of identification. Therefore, in order to address the above technical issues and obtain more accurate plant disease identification results, the present invention proposes a small sample fine-grained image classification method based on spatial-frequency domain hierarchical learning. Summary of the invention

[0005] The purpose of the present invention is to provide a small sample fine-grained image classification method based on space-frequency domain hierarchical learning, which can alleviate the problem of insufficient generalization ability caused by excessive fine-grained learning in existing methods and can be applied to real plant disease identification scenarios.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A small sample fine-grained image classification method based on spatial-frequency domain hierarchical learning, comprising:

[0008] Acquire a plant disease image to be identified;

[0009] The plant disease image to be identified is input into a preset disease classification model to obtain a plant disease identification result, wherein the disease classification model is used to hierarchically fuse spatial domain features and frequency domain features in the plant disease image to identify plant diseases.

[0010] Optionally, the disease classification model includes a dual-frequency domain interactive enhancement module, a spatial-frequency domain hierarchical learning module, and a support set-query set interactive learning module. The dual-frequency domain interactive enhancement module is used to extract features from the input image and obtain the spatial domain features and frequency domain features of the input image; the spatial-frequency domain hierarchical learning module is used to enhance the spatial domain features and frequency domain features based on hierarchical learning to obtain enhanced spatial domain features and frequency domain features; the support set-query set interactive learning module is used to divide the enhanced spatial domain features and frequency domain features into support sets and query sets, and interactively learn the divided support sets and query sets, optimize the disease classification model based on the interactive learning results, and output the final classification results.

[0011] Optionally, the dual-frequency domain interaction enhancement module extracts features from the input image, and obtaining spatial domain features and frequency domain features of the input image includes:

[0012] Inputting the RGB image into the neural network model to obtain the spatial domain features;

[0013] Performing color space conversion on the RGB image to obtain a YCbCr image, and performing block division and DCT transformation on the YCbCr image to obtain a DCT block;

[0014] Reorganizing the DCT blocks to construct DCT frequency bands, and splitting the DCT frequency bands into high-frequency features and low-frequency features;

[0015] A self-attention operation is used to enhance the high-frequency features and the low-frequency features respectively, and then a splicing operation is performed to obtain a total feature. The self-attention operation is then used to enhance the total feature and then a Reshape operation is performed to obtain a frequency domain feature equal in size to the spatial domain feature.

[0016] Optionally, reorganizing the DCT blocks to construct DCT frequency bands includes:

[0017] The DCT blocks of the same frequency domain components are spliced, and the DCT blocks of different frequency domain components are sorted and reorganized from low frequency to high frequency to obtain the DCT frequency band.

[0018] Optionally, the spatial-frequency domain hierarchical learning module enhances the spatial features and the frequency domain features based on hierarchical learning, and obtaining the enhanced spatial features and the frequency domain features includes:

[0019] The spatial domain features and the frequency domain features are respectively scaled to obtain spatial domain features and frequency domain features at several levels;

[0020] Adaptive learning between spatial domain features and frequency domain features is performed at each level, and adaptive feature fusion is performed on the spatial domain features and frequency domain features after adaptive learning from large to small levels to obtain spatial domain features and frequency domain features after hierarchical learning;

[0021] Self-attention operations are performed on the spatial domain features and frequency domain features after hierarchical learning to obtain enhanced spatial domain features and frequency domain features.

[0022] Optionally, performing adaptive learning between spatial domain features and frequency domain features at each level includes:

[0023] The spatial domain features and frequency domain features of the same level are concatenated to obtain concatenated features, and average pooling and softmax function are applied to the concatenated features to obtain descriptors of each level;

[0024] Multiplying the descriptor with the spatial features at the corresponding level and the first learnable matrix to obtain the spatial features after adaptive learning;

[0025] The descriptor is multiplied with the frequency domain features at the corresponding level and the second learnable matrix to obtain the frequency domain features after adaptive learning.

[0026] Optionally, the support set-query set interactive learning module divides the enhanced spatial domain features and frequency domain features into support sets and query sets, and interactively learns the divided support sets and query sets, including:

[0027] The enhanced spatial domain features and frequency domain features are divided into support sets and query sets respectively to obtain spatial domain support sets, spatial domain query sets, frequency domain support sets, and frequency domain query sets;

[0028] The spatial domain support set and the spatial domain query set are interactively learned with the frequency domain support set and the frequency domain query set respectively to obtain an interactive learning score, wherein the interactive learning score is obtained by calculating the Euclidean distance between the two parties performing the interactive learning.

[0029] Optionally, optimizing the disease classification model based on the interactive learning result includes: performing back-propagation optimization on the disease classification model based on the obtained interactive learning score calculation loss.

[0030] The beneficial effects of the present invention are:

[0031] The present invention uses frequency domain and spatial domain features to improve the original small sample fine-grained network, which can alleviate the problem of insufficient generalization ability caused by excessive fine-grained learning in existing methods, and enable the network model to solve the problem that traditional networks are difficult to accurately classify images with small and scattered discriminative areas under the condition of a small increase in calculation parameters, thereby significantly improving the accuracy of reasoning. In particular, it is more targeted at plant disease data represented by subtle, uneven and scattered discriminative features, and can solve the problem of plant disease identification in actual production activities and reduce costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A schematic diagram of the structure and workflow of a dual-frequency domain interaction enhancement module according to an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the structure and workflow of a spatial-frequency-domain hierarchical learning module according to an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the support set-query set interactive learning module structure and workflow of an embodiment of the present invention;

[0036] Figure 4 An image effect diagram reconstructed by a support set-query set interactive learning module according to an embodiment of the present invention;

[0037] Figure 5 This is a heat map of the recognition results of the disease classification model of an embodiment of the present invention on a plant disease dataset. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] This embodiment provides a small sample fine-grained image classification method based on space-frequency domain hierarchical learning, including:

[0041] Obtain a plant disease image to be identified, input the plant disease image to be identified into a preset disease classification model, and obtain a plant disease identification result, wherein the disease classification model is used to hierarchically fuse the spatial domain features and frequency domain features in the plant disease image to identify the plant disease.

[0042] The disease classification model includes a dual-frequency domain interactive enhancement module, a spatial-frequency domain hierarchical learning module, and a support set-query set interactive learning module. The dual-frequency domain interactive enhancement module is used to extract features of the input image and obtain the spatial and frequency domain features of the input image; the spatial-frequency domain hierarchical learning module is used to enhance the spatial and frequency domain features based on hierarchical learning to obtain the enhanced spatial and frequency domain features; the support set-query set interactive learning module is used to divide the enhanced spatial and frequency domain features into support sets and query sets, and interactively learn the divided support sets and query sets, optimize the disease classification model based on the interactive learning results, and output the final classification results.

[0043] Specifically, this embodiment uses frequency domain and spatial domain features to improve the original small sample fine-grained network, which can alleviate the problem of insufficient generalization ability caused by excessive fine-grained learning in existing methods, and enable the network model to solve the problem that traditional networks are difficult to accurately classify images with small and scattered discriminative areas under the condition of a small increase in calculation parameters, thereby significantly improving the accuracy of reasoning. In particular, it is more targeted at disease data represented by subtle, unequally sized and scattered discriminative features, and can detect plant disease identification problems in actual production activities and reduce costs.

[0044] Furthermore, the dual-frequency domain interaction enhancement module extracts features from the input image, and obtains spatial domain features and frequency domain features of the input image, including:

[0045] Input the RGB image into the neural network model to obtain spatial domain features;

[0046] Perform color space conversion on the RGB image to obtain a YCbCr image, and perform DCT transformation on the YCbCr image to obtain a DCT block;

[0047] Reorganizing the DCT blocks to construct DCT frequency bands, and splitting the DCT frequency bands into high-frequency features and low-frequency features, wherein reorganizing the DCT blocks to construct the DCT frequency bands includes: splicing DCT blocks of the same frequency domain components, and reorganizing DCT blocks of different frequency domain components in order from low frequency to high frequency to obtain DCT frequency bands;

[0048] The self-attention operation is used to enhance the high-frequency features and low-frequency features respectively, and then the splicing operation is performed to obtain the total features. The self-attention operation is then used to enhance the total features and then the Reshape operation is performed to obtain the frequency domain features that are equal to the spatial domain features.

[0049] Specifically, the structure and workflow of the dual-frequency domain interaction enhancement module of this embodiment are as follows: Figure 1 As shown, data processing is performed to obtain spatial domain feature representation and frequency domain feature representation of the image, including:

[0050] Step 1.1. Input the image into the Resnet12 / Conv4 network model to obtain the spatial feature F processed by the CNN neural network s , where the input must be divided into a support set and a query set, and the image color space must be RGB.

[0051] Specifically, Resnet12 contains 4 residual blocks, each with 3 convolutional layers. "12" indicates that there are 12 convolutional layers in total; after the image passes through the first residual block, 3×84×84 becomes 64×42×42 (2×2 maximum pooling); after passing through the second residual block, it becomes 160×21×21; the third residual block outputs 320×10×10; the fourth residual block outputs 640×5×5. After 5×5 average pooling, the output is 640×1×1, DropBlock is used to prevent overfitting, and then it is flattened to 640 dimensions according to the channel. Conv4 means that the network contains 4 repeated convolution blocks. The four convolution blocks are the same. The image shape changes as follows: after the image passes through the first convolution block, 3×84×84 becomes 64×42×42 (2×2 maximum pooling); after passing through the second convolution block, it becomes 64×21×21; the third convolution block outputs 64×10×10; and the fourth convolution block outputs 64×5×5.

[0052] Step 1.2. Convert the original RGB image to YCbCr color space. The YCbCr image includes brightness, blue, and red. Compared with color, the human eye is more sensitive to brightness, so using YCbCr representation will be more conducive to subsequent processing;

[0053] Specifically, the calculation process of RGB→YCbCr is as follows:

[0054] Y=0.257*R+0.564*G+0.098*B+16

[0055] Cb=-0.148*R-0.291*G+0.439*B+128

[0056] Cr=0.439*R-0.368*G-0.071*B+128

[0057] Step 1.3. Divide the obtained YCbCr image into blocks of 8×8 pixels;

[0058] Step 1.4. Use two-bit discrete cosine transform to convert the spatial domain to frequency domain information of the obtained 8×8 block. The low-frequency information of the converted block is concentrated in the upper left corner, and the high-frequency information is concentrated in the lower right corner;

[0059] Specifically, the DCT transform formula is as follows:

[0060] Let f(x,y) be an M×N digital image matrix:

[0061]

[0062] In the formula, x,u=0,1,2,…,M-1; y,v=0,1,2,…,N-1.

[0063] Step 1.5. Reorganize the DCT blocks, splice the same frequency domain components of each DCT block together, and then reorganize the blocks of different frequency domain components in the order of low frequency → high frequency to obtain the DCT frequency band;

[0064] Step 1.6. Split the DCT band into two from the middle to obtain the high-frequency feature F h and low frequency features F l ;

[0065] Step 1.7. Use self-attention operation to separately h and low frequency features F l Perform enhancement processing;

[0066] Specifically, the self-attention calculation formula is as follows:

[0067] First, for F hand F l Make three copies, Q h , K h 、V h , Q l , K l and V l :

[0068]

[0069] Step 1.8. The processed high-frequency features F h and low frequency features F l Splice them together to get the total feature F ALL ;

[0070] Step 1.9. Use the attention operation again on the total feature F f Perform enhancement processing;

[0071] Step 1.10. The processed total feature F ALL Perform the Reshape operation to obtain the spatial feature F s Frequency domain features F of equal magnitude f .

[0072] Furthermore, the spatial-frequency hierarchical learning module enhances the spatial features and the frequency features based on hierarchical learning, and the enhanced spatial features and the frequency features are obtained including:

[0073] The spatial domain features and the frequency domain features are scaled and classified respectively to obtain several levels of spatial domain features and frequency domain features;

[0074] Adaptive learning between spatial domain features and frequency domain features is performed at each level, and adaptive feature fusion is performed on the spatial domain features and frequency domain features after adaptive learning from large to small levels to obtain spatial domain features and frequency domain features after hierarchical learning;

[0075] Self-attention operations are performed on the spatial domain features and frequency domain features after hierarchical learning to obtain enhanced spatial domain features and frequency domain features.

[0076] Among them, the adaptive learning between spatial domain features and frequency domain features at each level includes:

[0077] The spatial domain features and frequency domain features of the same level are concatenated to obtain concatenated features, and average pooling and softmax functions are used on the concatenated features to obtain descriptors of each level;

[0078] Multiply the descriptor with the spatial features at the corresponding level and the first learnable matrix to obtain the spatial features after adaptive learning;

[0079] The descriptor is multiplied with the frequency domain features at the corresponding level and the second learnable matrix to obtain the frequency domain features after adaptive learning.

[0080] Specifically, the structure and workflow of the spatial-frequency domain hierarchical learning module of this embodiment are as follows: Figure 2 As shown, the spatial and frequency domain features are learned hierarchically to make them more sensitive to subtle, small and scattered discriminative areas, including:

[0081] Step 2.1. After obtaining the spatial feature F s And the frequency domain feature F f After that, the two features are scaled and divided into four levels respectively. as well as The scale sizes are 1, 1 / 2, 1 / 4 and 1 / 8 respectively;

[0082] Step 2.2. Between the various levels Perform adaptive feature learning in spatial domain and frequency domain respectively;

[0083] Step 2.2.1. and Splice it together;

[0084] Step 2.2.2. Use average pooling and softmax function on the concatenated tensors to obtain the descriptor μ;

[0085] Step 2.2.3. Compare the descriptor μ with And the learnable matrix W s Multiply them together and you get

[0086] Step 2.2.4. Compare the descriptor μ with And the learnable matrix W f Multiply them together and you get

[0087] Step 2.2.5. and The fused features are obtained by splicing them together.

[0088] Step 2.2.6. The remaining fusions between the layers are calculated according to this process;

[0089] Step 2.3. Adaptively fuse the features of each level in the spatial domain from large to small to obtain the spatial domain features after hierarchical learning;

[0090] Step 2.4. Perform self-attention operation on the spatial features to strengthen the spatial feature representation.

[0091] Step 2.5. Adaptively fuse the features of each level in the frequency domain from large to small to obtain the frequency domain features after hierarchical learning;

[0092] Step 2.6. Perform self-attention operation on the frequency domain features to strengthen the frequency domain feature representation.

[0093] Step 2.7. Get the final enhanced feature representation and

[0094] Furthermore, the support set-query set interactive learning module divides the enhanced spatial domain features and frequency domain features into support sets and query sets, and interactively learns the divided support sets and query sets, including:

[0095] The enhanced spatial domain features and frequency domain features are divided into support sets and query sets respectively to obtain spatial domain support sets, spatial domain query sets, frequency domain support sets, and frequency domain query sets;

[0096] The spatial domain support set and the spatial domain query set are interactively learned with the frequency domain support set and the frequency domain query set respectively to obtain an interactive learning score, wherein the interactive learning score is obtained by calculating the Euclidean distance between the two parties performing the interactive learning.

[0097] Optimizing the disease classification model based on the interactive learning results includes: calculating the loss based on the obtained interactive learning score and performing back-propagation optimization on the disease classification model.

[0098] Specifically, the structure and workflow of the support set-query set interactive learning module of this embodiment are as follows: Figure 3 As shown, and Divide the support set and query set, and conduct interactive learning to increase the variance between classes and reduce the variance within classes. The learning reconstruction effect is shown in Figure 4 The steps include:

[0099] Step 3.1. and Divide the support set and query set to get S sp , S fr , Q sp and Q fr .

[0100] Step 3.2. Perform S on the four sets sp →Q sp , S sp →Q tr , S fr →Q sp and S fr →Q fr interactive learning;

[0101] Step 3.2.1. S sp →Q sp For example, S sp and Q sp Copy each one and get and

[0102] Step 3.2.2. and Multiply the transpose of and pass through the softmax function to get the similarity matrix M;

[0103] Step 3.2.3. Similarity matrix M and as well as Multiply them separately and get and

[0104] Step 3.2.4. Calculate S sp and The Euclidean distance between them is recorded as scores;

[0105] Step 3.3. Calculate the loss based on the size of the scores and perform back-propagation optimization.

[0106] Specifically, the loss function is calculated as follows:

[0107]

[0108] L=L s +L f

[0109] Among them, N is the number of categories, U is the number of query sets, τ a is the control parameter and d is the Euclidean distance.

[0110] The disease classification model proposed in this embodiment is tested on the plant disease dataset to obtain the recognition result heat map. Figure 5 It can be seen that the method and disease classification model proposed in this embodiment can accurately identify plant disease areas and classification results, and can be applied to real plant disease identification scenarios, greatly reducing costs.

[0111] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A small sample fine-grained image classification method based on spatial-frequency domain hierarchical learning, characterized in that: include: Acquire a plant disease image to be identified; The plant disease image to be identified is input into a preset disease classification model to obtain a plant disease identification result, wherein the disease classification model hierarchically fuses the spatial domain features and frequency domain features in the plant disease image to identify the plant disease.

2. The small sample fine-grained image classification method based on space-frequency domain hierarchical learning according to claim 1 is characterized in that: The disease classification model includes a dual-frequency domain interactive enhancement module, a spatial-frequency domain hierarchical learning module, and a support set-query set interactive learning module. The dual-frequency domain interactive enhancement module is used to extract features from the input image and obtain the spatial domain features and frequency domain features of the input image; the spatial-frequency domain hierarchical learning module is used to enhance the spatial domain features and frequency domain features based on hierarchical learning to obtain enhanced spatial domain features and frequency domain features; the support set-query set interactive learning module is used to divide the enhanced spatial domain features and frequency domain features into support sets and query sets, and interactively learn the divided support sets and query sets, optimize the disease classification model based on the interactive learning results, and output the final classification results.

3. The small sample fine-grained image classification method based on space-frequency domain hierarchical learning according to claim 2 is characterized in that: The dual-frequency domain interaction enhancement module extracts features from the input image, and obtaining spatial domain features and frequency domain features of the input image includes: Inputting the RGB image into the neural network model to obtain the spatial domain features; Performing color space conversion on the RGB image to obtain a YCbCr image, and performing block division and DCT transformation on the YCbCr image to obtain a DCT block; Reorganizing the DCT blocks to construct DCT frequency bands, and splitting the DCT frequency bands into high-frequency features and low-frequency features; A self-attention operation is used to enhance the high-frequency features and the low-frequency features respectively, and then a splicing operation is performed to obtain a total feature. The self-attention operation is then used to enhance the total feature and then a Reshape operation is performed to obtain a frequency domain feature equal in size to the spatial domain feature.

4. The small sample fine-grained image classification method based on space-frequency domain hierarchical learning according to claim 3 is characterized in that: The DCT blocks are reorganized to construct DCT frequency bands, including: The DCT blocks of the same frequency domain components are spliced, and the DCT blocks of different frequency domain components are sorted and reorganized from low frequency to high frequency to obtain the DCT frequency band.

5. The small sample fine-grained image classification method based on space-frequency domain hierarchical learning according to claim 2 is characterized in that: The spatial-frequency domain hierarchical learning module enhances the spatial features and frequency domain features based on hierarchical learning, and obtaining the enhanced spatial features and frequency domain features includes: The spatial domain features and the frequency domain features are respectively scaled to obtain spatial domain features and frequency domain features at several levels; Adaptive learning between spatial domain features and frequency domain features is performed at each level, and adaptive feature fusion is performed on the spatial domain features and frequency domain features after adaptive learning from large to small levels to obtain spatial domain features and frequency domain features after hierarchical learning; Self-attention operations are performed on the spatial domain features and frequency domain features after hierarchical learning to obtain enhanced spatial domain features and frequency domain features.

6. The small sample fine-grained image classification method based on space-frequency domain hierarchical learning according to claim 5 is characterized in that: The adaptive learning between spatial domain features and frequency domain features at each level includes: The spatial domain features and frequency domain features of the same level are concatenated to obtain concatenated features, and average pooling and softmax function are applied to the concatenated features to obtain descriptors of each level; Multiplying the descriptor with the spatial features at the corresponding level and the first learnable matrix to obtain the spatial features after adaptive learning; The descriptor is multiplied with the frequency domain features at the corresponding level and the second learnable matrix to obtain the frequency domain features after adaptive learning.

7. The small sample fine-grained image classification method based on space-frequency domain hierarchical learning according to claim 2 is characterized in that: The support set-query set interactive learning module divides the enhanced spatial domain features and frequency domain features into support sets and query sets, and interactively learns the divided support sets and query sets, including: The enhanced spatial domain features and frequency domain features are divided into support sets and query sets respectively to obtain spatial domain support sets, spatial domain query sets, frequency domain support sets, and frequency domain query sets; The spatial domain support set and the spatial domain query set are interactively learned with the frequency domain support set and the frequency domain query set respectively to obtain an interactive learning score, wherein the interactive learning score is obtained by calculating the Euclidean distance between the two parties performing the interactive learning.

8. The small sample fine-grained image classification method based on space-frequency domain hierarchical learning according to claim 7 is characterized in that: Optimizing the disease classification model based on the interactive learning result includes: performing back-propagation optimization on the disease classification model based on the obtained interactive learning score calculation loss.

Citation Information

Patent Citations

  • Unmanned aerial vehicle landing landform image classification method based on DCT-CNN model

    CN107748895A

  • No-reference image quality evaluation method and device and storage medium

    CN115359000A

  • Small sample image classification method based on frequency domain analysis

    CN117372782A

  • Image feature enhancement algorithm fusing frequency domain and spatial domain features

    CN118172635A

  • Space domain-frequency domain combined enhanced steel member defect size intelligent measuring system

    CN118761959A