Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Self-supervised learning fine-grained image classification method based on twin network

A technology of supervised learning and twin networks, applied in the field of self-supervised learning fine-grained image classification based on twin networks, can solve problems such as difficult network loss functions, inability to accurately locate semantic components, and focus on subtle regional identification features to achieve improved classification Effects on Accuracy, Enhanced Robustness, and Generalization

Pending Publication Date: 2022-06-28
INST OF SOFTWARE - CHINESE ACAD OF SCI
View PDF0 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In order to solve this problem, at this stage, a weak supervision method based on the attention mechanism is used to locate distinguishable regions, and the classification task is completed by combining the characteristics of the region and the overall image features, but it is necessary to train multiple sub-networks / models, one for positioning semantics component, and the other is used for the final classification prediction. This multi-task mode makes it difficult to optimize the loss function of the network to an optimal situation.
Furthermore, due to only image-level annotation information, the network cannot accurately locate the region where the semantic component is located, and the acquired region contains more or less background noise
In contrast, the way of encoding high-order features can enhance feature representation, but it cannot explicitly guide the model to focus on discriminative features in subtle regions.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Self-supervised learning fine-grained image classification method based on twin network
  • Self-supervised learning fine-grained image classification method based on twin network
  • Self-supervised learning fine-grained image classification method based on twin network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0024] The self-supervised learning fine-grained image classification method based on the Siamese network according to the present invention comprises the following steps:

[0025] Step S1: Build an attention encoder, first extract the feature F of the input image by using the backbone network pre-trained on the large-scale classification dataset; the backbone network has a wide range of options, and you can choose the residual network (ResNet) and the classical depth Learning model VGG16 et al. The backbone network is pretrained on large-scale classification datasets such as ImageNet or Microsoft COCO. Second, the rich image information is captured by the bilinear pooling module. First, the image feature F is processed by a 1×1 convolutional layer (Conv), activation function (ReLU) and batch normalization (BN), thereby reducing the channel number, discard redundant information, and obtain attention map A with rich semantic information. Then, the image feature F and the atte...

Embodiment 2

[0052] Embodiment 2 of the present invention provides an electronic device, which is a memory and a processor, and is characterized in that, when a fine-grained image classification program based on self-supervised learning based on a twin network is stored and executed by the processor, the processor executes the algorithm based on the twin network. A self-supervised learning method for fine-grained image classification, which includes the following steps:

[0053] 1) Extract the feature representation of the input image using the constructed attention encoder;

[0054] 2) Obtain two different perspective samples of the image based on the attention and semantic erasure mechanism;

[0055] 3) Through the Siamese network, extract the sample features of different perspectives and guide the network to learn the perspective invariant features in a self-supervised manner;

[0056] 4) The central loss function is used to constrain the sample feature distance within the class;

[0...

Embodiment 3

[0059] Embodiment 3 of the present invention provides a computer-readable storage medium, characterized in that, when the program is executed by a processor, the processor is caused to execute a self-supervised learning fine-grained image classification method based on a twin network, and the method includes the following steps:

[0060] 1) Extract the feature representation of the input image using the constructed attention encoder;

[0061] 2) Using the attention mechanism and the semantic erasure mechanism to obtain two different perspective samples of the image;

[0062] 3) Through the Siamese network, extract the sample features of different perspectives and guide the network to learn the perspective invariant features in a self-supervised manner;

[0063] 4) The central loss function is used to constrain the sample feature distance within the class;

[0064] 5) The samples learned by the network and their features from different perspectives are sent to the classifier t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a self-supervised learning fine-grained image classification method based on a twin network. According to the method, firstly, an attention encoder is used for extracting deep convolution features of an image, an attention graph containing semantic information is obtained, and image features are encoded in a bilinear pooling mode; secondly, positioning a salient region where a high response value is located on the attention map from the original image, cutting and erasing the salient region so as to form views of different view angles, and learning view angle invariance features in a self-supervision mode; and finally, combining the center loss function and the consistency loss function to display and constrain different view angle characteristics, and keeping the intra-class consistency of the different view angle characteristics. The method enables the network to obtain significant performance gain, and can significantly improve the classification accuracy on the baseline of fine-grained image classification.

Description

technical field [0001] The invention relates to the technical field of image classification, in particular to fine-grained image classification, in particular to a self-supervised learning fine-grained image classification method based on a twin network. Background technique [0002] With the increasing maturity of deep learning technology and the continuous improvement of social intelligence, people have put forward higher requirements for the classification ability of computers. It is expected that computers can not only correctly identify the objects of general image classification with large differences in appearance, but also need to be able to Identify objects at a fine-grained level. Fine-grained image classification refers to distinguishing sub-categories belonging to the same basic category, and it has a wide range of applications in vehicle tracking, automatic commodity settlement, and wildlife protection. Because obtaining part and object label information requir...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06N3/04G06N3/08G06V10/764G06V10/82
CPCG06N3/08G06N3/045G06F18/2415
Inventor 汲如意李佳盈张立波武延军
Owner INST OF SOFTWARE - CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com