A cross-domain few-shot picture classification method and device based on an input adaptive neural network architecture

Abstract

A cross-domain small sample picture classification method and device based on an input adaptive neural network architecture, the method comprising: collecting a large picture dataset for pre-training; building an input adaptive neural network architecture; pre-training the neural network on the large picture dataset, training to obtain the weights of each layer of the neural network; on a target cross-domain small sample picture classification task, removing the fully connected layer of the neural network and fixing the weights of the neural network, training a logistic regression method on the cross-domain small sample pictures for training to obtain the weights of the logistic regression classifier; after training, inputting the cross-domain small sample pictures for testing into the neural network to obtain a picture classification result. The method has the advantages that the neural network used by the method has the ability to adaptively adjust the weights according to the input, and retains the inductive bias ability of the convolutional layer, avoiding the use of a large amount of training data, and is particularly suitable for cross-domain small sample picture classification scenarios.

Description

Technical Field

[0001] This invention relates to the field of cross-domain few-shot image classification technology, and in particular to a cross-domain few-shot image classification method and apparatus based on an input adaptive neural network architecture. Background Technology

[0002] Neural network models typically require a large amount of labeled image data for training to achieve good image classification results. However, in practical applications, collecting large amounts of labeled image data can be quite costly. Massive amounts of images require expert manual annotation, which is inefficient in terms of manpower and time. Furthermore, in fields such as medicine, it is difficult or impossible to collect labeling information due to privacy, security, or ethical concerns. Compared to machines, humans can quickly learn from a small number of image samples based on prior experience. These factors have given rise to few-shot image classification methods, which aim to obtain a reasonably effective few-shot image classification model through training with a small number of labeled image samples. Most few-shot image classification methods assume that the target data follows the same distribution as the training data. However, in real-world scenarios, the distribution of the target classification images may differ from the data used for training; this is called cross-domain few-shot image classification. For example, consider a scenario where we want to use a few-shot model trained on distinguishing cats and dogs to distinguish between Labrador Retrievers and Huskies.

[0003] Existing cross-domain few-shot image classification methods typically choose convolutional neural networks (CNNs) as their neural network architecture. Convolutional layers apply the same weights to all regions of an image and maintain these weights fixed after training, lacking the ability to dynamically adjust weights based on the input cross-domain few-shot images. In contrast, adaptive layers adaptively calculate weights based on the input, a feature that meets the needs of cross-domain few-shot image classification. However, due to the lack of inductive bias, pure adaptive layer architectures require large amounts of training data, making them incompatible with few-shot image classification applications. Summary of the Invention

[0004] To overcome the aforementioned shortcomings of existing technologies, this invention proposes a cross-domain few-sample image classification method based on an input adaptive neural network architecture, enabling the neural network architecture to quickly and adaptively adjust weights according to the input cross-domain few-sample images and avoiding the use of a large amount of training data.

[0005] A cross-domain few-shot image classification method based on an input adaptive neural network architecture includes the following steps:

[0006] 1) Collect a large image dataset for pre-training;

[0007] 2) Build an input adaptive neural network architecture;

[0008] 3) Pre-train the neural network on a large image dataset to obtain the weights of each layer of the neural network;

[0009] 4) In the target cross-domain few-sample image classification task, remove the fully connected layer used in the pre-training of the neural network and fix the weights of the neural network. Use the logistic regression method to train on the cross-domain few-sample images used for training to obtain the weights of the logistic regression classifier.

[0010] 5) After training, input cross-domain small sample images for testing into the neural network to obtain image classification results.

[0011] In step 2), the input adaptive neural network architecture consists of the following components, starting from the input:

[0012] 21) Adaptive prefixes are divided into pseudo-adaptive prefixes and local adaptive prefixes;

[0013] 22) Stage 1 of the neural network consists of an adaptive layer and two 1×1 convolutional layers with a stride of 1;

[0014] 23) In the second stage of the neural network, the number of input channels and the width and height are first adjusted through the connection layer, and then through the adaptive layer, four 1×1 convolutional layers with a stride of 1 and another adaptive layer.

[0015] 24) In the third stage of the neural network, the number of input channels and the width and height are first adjusted through the connection layer, and then through 8 1×1 convolutional layers with a stride of 1 and an adaptive layer.

[0016] 25) In the fourth stage of the neural network, the number of input channels and the width and height are first adjusted through the connection layer, and then through the adaptive layer and four 1×1 convolutional layers with a stride of 1.

[0017] 26) Finally, the classification probability of each class is output through the average pooling layer and the fully connected layer.

[0018] In step 21), the pseudo-adaptive prefix approximates the adaptive weights between all points using adaptive weights between regions. Assuming each region is P×P in size, a P×P convolutional layer with stride P is used to extract region features. Following the pseudo-adaptive layer are a 1×1 convolutional layer with stride 1 and a max-pooling layer with stride P.

[0019] In step 21), the local adaptive prefix calculates local adaptive weights for each region. Assuming each region is P×P, all P×P regions are input into the same local adaptive layer to obtain the adaptive weights for each region. Following the local adaptive layer are a multilayer perceptron module and a max-pooling layer with a stride of P.

[0020] Among them, the connection layers in steps 23), 24), and 25) consist of a 1×1 convolutional layer with a stride of 1, a batch normalization layer, a modified linear unit, and a max pooling layer with a stride of 2.

[0021] In particular, the data distribution of the target cross-domain small sample image classification task in step 4) is different from the data distribution of the large image dataset in step 3).

[0022] A second aspect of the present invention provides a cross-domain few-shot image classification device based on an input adaptive neural network architecture, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a cross-domain few-shot image classification method based on an input adaptive neural network architecture according to the present invention.

[0023] The third aspect of this invention is to provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a cross-domain few-sample image classification method based on an input adaptive neural network architecture.

[0024] The advantages of this invention are: the adaptive prefix design places the adaptive layer at the input of the neural network, enabling the network to adaptively adjust weights based on the input; the use of adaptive and convolutional layers across different stages of the neural network preserves the inductive bias capability of the convolutional layers and avoids the need for large amounts of training data. This characteristic of adaptively adjusting weights based on input and having low requirements for training data is highly suitable for cross-domain small-sample image classification. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method of the present invention.

[0026] Figure 2 This is a diagram of the neural network architecture of the present invention.

[0027] Figure 3 is a diagram of the pseudo-adaptive prefix and local adaptive prefix used in the neural network architecture of this invention.

[0028] Figure 4 This is the connection layer diagram used in the neural network architecture of this invention. Detailed Implementation

[0029] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0030] Example 1

[0031] Referring to the accompanying drawings, the present invention provides a cross-domain few-sample image classification method based on an input adaptive neural network architecture, the method comprising the following steps:

[0032] 1) Collect a large image dataset for pre-training;

[0033] 2) Build an input adaptive neural network architecture;

[0034] 3) Pre-train the neural network on a large image dataset to obtain the weights of each layer of the neural network;

[0035] 4) In the target cross-domain few-sample image classification task, remove the fully connected layer used in the pre-training of the neural network and fix the weights of the neural network. Use the logistic regression method to train on the cross-domain few-sample images used for training to obtain the weights of the logistic regression classifier.

[0036] 5) After training, input cross-domain small sample images for testing into the neural network to obtain image classification results.

[0037] In step 2), the input adaptive neural network architecture consists of the following components, starting from the input:

[0038] 21) Adaptive prefixes are divided into pseudo-adaptive prefixes and local adaptive prefixes;

[0039] 22) Stage 1 of the neural network consists of an adaptive layer and two 1×1 convolutional layers with a stride of 1;

[0040] 23) In the second stage of the neural network, the number of input channels and the width and height are first adjusted through the connection layer, and then through the adaptive layer, four 1×1 convolutional layers with a stride of 1 and another adaptive layer.

[0041] 24) In the third stage of the neural network, the number of input channels and the width and height are first adjusted through the connection layer, and then through 8 1x1 convolutional layers with a stride of 1 and an adaptive layer.

[0042] 25) In the fourth stage of the neural network, the number of input channels and the width and height are first adjusted through the connection layer, and then through the adaptive layer and four 1×1 convolutional layers with a stride of 1.

[0043] 26) Finally, the classification probability of each class is output through the average pooling layer and the fully connected layer.

[0044] In step 21), the pseudo-adaptive prefix approximates the adaptive weights between all points using adaptive weights between regions. Assuming each region is P×P in size, a P×P convolutional layer with stride P is used to extract region features. Following the pseudo-adaptive layer are a 1×1 convolutional layer with stride 1 and a max-pooling layer with stride P.

[0045] In step 21), the local adaptive prefix calculates local adaptive weights for each region. Assuming each region is P×P, all P×P regions are input into the same local adaptive layer to obtain the adaptive weights for each region. Following the local adaptive layer are a multilayer perceptron module and a max-pooling layer with a stride of P.

[0046] Among them, the connection layers in steps 23), 24), and 25) consist of a 1×1 convolutional layer with a stride of 1, a batch normalization layer, a modified linear unit, and a max pooling layer with a stride of 2.

[0047] In particular, the data distribution of the target cross-domain small sample image classification task in step 4) is different from the data distribution of the large image dataset in step 3).

[0048] This invention performs the following operations on each cross-domain few-sample image: Collect a large image dataset for pre-training; construct an input adaptive neural network architecture; pre-train the neural network on the large image dataset to obtain the weights of each layer; on the target cross-domain few-sample image classification task, remove the fully connected layers of the neural network and fix the weights, then train the network using logistic regression on the cross-domain few-sample images used for training to obtain the weights of the logistic regression classifier; after training, input the cross-domain few-sample images used for testing into the neural network to obtain the image classification result. Its advantages are: the neural network used in this method has the ability to adaptively adjust weights according to the input, retains the inductive bias capability of the convolutional layers, avoids the use of a large amount of training data, and is particularly suitable for cross-domain few-sample image classification.

[0049] Example 2

[0050] This embodiment provides a cross-domain few-shot image classification device based on an input adaptive neural network architecture, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a cross-domain few-shot image classification method based on an input adaptive neural network architecture according to Embodiment 1.

[0051] Example 3

[0052] This embodiment provides a computer-readable storage medium storing a program that, when executed by a processor, implements a cross-domain few-sample image classification method based on an input adaptive neural network architecture, as described in Embodiment 1.

[0053] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A cross-domain few-sample image classification method based on an input adaptive neural network architecture, characterized in that, Includes the following steps: 1) Collect a large image dataset for pre-training; 2) Construct an input adaptive neural network architecture; the components of the input adaptive neural network architecture, starting from the input, are as follows: 21) Adaptive prefixes are divided into pseudo-adaptive prefixes and local adaptive prefixes; 22) Stage 1 of the neural network consists of an adaptive layer and two steps with a step size of 1. Convolutional layer composition; 23) In stage 2 of the neural network, the number of input channels and the width and height are first adjusted through a connection layer, and then through an adaptive layer with four steps of 1. Convolutional layer and another adaptive layer; 24) In stage 3 of the neural network, the number of input channels and the width and height are first adjusted through the connection layer, and then through 8 steps with a step size of 1. Convolutional layers and adaptive layers; 25) In stage 4 of the neural network, the number of input channels and the width and height are first adjusted through a connection layer, and then through an adaptive layer and four steps with a step size of 1. Convolutional layer; 26) Finally, the classification probability of each class is output through the average pooling layer and the fully connected layer; 3) Pre-train the neural network on a large image dataset to obtain the weights of each layer of the neural network; 4) In the target cross-domain few-sample image classification task, remove the fully connected layer used in the pre-training of the neural network and fix the weights of the neural network. Use the logistic regression method to train on the cross-domain few-sample images used for training to obtain the weights of the logistic regression classifier. 5) After training, input cross-domain small sample images for testing into the neural network to obtain image classification results.

2. The cross-domain few-sample image classification method based on an input adaptive neural network architecture as described in claim 1, characterized in that, Step 21) describes a pseudo-adaptive prefix that approximates the adaptive weights between all points using adaptive weights between regions. Assuming the size of each region is Using a step size of P Convolutional layers are used to extract region features, followed by a pseudo-adaptive layer with a stride of 1. Convolutional layers and max-pooling layers with stride P.

3. The cross-domain few-sample image classification method based on an input adaptive neural network architecture as described in claim 1, characterized in that, Step 21) involves calculating local adaptive weights for the local adaptive prefix in each region; assuming the size of each region is... , take all The regions are input into the same local adaptive layer to obtain the adaptive weights of each region. After the local adaptive layer, there are multilayer perceptron modules and a max pooling layer with a step size of P.

4. The cross-domain few-sample image classification method based on an input adaptive neural network architecture as described in claim 1, characterized in that, The connection layers in steps 23), 24), and 25) consist of a step size of 1. It consists of convolutional layers, batch normalization layers, modified linear units, and max pooling layers with a stride of 2.

5. The cross-domain few-sample image classification method based on an input adaptive neural network architecture as described in claim 1, characterized in that, The data distribution for the target cross-domain small sample image classification task in step 4) is different from the data distribution for the large image dataset in step 3).

6. A cross-domain few-sample image classification device based on an input adaptive neural network architecture, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a cross-domain few-sample image classification method based on an input adaptive neural network architecture as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements a cross-domain few-sample image classification method based on an input adaptive neural network architecture, as described in any one of claims 1-5.

Images