A method, system, device and medium for classifying small sample images of relational networks

Abstract

The present invention discloses a method, system, device and medium for classifying small sample images of a relational network, wherein the method comprises: dividing a data set, in each training batch, dividing the data set into a training set and a test set, and the samples of the training set are annotated with sample labels; using an embedding module to extract the image feature representation of the image in the data set; using a self-attention module to learn the context information between tasks, and converting the image feature representation irrelevant to the task into the image feature representation relevant to the task; according to the image feature representation relevant to the task, training a neural network in a relational network metric based on the original learning, and using the trained neural network for image classification. The present invention converts the task-irrelevant feature vector into the task-related feature vector through the attention mechanism, has the advantages of considering the task context information and reducing the distance within the class, and is more conducive to the subsequent metric calculation of the similarity score, and can be widely used in the field of computer vision.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method, system, device and medium for classifying small sample images of a relational network. Background Art

[0002] Small sample learning is one of the important research directions in artificial intelligence. Humans have the ability to quickly learn and summarize from a few samples. Taking the image classification task as an example, humans can quickly distinguish pictures of the same or different categories by observing a few picture samples and learning features from these samples. When children learn to recognize different animals when they are young, they usually only need to look at one or two images of this animal to recognize this species in different scenes, because they have learned a lot of experience and knowledge, and can quickly complete the recognition of new targets using this experience and knowledge. This ability to quickly learn from a few samples and learn new knowledge using the experience learned, rather than learning from scratch for each task, is called small sample learning.

[0003] The model is trained through limited samples, and then it has the same recognition ability when facing new samples that have not appeared in the training process. With the development of deep learning, deep learning technology has been widely used in visual fields such as video classification, identity recognition, and autonomous driving. However, in these fields, data scarcity is often faced. At the same time, it is also difficult to collect some data, and it is time-consuming and laborious to label the collected data. Humans can learn new concepts from a single or a few samples, and can obtain extremely rich representations from sparse data, and can further enhance future learning capabilities through previous learning experience. How to enable machines to have this ability is crucial for artificial intelligence. However, most of this field currently directly extracts feature vectors from feature extraction networks for measurement. The disadvantage of this type of device is that it does not fully consider the connection between each category in each task, and then extracts a more representative and discriminative feature vector that best suits this task. For example, the representative feature vectors extracted from cats and tigers and cats and dogs are different. Summary of the invention

[0004] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the present invention aims to provide a method, system, device and medium for classifying small sample images in a relational network.

[0005] The technical solution adopted by the present invention is:

[0006] A method for classifying small sample images using a relational network comprises the following steps:

[0007] Divide the data set. In each training batch, divide the data set into a training set and a test set. The samples in the training set are marked with sample labels.

[0008] An embedding module is used to extract the image feature representation of the images in the dataset;

[0009] Since the image feature representation extracted by the embedding module is task-independent, the self-attention module is used to learn the contextual information between tasks and transform the task-independent image feature representation into the task-related image feature representation;

[0010] According to the task-related image feature representation, the neural network is trained using the relational network metric based on the original learning, and the trained neural network is used for image classification.

[0011] Furthermore, the data set is divided into a training set and a test set in each training batch, including:

[0012] Each batch corresponds to a task. Each batch divides the data set into a training set and a test set. The support set in the training set is input into the embedding module to obtain the image feature representation. The query set in the training set is labeled and is responsible for measuring the distance with the samples in the support set.

[0013] The support set and query set in the training set are responsible for continuously updating the parameters of the network model.

[0014] Furthermore, the embedding module is implemented by a residual network, the residual network includes 4 basic modules connected in sequence, each basic module includes three convolutional small modules, a 2×2 maximum pooling layer is attached to the output end of each basic module, and a 2×2 average pooling layer is attached to the output end of the last maximum pooling layer;

[0015] Support set D support and query set D query The image data in the embedding module f θ In the example, we obtain the corresponding image feature representation R p , where p = c × h × w, c, h, and w represent the number of channels, height, and width of the feature representation, respectively.

[0016] Furthermore, the steps of multi-classification learning are also included:

[0017] definition n represents a category, and z represents a picture in that category;

[0018] Calculate the center point of the feature representation of each category, that is, add up the feature representations of the same category of pictures and take the average, expressed as:

[0019]

[0020] If there is only one sample per category, the image feature representation of the sample is used as the class center feature representation;

[0021] Define the class vector C of each batch = [c1, c2, c3, ..., c N ](C∈R N×c×h×w ), input the class vector C into the self-attention module f β Before, change the shape of the class vector C, that is, C∈R N×l×d ,in, d is a hyperparameter of the hidden layer and N is the number of categories.

[0022] Furthermore, the self-attention module is used to learn contextual information between tasks, and the task-independent image feature representation is converted into the task-related image feature representation, including:

[0023] Multiply the image feature representation output by the embedding module with the three preset matrices to be trained to obtain the query vector d q , key vector d k Sum value vector d v ;

[0024] According to the query vector d q and the key vector d k Get the attention score matrix M H , the value vector d v Multiply by the attention score matrix M H After that, a softmax operation is performed to obtain a more discriminative task-related image feature representation C′.

[0025] Furthermore, in order to speed up the calculation, the matrix form is used instead of the vector form for calculation. The formula for converting the vector form into the matrix form is:

[0026]

[0027]

[0028]

[0029] Where Q represents the query matrix, K represents the key matrix, V represents the value matrix, C represents the support set embedding, and W Q represents the query linear mapping matrix, W K represents the key linear mapping matrix, W V Represents the value linear mapping matrix, Indicates the input and output dimensions, the input dimension is d, and the output dimension is d model , d model is the dimension of the hidden layer;

[0030] The definition formula of the task-related image feature representation C′ is:

[0031]

[0032] In the formula, T Represents the transpose of the key matrix. Attn(Q,K,V) represents the calculation of the self-attention module.

[0033] Furthermore, the training of the neural network in a relational network metric based on the original learning according to the task-related image feature representation includes:

[0034] The distance between samples is measured by using the relational network metric. From the perspective of metric, the similarity of samples is constrained by splicing the task-related image feature representation with the samples in the query set.

[0035] The measurement formula is:

[0036] Loss(s m,n ,y m )=(s m,n -1(n==y m )) 2

[0037] In the formula, y m is the standard label of the sample in the query set, n is the category of the query set, s m,n Represents the similarity score between the task-related feature vector and the query feature vector.

[0038] Another technical solution adopted by the present invention is:

[0039] A relational network small sample image classification method system, comprising:

[0040] The data partitioning module is used to partition the data set. In each training batch, the data set is divided into a training set and a test set. The samples in the training set are annotated with sample labels.

[0041] A feature extraction module, used to extract image feature representations of images in the dataset using an embedding module;

[0042] The feature conversion module is used to convert the image feature representation that is irrelevant to the task into the image feature representation that is relevant to the task by using the self-attention module to learn the context information between tasks.

[0043] The network training module is used to train the neural network according to the task-related image feature representation using the original learned relational network metric method, and use the trained neural network for image classification.

[0044] Another technical solution adopted by the present invention is:

[0045] A relational network small sample image classification device, comprising:

[0046] at least one processor;

[0047] at least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0049] Another technical solution adopted by the present invention is:

[0050] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to execute the method described above when executed by the processor.

[0051] The beneficial effect of the present invention is that the present invention transforms task-independent feature vectors into task-related feature vectors through the attention mechanism, has the advantages of considering task context information and reducing the intra-class distance, and is more conducive to the subsequent measurement and calculation of similarity scores. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 is a flowchart of the steps of a method for classifying small sample images of a relational network in an embodiment of the present invention;

[0054] Figure 2 It is a structural diagram of a method for classifying small sample images using a relational network in an embodiment of the present invention;

[0055] Figure 3 is a structural diagram of an embedded module in an embodiment of the present invention;

[0056] Figure 4 is a schematic diagram of the structure of the attention module in an embodiment of the present invention;

[0057] Figure 5 It is a schematic diagram of the visualization effect of each layer in the residual network resnet-12 in an embodiment of the present invention;

[0058] Figure 6It is a quantitative visualization diagram of the self-attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0061] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0062] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0063] like Figure 1 and Figure 2 As shown, this embodiment provides a relationship network small sample image classification method, including the following steps:

[0064] S1. Divide the data set. In each training batch, divide the data set into a training set and a test set. The samples in the training set are marked with sample labels.

[0065] Divide the mini-ImageNet dataset: In each training batch, the mini-ImageNet dataset will be divided into a training set and a test set. The training set and the test set will be further divided into a support set and a query set. The samples in the training set will be labeled with sample labels.

[0066] Specifically, each batch (episode) will have a task, and each batch will divide the mini-ImageNet dataset into a support set and a query set. The support set is responsible for inputting the model to obtain feature representation, while the query set in the training set also has labels, which are responsible for measuring the distance between samples in the support set. The principle is that samples of the same type should be closer, and samples of different types should be farther and farther away. The support set and query set of the training set are responsible for continuously updating the parameters of the network model. Assuming that there are C categories of small samples to be classified, and each category has Z samples, we call it C-way-Z-key small sample classification learning. Specifically: randomly select C categories from the mini-ImageNet categories, and then randomly select Z samples from these C categories. The task is called T support Many tasks constitute the support set D support The query set samples are composed of the remaining data samples in the C categories, called D query Dataset D support and dataset D query The data intersection is empty and does not interfere with each other.

[0067] S2. Use the embedding module to extract the image feature representation of the images in the dataset.

[0068] In this embodiment, the residual network resnet-12 is used as the embedding module, such as Figure 3 As shown. The residual network embedding module contains 4 basic modules, each of which contains three convolutional modules. A 2×2 maximum pooling layer is added after each basic module. In particular, a 2×2 average pooling layer is added after the last maximum pooling layer. Figure 5 , Figure 5 It is a visualization diagram of the output of the four basic modules.

[0069] The traditional small sample classification learning method is adopted, which will support set D support and query set D query The data is passed into the embedding module f θ To obtain the corresponding image feature representation R p , p = c × h × w, where the letters c, h, and w represent the number of channels, height, and width of the feature representation, respectively. We define n represents a category, and z represents an image in that category.

[0070] Furthermore, in multi-classification learning, we will calculate the center point of the feature representation of each category in advance, that is, add up the feature representations of the same category of images and take the average, expressed as:

[0071]

[0072] If there is only one sample per category, let the image feature representation of that sample serve as the class center feature representation. Then we define the class vector C = [c1, c2, c3, ..., c N ](C∈R N×c×h×w ), and then input the class vector C into the self-attention module f β Before, the shape of the class vector C is changed, that is, C∈R N×l×d .first d is a hyperparameter of the hidden layer.

[0073] S3. Since the image feature representation extracted by the embedding module is task-independent, the self-attention module is used to learn the contextual information between tasks and transform the task-independent image feature representation into task-related image feature representation.

[0074] The attention mechanism mimics the internal process of biological observation behavior, that is, a mechanism that aligns internal experience and external sensations to increase the observation precision of some areas. The attention mechanism can quickly extract important features of sparse data and is therefore widely used in natural language processing tasks, especially machine translation.

[0075] The self-attention mechanism in this embodiment is an improvement of the attention mechanism, which reduces the dependence on external information and is better at capturing the internal correlation of data or features. This embodiment uses self-attention in image classification, so that the extracted feature representation has more unique recognition and representation in each task. We regard the attention mechanism as a component, the purpose of which is to learn more adaptive and discriminative task-related embeddings from extracted irrelevant features. Figure 6 It is a quantitative visualization diagram of the self-attention mechanism. Figure 6 It can be seen that the self-attention mechanism can reduce the distance between samples within a class, expand the distance between classes, and learn a feature vector related to the task. Figure 6 In each figure, the image shows the input to the self-attention module f β The PCA predictions before and after support the position of the embedding; the values ​​in the figure are the support set embeddings entering the self-attention module f β 5-way-5-shot classification accuracy before and after.

[0076] like Figure 4 As shown in Figure 1, the self-attention mechanism works as follows: three vectors are created, called query vector, key vector, and value vector to calculate self-attention. These three vectors are obtained by multiplying the irrelevant image feature representation extracted in the first module with the three matrices to be trained. We label these three matrix vectors d accordingly. q d k and d v According to experience, d q dk and d v In normal cases, it is smaller than the original irrelevant image feature representation. By implementing the inner product between the query and the key of each class, we can get the initial attention score matrix M representing the similarity between them. H In order to make the attention score matrix M H With a more stable gradient, the attention score matrix M H Divide by a value It can also be any other possible value. After multiplying each value vector by the final attention score matrix, we finally perform a softmax operation to obtain a more discriminative task-related image feature representation C′. In actual implementation, in order to speed up the calculation, we use matrix form instead of vector form for calculation. The formula for converting vector form to matrix form is:

[0077]

[0078]

[0079]

[0080] Where, d model is the dimension of the hidden layer. The definition formula of the task-related image feature representation C′ is:

[0081]

[0082] S4. According to the task-related image feature representation, the neural network is trained using the relational network measurement method based on the original learning, and the trained neural network is used for image classification.

[0083] This embodiment uses the metric method of the relational network to measure the distance between samples. From the perspective of measurement, we let the task-related image feature representation and the query set sample splicing method constrain the similarity of the judgment sample. The main method is to train a network to learn (such as a convolutional neural network) the distance metric method, so that the classification of small samples is more reasonable and generalizable. We use the standard deviation MSE loss function, which is suitable for updating the parameters of stochastic gradient descent in each batch. The specific measurement formula is:

[0084] Loss(s m,n ,y m )=(s m,n -1(n==y m )) 2

[0085] In the formula, y m is the standard label of the query sample, and n is the category of this query set.

[0086] In summary, the method of this embodiment has the following advantages and beneficial effects compared with the prior art:

[0087] (1) The method of this embodiment is based on the attention mechanism, which converts the task-independent feature vector generated in the prior art into a task-related feature vector. It fully considers the commonality between samples in the class and learns a feature vector that is representative and discriminative between categories. The generated feature vector is based on the task level, which can make the distance between samples of the same class close and the distance between samples of different classes far, which is more conducive to the subsequent measurement and calculation of similarity scores.

[0088] (2) The method in this embodiment has the highest accuracy in small-sample classification learning of the mini-imagenet dataset. The accuracy reaches 73.69 in the setting of 5 categories and 5 samples per category, and the accuracy reaches 61.57 in the setting of 5 categories and 1 sample per category, which is far higher than other competitive small-sample classification learning models. At the same time, our model also has the highest classification accuracy among all attention-based models.

[0089] (3) The number of parameters required to be calculated in the architecture of the method of this embodiment is very small, because the self-attention mechanism will reduce the dimension of the feature vector in advance when performing operations, thereby greatly improving the operation speed.

[0090] The present invention also provides a relational network small sample image classification method system, comprising:

[0091] The data partitioning module is used to partition the data set. In each training batch, the data set is divided into a training set and a test set. The samples in the training set are annotated with sample labels.

[0092] A feature extraction module, used to extract image feature representations of images in the dataset using an embedding module;

[0093] The feature conversion module is used to convert the image feature representation that is irrelevant to the task into the image feature representation that is relevant to the task by using the self-attention module to learn the context information between tasks.

[0094] The network training module is used to train the neural network according to the task-related image feature representation using the original learned relational network metric method, and use the trained neural network for image classification.

[0095] A relational network small sample image classification system of this embodiment can execute a relational network small sample image classification method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0096] The present invention also provides a relational network small sample image classification device, comprising:

[0097] at least one processor;

[0098] at least one memory for storing at least one program;

[0099] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0100] A relational network small sample image classification device of this embodiment can execute a relational network small sample image classification method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0101] The present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 1 The method shown.

[0102] This embodiment also provides a storage medium, which stores instructions or programs that can execute a method for classifying small sample images of a relationship network provided by an embodiment of the method of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.

[0103] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0104] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0105] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0106] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0107] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0108] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0109] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0110] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0111] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A relational network small sample image classification method, characterized in that: The following steps are involved: Divide the data set. In each training batch, divide the data set into a training set and a test set. The samples in the training set are marked with sample labels. An embedding module is used to extract the image feature representation of the images in the dataset; Since the image feature representation extracted by the embedding module is task-independent, the self-attention module is used to learn the contextual information between tasks and transform the task-independent image feature representation into the task-related image feature representation; According to the task-related image feature representation, the neural network is trained using the original learning-based relational network metric, and the trained neural network is used for image classification; The embedding module is implemented by a residual network, which includes four basic modules connected in sequence, each of which includes three convolutional small modules, a 2×2 maximum pooling layer is attached to the output end of each basic module, and a 2×2 average pooling layer is attached to the output end of the last maximum pooling layer; Support set D support and query set D query The image data in the embedding module f θ In the example, we obtain the corresponding image feature representation R p , where p = c × h × w, c, h, and w represent the number of channels, height, and width of the feature representation, respectively; The self-attention module is used to learn the context information between tasks, and the task-independent image feature representation is converted into the task-related image feature representation, including: Multiply the image feature representation output by the embedding module with the three preset matrices to be trained to obtain the query vector d q , key vector d k Sum value vector d v ; According to the query vector d q and key vector d k Get the attention score matrix M H , the value vector d v Multiply by the attention score matrix M H After that, a softmax operation is performed to obtain a more discriminative task-related image feature representation C′; In order to speed up the calculation, the matrix form is used instead of the vector form for calculation. The formula for converting the vector form into the matrix form is: Where Q represents the query matrix, K represents the key matrix, V represents the value matrix, C1 represents the support set embedding, and W Q represents the query linear mapping matrix, W K represents the key linear mapping matrix, W V represents the value linear mapping matrix, Indicates the input and output dimensions, the input dimension is d, and the output dimension is d model , d model is the dimension of the hidden layer; The definition formula of the task-related image feature representation C′ is: In the formula, K T Represents the transpose of the key matrix.

2. The method for classifying small sample images using a relational network according to claim 1, characterized in that: The data set is divided into a training set and a test set in each training batch, including: Each batch corresponds to a task. Each batch divides the data set into a training set and a test set. The support set in the training set is input into the embedding module to obtain the image feature representation. The query set in the training set is labeled and is responsible for measuring the distance with the samples in the support set. The support set and query set in the training set are responsible for continuously updating the parameters of the network model.

3. The method for classifying small sample images using a relational network according to claim 1, characterized in that: It also includes the steps of multi-classification learning: definition n represents a category, and z represents a picture in that category; Calculate the center point of the feature representation of each category, that is, add up the feature representations of the same category of pictures and take the average, expressed as: If there is only one sample per category, the image feature representation of the sample is used as the class center feature representation; Define the class vector C of each batch = [c1, c2, c3, ..., c N ],C∈R N×c×h×w , input the class vector C into the self-attention module f β Before, change the shape of the class vector C, that is, C∈R N×l×d ,in, d is a hyperparameter of the hidden layer and N is the number of categories.

4. A relational network small sample image classification method system, applied to a relational network small sample image classification method according to any one of claims 1 to 3, characterized in that: include: The data partitioning module is used to partition the data set. In each training batch, the data set is divided into a training set and a test set. The samples in the training set are annotated with sample labels. A feature extraction module, used to extract image feature representations of images in the dataset using an embedding module; The feature conversion module is used to convert the image feature representation that is irrelevant to the task into the image feature representation that is relevant to the task by using the self-attention module to learn the context information between tasks. The network training module is used to train the neural network according to the task-related image feature representation using the original learned relational network metric method, and use the trained neural network for image classification.

5. A relational network small sample image classification device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 3.

6. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 3 when executed by the processor.

Images