A visual cognitive driven small sample image classification method, system and medium

Abstract

The application discloses a kind of visual cognitive drive small sample image classification method, system and medium, the method includes pre-training stage and meta-learning stage;The pre-training stage includes: obtaining initial training set, data enhancement is carried out to the initial training set, obtains training dataset;The training dataset is embedded coding, mapping is carried out to embedding by linear layer, pre-trains the classification task of upstream;The meta-learning stage includes: the training dataset is divided into support set and query set;Image of the support set and the query set is mapped to embedding space, obtains corresponding embedding vector representation;The similarity of embedding in high-dimensional space is evaluated by relationship network, and the classification result of small sample learning output is obtained: the application can solve the problem that embedding space cannot be correctly perceived and the relationship between support set prototype and query set embedding is not accurate in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, in particular, the present application is applied to the field of SAR image classification, and particularly relates to a visual cognitive driven sample and embedded adaptive network for SAR image classification. BACKGROUND

[0002] With the development of small sample learning, more and more work has begun to explore the possibility of small sample learning applied to SAR images; it has small training overhead, less data demand and autonomous learning ability under specific tasks, and is applied and popularized in SAR image classification tasks.

[0003] For example, Wang et al. use neural networks to obtain the embedding of SAR images, then combine it with the non-parametric flexibility of Gaussian processes, and finally use a series of kernel functions to measure the similarity between embeddings; Rostami et al. trained a shared domain constant embedding for cross-domain knowledge transfer in SAR long-term environmental monitoring tasks; Zhang et al. used a deep dual-flow network based on domain knowledge to learn specific features of SAR images in training; Lu et al. use a triplet loss function to train a network model with less labeled data; Wang et al. use a convolutional bidirectional long short-term memory (Conv BiLSTM) network to better realize the expression of images to embedding space; Cai et al. propose an improved prototype network (PN) based on spatial transformation, which realizes feature alignment through a spatial transformer module, thereby utilizing more semantic information compared to pixel alignment.

[0004] The above researches are all exploring how to better express the embedding; on the other hand, the research is to improve the performance of the model by exploring how to evaluate the existing embedding; they use graph neural networks in small sample tasks to mine the relationship between samples and samples; for example, Yang et al. use a graph attention network for classification after obtaining the embedding expression of neural networks; they introduce a multi-shot learning method based on relational networks and graph neural networks; Yang et al. aims to learn the feature similarity between samples and support samples by using the non-Euclidean structure characteristics of graphs; Chen et al. use attention mechanism to highlight class-specific features while filtering irrelevant background information, and use graph convolution network in the downstream to mine the relationship between query set and support set; Li et al. construct the obtained embedding vector into a fully connected graph, and use graph neural network neighborhood aggregation to update the graph, thereby obtaining a new representation of nodes and their relationships, and finally perform classification; the above methods improve the performance of SAR image small sample learning, but do not consider the high-dimensional embedding space.

[0005] In SAR small sample learning, according to the type of enhanced data, data enhancement can be divided into upstream data space enhancement and downstream feature space enhancement; the gray image in SAR image reflects the radar reflection intensity received by SAR under a certain polarization mode, so the past research focuses on the processing of image noise; for example, Gishkori et al. proposed a knowledge transfer learning method, which uses camera images to improve (SAR) images in manifold domain; Yang et al. proposed to use power transformation to enhance small pixel values and suppress large pixel values; Wang et al. set a transformation set, and perform the specified transformation operation in the transformation set by randomly sampling part of the data; Ding et al. use Translation, Speckle Noising, Pose Synthesis three methods for data augmentation operation; Gao et al. propose to use GAN to generate data augmentation data set; Song et al. use an adversarial autoencoder to increase the distribution density and the size of the training data set.

[0006] The above data enhancement methods are in the category of data space; through the upstream encoder, the picture is converted to the embedding space; in the embedding space, the data enhancement method has deviated from specific data such as images, texts, and sounds, and the data enhancement problem in the embedding space has become how to maximize the perception of the corresponding class embedding space and find other embedding points in the embedding space as training data; Therefore, when discussing the enhancement work in the embedding space, the work in other fields (such as NLP) is introduced; for example, Sun et al. proposed to use the idea of Welzl in the embedding space, use the embedding of known samples to construct a minimum closed ball for each class, and regard the closed ball as the approximation of the corresponding class embedding space, and then perform data augmentation in the embedding space; Wang and Lillis proposed to use the synthetic minority over-sampling technique (SMOTE) for data enhancement, and the SMOTE interpolation method can smooth the boundary, which means that the algorithm can better generalize and has less overfitting relative to the training data; The methods after that improve the small sample classification performance through data enhancement, but they only focus on the overall features of SAR images and ignore the influence of background noise; In addition, the relationship between pixel value and radar reflection intensity is not considered, that is, the relationship between SAR gray value and pixel importance.

[0007] It can be understood from the above introduction that the existing method ignores the high-dimensional embedding space; embedding as the unique identifier of data in space, it determines the accuracy of downstream tasks, different support set sampling methods and collection quantity will greatly affect the embedding space distribution expressed by the model, affect the accuracy of sample task and the robustness of the model; in the case of sample imbalance, a small part of the embedding cannot be correctly expressed in the ideal state; these embeddings are often represented on the boundary between different categories, or worse, the embedding is represented in the wrong class; especially in the condition setting of small sample learning, there are two problems: (1) the encoder cannot get enough data to correctly perceive the embedding space; (2) the shot quantity will affect the calculation of the prototype downstream, resulting in inaccurate relationship between the support set prototype and the query set embedding. SUMMARY

[0008] The purpose of the present application is to solve the problem that the existing technology cannot correctly perceive the embedding space and the relationship between the support set prototype and the query set embedding is inaccurate.

[0009] To solve the above technical problems, the specific technical solutions of the present application are as follows:

[0010] On the one hand, the present application provides a visual cognitive driven small sample image classification method, comprising: a pre-training stage and a meta-learning stage.

[0011] The pre-training stage comprises:

[0012] Obtain an initial training set, perform data augmentation on the initial training set, and obtain a training data set;

[0013] Embedding the training data set, mapping the embedding through a linear layer, and pre-training the upstream classification task;

[0014] The meta-learning stage comprises:

[0015] Divide the training data set into a support set and a query set;

[0016] Map the images of the support set and the query set to the embedding space to obtain the corresponding embedding vector representation;

[0017] Evaluate the embedding similarity in the high-dimensional space through the relationship network to obtain the classification result of small sample learning output.

[0018] As an improved scheme, the data augmentation on the initial training set further comprises: using a target domain center adaptive enhancement algorithm to enhance the initial training set.

[0019] As an improved scheme, the obtaining the initial training set, the data augmentation on the initial training set to obtain the training data set further comprises:

[0020] Obtaining a grayscale pixel matrix of the picture, and calculating an image center coordinate based on the grayscale pixel matrix;

[0021] Calculating an enhancement factor of each pixel according to the image center coordinate, performing standardization on the enhancement factor to obtain an enhancement factor matrix, and obtaining a distance matrix based on the enhancement factor matrix;

[0022] Point-multiplying the grayscale pixel matrix and the distance matrix to obtain the training data set.

[0023] As an improved scheme, the embedding coding of the training data set further comprises: embedding coding of the training data set through an encoder based on a ResNet12 structure.

[0024] As an improved scheme, the embedding coding of the training data set, mapping the embedding through a linear layer, pre-training the classification task of the upstream, further comprises:

[0025] Inputting the pictures in the training data set into a convolution block, first passing through the first two layers of the same Conv64(K3, S1, P1) / BN / ReLU, and then passing through a layer of Conv64(K3, S1, P1) / BN to obtain an output h; then outputting an h with the same size as h through a layer of Conv64(K1, S1, P0) / BN identity , adding the two to realize a residual network; passing through a ReLU activation function and MaxPooling(2, 2) to obtain the final output of the convolution block; unfolding the final output of the convolution block to obtain a vector with a dimension of dim, inputting the vector into a linear layer, pre-training the classification task of the upstream, and calculating the loss.

[0026] As an improved scheme, the mapping of the images of the support set and the query set to the embedding space to obtain the corresponding embedding vector representation further comprises:

[0027] Mapping the images of the support set and the query set to the embedding space using the encoder, each image obtaining a corresponding embedding vector representation; for the support set, the embedding of the corresponding class participates in average calculation to obtain an average prototype; the average prototype and the embedding are input into a boundary fuzzy embedding correction module, and the corrected query set embedding is output.

[0028] As an improved scheme, the embedding of the corresponding class participates in average calculation to obtain an average prototype; the average prototype and the embedding are input into a boundary fuzzy embedding correction module, and the corrected query set embedding is output, which further comprises:

[0029] average embedding of all classes in the support set is calculated, and the average embedding is spliced to obtain an average prototype matrix;

[0030] query set embedding is obtained, and the query set embedding is spliced to obtain a query set embedding matrix;

[0031] The average prototype matrix and the query set embedding matrix are spliced to obtain an input matrix of the attention module;

[0032] An updated embedding matrix is obtained according to the self-attention module and the input matrix.

[0033] As an improved scheme, the method further comprises the following steps:

[0034] The updated query set and the pre-updated support set are obtained, and the similarity vector of the query set and the support set embedding is obtained through the calculation of the relationship network module;

[0035] A loss value is obtained by loss calculation through the loss function and the similarity vector;

[0036] The parameters of the encoder, the boundary fuzzy embedding correction module and the relationship network module are integrated to obtain a model parameter set;

[0037] Based on the loss value, the model parameter set is updated in the direction of the back propagation gradient descent.

[0038] In another aspect, the present application also provides a visual cognitive driven small sample image classification system, comprising:

[0039] a pre-training module and a meta-learning module;

[0040] The pre-training module is configured to obtain an initial training set, perform data augmentation on the initial training set to obtain a training data set, embed the training data set, map the embedding through a linear layer, and pre-train an upstream classification task.

[0041] The meta-learning module is configured to divide the training data set into a support set and a query set, map images of the support set and the query set to embedding space to obtain corresponding embedding vector representations, evaluate embedding similarity in high-dimensional space through a relationship network, and obtain a classification result output by small sample learning.

[0042] In another aspect, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the visual cognitive driven small sample image classification method.

[0043] The beneficial effects of the technical scheme of the present application are:

[0044] 1. The visual cognitive driven small sample image classification method provided by the present application proposes a meta-learning framework: sample embedding adaptive network (Sea-Net), which is suitable for small sample classification tasks of SAR images and has a short training time.

[0045] A target domain center adaptive enhancement algorithm is proposed, which can adaptively calculate the center of the SAR image based on the semantic characteristics of the image, and the parameters do not need to be manually adjusted. Based on this, the Manhattan distance is used to quickly calculate the pixel-level enhancement factor to realize small sample enhancement. In addition, when training the encoder in the upstream, unlike training the classifier, the enhanced input data should be regarded as another category.

[0046] An edge blur embedding correction algorithm based on the self-attention mechanism is proposed, which increases the inter-class distance of different classes to solve the influence of the coherent speckle noise caused by the imaging principle.

[0047] 2. The visual cognitive driven small sample image classification system provided by the present application can realize the visual cognitive driven small sample image classification method provided by the present application through the cooperation of the pre-training module and the meta-learning module.

[0048] 3. The computer readable storage medium provided by the present application can realize the cooperation of the pre-training module and the meta-learning module to realize the visual cognitive driven small sample image classification method provided by the present application, and the computer readable storage medium provided by the present application also effectively improves the operability of the visual cognitive driven small sample image classification method. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the specific embodiments of the present application or the technical scheme in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0050] Figure 1 is the architecture diagram of the sample and the embedding adaptive network model training in the visual cognitive driven small sample image classification method of the present application embodiment 1;

[0051] Figure 2 is the visual result schematic diagram of the sample and the embedding adaptive network model after adding various training sets in the present application embodiment 1. DETAILED DESCRIPTION

[0052] The advantages and features of the present application will become more apparent from the following detailed description of preferred embodiments, given by way of example only, in conjunction with the accompanying drawings, in which:

[0053] In the description of the present application, it should be noted that the embodiments described in the present application are part of the embodiments of the present application, not all the embodiments; based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0054] The terms "include", "comprise" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusion, for example, a process, method, device, product or equipment including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0055] Embodiment 1

[0056] The embodiment provides a visual cognitive driven small sample image classification method, comprising: constructing a sample and an embedding adaptive network model, training the sample and the embedding adaptive network model, and the training architecture of the sample and the embedding adaptive network model is shown in Figure 1 .

[0057] The training of the sample and the embedding adaptive network model comprises a pre-training stage and a meta-learning stage;

[0058] The pre-training stage comprises training an upstream encoder pre-training model, in the training of the upstream encoder pre-training model, an initial training set is obtained, the initial training set is first subjected to data enhancement through a TDCSA module to obtain a training data set;

[0059] The training data set is embedded and coded through an encoder fθ based on a ResNet12 structure; finally, the embedding is mapped through a linear layer, the classification one-hot vector result of the embedding is calculated, and the loss is calculated.

[0060] In the downstream meta-learning phase, first, the training data set is divided into a support set and a query set; the encoder fθ at the end of the upstream training is used to map the images of the support set and the query set to the embedding space, and each image will obtain a corresponding embedding vector representation; for the support set, the embedding of the corresponding class participates in the average calculation to obtain the average prototype; the average prototype and the embedding are used as the input of the boundary fuzzy embedding correction module, and the corrected query set embedding is output; finally, the relationship network is used to evaluate the embedding similarity in the high-dimensional space to obtain the classification result of the small sample learning output.

[0061] As an embodiment of the present application, the initial training set is first subjected to data enhancement through the TDCSA module, which further comprises enhancing the initial training set by using a target domain center adaptive enhancement algorithm; in small sample learning, due to the small amount of data, appropriate data enhancement is often needed to improve the learning ability of the model; SAR images have the characteristics of large speckle noise, unobvious features and fuzzy edges; therefore, in data enhancement, it is hoped that the contrast between the center and the periphery of the enhanced image will be enhanced to enhance the features of the central object of the image; based on the above idea, a target domain center adaptive enhancement algorithm is proposed;

[0062] The light and shade of the SAR grayscale image represents the strength of the radar signal, and the smaller the radar reflection intensity of the darker part of the pixel, the greater the radar reflection intensity of the brighter part of the pixel; and in the natural environment, the brightness of the man-made object is often greater than that of the natural environment, so the calculation method of the target domain center adaptive enhancement algorithm is:

[0063]

[0064]

[0065] In the above formula, C H ,C W represents the coordinates of the center of the image after calculation, x∈R H×W represents the grayscale pixel matrix of the picture, and H, W represent the size of the matrix;

[0066] After obtaining the center of the image, the enhancement factor of each pixel is calculated; since the image is a regular representation of a grayscale matrix, the Manhattan distance is used here to quickly calculate the distance between nodes, and the enhancement factor is greater when the distance is closer to the center, and vice versa; the distance d ij is an element in the Manhattan distance matrix D∈R H×W d ij is calculated by the current position and the image center coordinates:

[0067] d ij =|i-C row |+|j-C col |

[0068] D is normalized again, so that 0≤d ij ≤2, get the enhancement factor matrix

[0069]

[0070] In the above formula, min(·) represents the minimum value of the elements in the matrix, max(·) represents the maximum value of the elements in the matrix; if after enhancement

[0071] The gray pixel matrix x is in R H×W and the distance matrix D is in R H×W Dot product to get the enhanced picture

[0072]

[0073] The setting of the small sample task is based on the embedding of the input corresponding to the encoder calculated in the upstream, and the similarity between the embeddings is calculated in the downstream; because the purpose of data enhancement is not to train a best classifier in the upstream, but to train a best encoder, and the hope is to enhance the correct position of the picture in the embedding space, in order to prevent the enhanced means from becoming noise to affect the space embedding during the downstream training, so for each corresponding class enhanced picture, it is calculated as a new class; similarly, as a processing means for general gray images, the data enhancement stage also introduces the methods of random vertical flip and random horizontal flip.

[0074] As an embodiment of the application, the training data set is embedded and coded through an encoder fθ based on the ResNet12 structure; finally, the embedding is mapped through a linear layer, the classification one-hot vector result of the embedding is calculated, and the loss is calculated, further comprising:

[0075] The picture in the training data is input into the convolution block, first through the first two layers of Conv64(K3, S1, P1) / BN / ReLU, and then through a layer of Conv64(K3, S1, P1) / BN to get the output h. Then output h identity , and the two are added to realize the residual network; finally, a ReLU activation function and MaxPooling(2, 2) are used to get the final output of the convolution block; after passing through 4 convolution blocks, the result is unfolded to get a vector with a dimension of dim, which is input into the linear layer, the upstream classification task is pre-trained, and the loss is calculated.

[0076] ​In order to solve the influence of coherent speckle noise caused by imaging principle, the embodiment adopts an edge blur embedding correction algorithm of self-attention mechanism; specifically includes: the embedding network maps the picture to a certain position in the embedding space, so that the same class embedding is as close as possible, and the different classes are far away from each other. The embedding representation z i of the i-th input picture x i can be calculated as:

[0077] z i =f θ (x i )

[0078] In the above formula, f θ :R H×W →R dim is a mapping function, θ represents a set of parameters that can be trained for the encoder, H, W represents the height and width of the input picture gray matrix, and dim represents the embedding dimension after mapping; the upstream pre-trained embedding network selects a common ResNet12, and the network has a total of 4 convolutional blocks, each block contains a convolutional neural network, batch normalization and an activation function ReLU.

[0079] The convolutional block calculation process is to input the picture x i into the convolutional block, first pass through the first two layers of Conv64(K3, S1, P1) / BN / ReLU, and then pass through a layer of Conv64(K3, S1, P1) / BN to obtain the output h; then output h identity with the same size as h through a layer of Conv64(K1, S1, P0) / BN, add the two to realize the residual network; finally, pass through a ReLU activation function and MaxPooling(2, 2) to obtain the final output of the convolutional block; wherein, Conv64(K3, S1, P1) / BN / ReLU represents a network layer composed of a convolutional neural network layer with an output dimension of 64, a convolution kernel size of 3, a step of 1, and a padding of 1, batch normalization and ReLU activation function; MaxPooling(2, 2) represents a maximum pooling layer with a pooling kernel size of 2 and a step of 2.

[0080] After passing through 4 convolutional blocks, the obtained result is unfolded to obtain a vector with a dimension of dim, and the vector is input into a linear layer to pre-train the upstream classification task.

[0081] In the downstream training of the meta-learning task, the linear layer classifier is deleted to obtain the encoder.

[0082] After the encoder calculates the corresponding embedding, the prototype of the corresponding class in the embedding space needs to be calculated by averaging the embedding of the corresponding class. However, due to the small sample task setting, the shot number of each class is very small, and the calculation of the average is extremely susceptible to outliers, which may cause the prototype calculation to deviate too much from the real prototype. Therefore, a boundary fuzzy embedding correction based on the self-attention mechanism is proposed to maximize the inter-class distance and obtain an accurate expression of the feature.

[0083] The self-attention mechanism has achieved great success in the field of natural language processing, and is widely used in various fields due to its ability to extract global information and parallelize calculations. In this embodiment, the self-attention mechanism is used to focus on global information, which allows the relative position of the query set embedding to be corrected. The prototype of the support set of the corresponding class is assumed to be the ideal position in the embedding space of the corresponding class. It is hoped that similar embeddings will have a greater impact on each other, and dissimilar embeddings will have a smaller impact on each other, i.e., the inter-class distance is increased. Since our assumption is that the prototype of the original support set is not affected.

[0084] For an m-way n-shot task, the prototype of the i-th class support set is The calculation is as follows:

[0085]

[0086] The average embedding of all classes in the support set is concatenated to obtain the average prototype matrix of the support set S

[0087]

[0088] In the above formula, || represents the concatenation operation, and the embedding of a single query set is If there are multiple pictures in the query set, the query set embedding is concatenated to obtain the query set embedding matrix of this time

[0089]

[0090] In the above formula, p is the size of the query set this time. The query set embedding matrix and the support set average embedding matrix are concatenated to obtain the input matrix of the attention module

[0091]

[0092] According to the self-attention module, the input matrix I is projected through three different linear projections to obtain the query, key, and value in the self-attention.

[0093] Q = IW Q , K = IW K , V = IW V

[0094] In the above formula, W Q , W K , W V ∈R dim×(p+m) is a linear projection matrix, then the Q, K, V matrices are input into the self-attention mechanism formula to calculate the updated embedding matrix

[0095]

[0096] In the above formula, is a scaling factor, and multi-head attention is introduced in the self-attention mechanism to improve the generalization ability of the model, and θ represents a set of trainable parameters of the attn module; obtain After that, in order to prevent the influence of the query set embedding on the support set embedding, only the updated query set and the support set before updating are taken to perform the subsequent similarity calculation.

[0097] As an embodiment of the present application, the traditional similarity based on the Euclidean space, such as cosine similarity, L2 distance, etc. Linear method has high requirements for embedding in embedding space, and is prone to errors in high-dimensional space; in order to enable the model to learn how to evaluate the similarity in the high-dimensional embedding space, a trainable model is proposed for similarity evaluation.

[0098] In order to reduce the model parameters, the similarity vectors of the query set and the support set embedding are obtained through the calculation of the relationship network

[0099]

[0100] In the above formula, represents the predicted similarity of each sample of the query set to the m support set prototypes, and this similarity will be used for loss calculation in the next section, and θ represents a set of trainable parameters in the relationship network.

[0101] The commonly used cross-entropy is selected as the loss function, and the loss value is calculated as follows:

[0102]

[0103] In the above formula, y represents the real label vector, which is represented using one-hot code;

[0104] In the model, the trainable parameters include the encoder parameters θ Encoder in pre-training Attn , the embedding correction module parameters θ Relation in meta-learning Encoder , and the relationship network module parameters θ Attn Then the parameter set of the entire model is represented as Θ = {θ Relation , θ new , θ old}

[0105] The optimizer selects SGD, and in the training phase, the model parameters are updated in the direction of the back propagation gradient descent:

[0106]

[0107] In the above formula, Θ new represents the updated parameter set, Θ old represents the parameter set before updating, lr represents the learning rate, represents the partial derivative of the parameter set before updating.

[0108] Embodiment 2

[0109] This embodiment is based on the same inventive concept as the visual cognitive driven small sample image classification method described in embodiment 1, and provides a visual cognitive driven small sample image classification system, comprising a pre-training module and a meta-learning module.

[0110] The pre-training module is configured to obtain an initial training set, perform data augmentation on the initial training set to obtain a training data set, embed the training data set, map the embedding through a linear layer, and pre-train the upstream classification task.

[0111] The meta-learning module is configured to divide the training data set into a support set and a query set, map the images of the support set and the query set to an embedding space to obtain corresponding embedding vector representations, evaluate the embedding similarity in the high-dimensional space through a relationship network, and obtain the classification result of the small sample learning output.

[0112] Embodiment 3

[0113] This embodiment provides a computer readable storage medium, comprising:

[0114] The storage medium is used to store computer software instructions for implementing the visual cognition driven small sample image classification method described in Embodiment 1, and contains an executable program for executing the program set for the visual cognition driven small sample image classification method. Specifically, the executable program can be built into the visual cognition driven small sample image classification system described in Embodiment 2, so that the visual cognition driven small sample image classification system can implement the visual cognition driven small sample image classification method described in Embodiment 1 by executing the built-in executable program.

[0115] In addition, the computer readable storage medium of the embodiment can adopt any combination of one or more readable storage media, wherein the readable storage medium includes an electrical, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any combination thereof.

[0116] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the foregoing description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0118] In several embodiments provided herein, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other forms of connection.

[0119] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme herein.

[0120] In addition, each functional unit in each embodiment herein can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0121] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions herein, essentially or the part of the prior art that contributes to the technical solutions, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment herein. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0122] The above is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A visual cognitive driven small sample image classification method applied to training of samples and embedded adaptive network model, characterized in that, The sample and the training of the embedded adaptive network model comprise a pre-training stage and a meta-learning stage; The pre-training stage comprises: An initial training set is obtained, and data augmentation is performed on the initial training set to obtain a training data set; Further comprising, obtaining a gray pixel matrix of the picture, calculating the image center coordinates based on the gray pixel matrix; calculating the enhancement factor of each pixel according to the image center coordinates, performing standardization on the enhancement factor to obtain an enhancement factor matrix, and obtaining a distance matrix based on the enhancement factor matrix; and multiplying the gray pixel matrix and the distance matrix to obtain the training data set; The training data set is embedded and coded, the embedding is mapped through a linear layer, and the classification task of the upstream is pre-trained; The meta-learning stage comprises: The training data set is divided into a support set and a query set; The images of the support set and the query set are mapped to an embedding space to obtain corresponding embedding vector representations; Further comprising, calculating the average embedding of all classes in the support set, performing a splicing operation on the average embedding to obtain an average prototype matrix; obtaining a query set embedding, splicing the query set embedding to obtain a query set embedding matrix; splicing the average prototype matrix and the query set embedding matrix to obtain an input matrix of an attention module; and obtaining an updated embedding matrix according to the self-attention module and the input matrix; The similarity of the embedding in the high-dimensional space is evaluated through a relationship network to obtain a classification result of small sample learning output.

2. The visual cognitive driven small sample image classification method according to claim 1, characterized in that, The data augmentation of the initial training set further comprises: performing augmentation on the initial training set by using a target domain center adaptive augmentation algorithm.

3. The visual cognitive driven small sample image classification method according to claim 1, characterized in that, The embedding coding of the training data set further comprises: embedding coding of the training data set through an encoder based on a ResNet12 structure.

4. The visual cognitive driven small sample image classification method according to claim 3, characterized in that, The embedding coding of the training data set, the mapping of the embedding through a linear layer, and the pre-training of the classification task of the upstream further comprise: The picture in the training data set is input into a convolution block, first through the first two layers of the same Conv64(K3, S1, P1) / BN / ReLU, and then through a layer of Conv64(K3, S1, P1) / BN to obtain an output h; then through a layer of Conv64(K1, S1, P0) / BN, an h with the same size as h is output identity The two are added to realize a residual network; after a ReLU activation function and MaxPooling(2, 2), the final output of the convolution block is obtained; the final output of the convolution block is unfolded to obtain a vector with a dimension of dim, and the vector is input into a linear layer to pre-train the upstream classification task.

5. The visual cognitive driven small sample image classification method according to claim 4, characterized in that, The mapping of the images of the support set and the query set to the embedding space to obtain corresponding embedding vector representations further comprises: The images of the support set and the query set are mapped to the embedding space by using the encoder, and each image obtains a corresponding embedding vector representation; for the support set, the embedding of the corresponding class participates in average calculation to obtain an average prototype; and the average prototype and the embedding are input into a boundary fuzzy embedding correction module, and the output is a corrected query set embedding.

6. The visual cognitive driven small sample image classification method according to claim 5, characterized in that, The similarity of the embedding in the high-dimensional space is evaluated through a relationship network to obtain a classification result of small sample learning output, which further comprises: The updated query set and the support set before updating are obtained, the similarity vector of the query set embedding and the support set embedding is obtained through calculation of a relationship network module; A loss value is obtained by loss calculation through a loss function and the similarity vector; The parameters of the encoder, the boundary fuzzy embedding correction module and the relationship network module are integrated to obtain a model parameter set; Based on the loss value, the model parameter set is updated in the gradient descent direction by using back propagation.

7. A visual cognitive driven few-shot image classification system, employing the visual cognitive driven few-shot image classification method of claim 1, characterized in that, The system comprises a pre-training module and a meta-learning module; The pre-training module is configured to obtain an initial training set, perform data enhancement on the initial training set to obtain a training data set, embed the training data set, map the embedding through a linear layer, and pre-train a classification task of an upstream; the pre-training module is further configured to obtain a grayscale pixel matrix of a picture, calculate image center coordinates based on the grayscale pixel matrix, calculate an enhancement factor of each pixel according to the image center coordinates, perform standardization on the enhancement factor to obtain an enhancement factor matrix, and obtain a distance matrix based on the enhancement factor matrix; the pre-training module point-multiplies the grayscale pixel matrix and the distance matrix to obtain the training data set; The meta-learning module is configured to divide the training data set into a support set and a query set, map images of the support set and the query set to an embedding space to obtain corresponding embedding vector representations, evaluate embedding similarity in a high-dimensional space through a relation network to obtain a classification result of small sample learning output, and calculate average embedding of all classes in the support set, perform a splicing operation on the average embedding, and obtain an average prototype matrix; the meta-learning module obtains query set embedding, splices the query set embedding to obtain a query set embedding matrix, splices the average prototype matrix and the query set embedding matrix to obtain an input matrix of an attention module, and obtains an updated embedding matrix according to a self-attention module and the input matrix.

8. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the small sample image classification method driven by visual cognition according to any one of claims 1-6.

Images