Small sample image classification method fusing global and adaptive local information

Abstract

The invention provides a small sample image classification method fusing global and adaptive local information, and relates to the technical field of image processing, and the method comprises the steps: S1, obtaining small sample images to establish a data set, and dividing the data set into a support set and a query set; s2, inputting the data set into a global sensing network DCCP-Net for pre-training; s3, inputting the support set and the query set into a task aware network QSA-TAN for meta-training; s4, inputting a pre-training result and a meta-training result into a global-local knowledge collaborative network GLKC for joint training to output probability distribution differences of the small sample images, and classifying the small sample images according to the probability distribution differences to obtain a classification result; according to the method, through collaborative optimization of knowledge in the global data space and the local task space, image characteristics and category information to which the images belong can be better highlighted, generalization of a small sample image classification model is remarkably improved, and therefore the intra-class change problem of the images is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a small sample image classification method that integrates global and adaptive local information. Background Art

[0002] Image classification tasks are becoming increasingly important in daily life and industrial development. Most high-performance image classification methods are based on long-term training of large-scale labeled data. However, in some special scenarios, there is a problem of insufficient available data. For example, in the medical field, relevant information on rare diseases is scarce; in the military field, many data involve confidential information and cannot be used to train models. Since it is difficult for the target model to fully capture rich feature information from limited data and learn effective knowledge with discernment, solving image classification problems in small sample scenarios such as these becomes extremely challenging.

[0003] In recent years, image classification methods for small sample scenarios have developed rapidly, especially methods related to pre-training and meta-learning have received widespread attention from researchers. The core idea of ​​pre-training is to learn the general knowledge of base classes in the global space, and deeply explore the similarities of images of the same category and the differences between different categories. The core goal of meta-learning is to build a model that can quickly learn new tasks, make full use of previously accumulated prior knowledge, and enable deep learning methods to update and adjust according to a small number of new category images to master the ability to learn to learn. In recent years, researchers have recognized the synergy between pre-training and meta-learning, and further proposed a two-stage training strategy that combines the two. Meta-Baseline, a simple meta-learning baseline method that was formally proposed, first trains in the global space to learn a powerful and class-transferable general feature representation, and then applies metric-based meta-learning to adjust model parameters to achieve rapid adaptation on specific tasks. The effectiveness of Meta-Baseline proves that there is a certain trade-off and complementary role between pre-training and meta-learning.

[0004] Although the existing research work has made significant progress and the various methods proposed have shown certain effectiveness in dealing with the small sample problem, there are still some shortcomings. First, these methods perform poorly in dealing with optimization difficulties and intra-class changes in images. Under the condition of limited training data, updating a large number of parameters is prone to overfitting. At the same time, intra-class changes in images make it difficult for the model to accurately capture the core characteristics of the image, and the adaptability of existing methods needs to be improved. Secondly, the pre-training method focuses on exploring the general features of the global space, but ignores the properties related to specific tasks; the meta-learning method emphasizes task-level considerations, but ignores the potential correlation between categories in the global scope. Although the two-stage training strategy combines the advantages of pre-training and meta-learning, such methods only consider the one-way gain from the global data space to the local task space, without taking into account the mutual promotion between the two, especially the impact of the local on the global.

[0005] Based on this, this application proposes a small sample image classification method that integrates global and adaptive local information to solve the above problems. Summary of the invention

[0006] The purpose of the present invention is to provide a small sample image classification method that integrates global and adaptive local information, which can significantly improve the generalization of the small sample image classification model and effectively deal with the problem of intra-class changes in images.

[0007] The technical solution of the present invention is:

[0008] In a first aspect, the present application provides a small sample image classification method that integrates global and adaptive local information, which comprises the following steps:

[0009] S1, obtain a small sample image to build a data set, and divide the data set into a support set and a query set;

[0010] S2, input the data set into the global perception network DCCP-Net for pre-training;

[0011] S3, input the support set and query set into the task-aware network QSA-TAN for meta-training;

[0012] S4. Input the results of pre-training and meta-training into the global-local knowledge collaborative network GLKC for joint training to output the probability distribution difference of small sample images, and classify the small sample images according to the probability distribution difference to obtain the classification results.

[0013] Further, step S2 includes:

[0014] Input the images of the dataset into the Resnet network for feature extraction;

[0015] The classifier is used to calculate the similarity of the image after feature extraction, and normalized to obtain the classification prediction probability of the data set;

[0016] A loss function is constructed based on the classification prediction probability for pre-training; wherein, the expression of the loss function for each task T* includes:

[0017]

[0018] Where, L SL1 represents the loss function of the global perception network DCCP-Net, x i is the input image, y i is the real image label of the image, E is the average loss calculation for all training samples in the task, w is the horizontal dimension of the feature map, h is the vertical dimension of the feature map, p is the row index, q is the column index, σ(·) is the classification prediction probability, is the feature point of the input image, φ θ is a tensor, W is the classifier, and T is the transposed matrix.

[0019] Further, step S3 includes:

[0020] The images of the support set and the images of the query set are subjected to feature extraction through the Resnet network to obtain the support set image features and the query set image features;

[0021] Calculate the cross entropy loss of the support set image features to initialize the classifier;

[0022] Input the initialized classifier and query set image features into the classifier reweighted optimizer CRO to learn the classifier weights to update the classifier weights;

[0023] Use the updated classifier weights to calculate the similarity of the query set image features and output the classification prediction probability of the query set;

[0024] Meta-training is performed using the cross entropy loss of the query set image features and the classification prediction probability as the loss function; the expression of the loss function is:

[0025]

[0026] Where, L SL2 represents the loss function of the task-aware network QSA-TAN, x i is the input image, y i is the true category label of the image, E is the average loss calculation for all training samples in the task, Q is the query set, σ(·) is the classification prediction probability, φ θ is a tensor, w * is the horizontal dimension of the feature map, and T is the transposed matrix.

[0027] Furthermore, in step S4, the above-mentioned calculation formula for inputting the pre-training results and the meta-training results into the global-local knowledge collaborative network GLKC for joint training to output the probability distribution difference of the small sample image includes:

[0028]

[0029] Where D KL (p QSA ||p DCCP ) is p QSA and p DCCP Probability distribution difference, D KL (p DCCP ||p QSA ) is p DCCP and p QSA The probability distribution difference, p DCCP and p QSA They represent the classification prediction outputs of the global perception network DCCP-Net and the task perception network QSA-TAN, respectively. i is the input image, and N is the total number of samples.

[0030] Furthermore, in step S4, the process of classifying the small sample images according to the probability distribution difference includes:

[0031] The final loss function of the global perception network DCCP-Net and the task perception network QSA-TAN is calculated based on the probability distribution difference;

[0032] A small sample image classification model is constructed based on the final loss function, and the small sample images are classified by the small sample image classification model to obtain the classification results.

[0033] Furthermore, the final loss function of the above global perception network DCCP-Net and task perception network QSA-TAN is expressed as:

[0034] Loss DCCP =λ1L SL1 +λ2D KL (p QSA ||p DCCP )

[0035] Loss QSA =λ3L SL2 +λ4D KL (p DCCP ||p QSA )

[0036] In the formula, Loss DCCP is the final loss function of the global perception network DCCP-Net, LossQSA is the final loss function of the task-aware network QSA-TAN, D KL (p QSA ||p DCCP ) is p QSA and p DCCP Difference in probability distribution, L SL1 represents the loss function of the global perception network DCCP-Net, L SL2 represents the loss function of the task-aware network QSA-TAN, D KL (p DCCP ||p QSA ) is p DCCP and p QSA The probability distribution difference of , λ1, λ2, λ3 and λ4 are all hyper parameters.

[0037] In a second aspect, the present application provides an electronic device, including:

[0038] A memory for storing one or more programs;

[0039] processor;

[0040] When the one or more programs are executed by the processor, a small sample image classification method integrating global and adaptive local information as described in any one of the first aspects is implemented.

[0041] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a small sample image classification method that integrates global and adaptive local information as described in any one of the first aspects above.

[0042] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0043] (1) The present invention provides a small sample image classification method that integrates global and adaptive local information. It considers the bidirectional knowledge enhancement effect of global and local feature spaces, proposes a global-local knowledge collaboration module GLKC, combines the global perception network DCCP-Net based on dense classification and category agent and the task perception network QSA-TAN based on query set adaptation, calibrates the model training deviation through feature distribution consistency, realizes the collaborative optimization of knowledge in the global data space and the local task space, completes implicit bidirectional knowledge enhancement, and enables the two networks to not only play their respective advantages, but also influence each other and learn together, and combine the features learned by the two networks, thereby significantly improving the generalization of the model;

[0044] (2) In the training of QSA-TAN, the present invention innovatively proposes to perform meta-learning only on the classifier to simplify the task and alleviate the overfitting problem of the model, and designs a classifier reweighted optimizer CRO. At the same time, a multi-head attention mechanism is introduced to capture the key area features of new category images. The optimized classifier can adaptively process each input query sample to better highlight the image characteristics and category information, thereby effectively dealing with the problem of intra-class changes in images. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A step diagram of a small sample image classification method that integrates global and adaptive local information according to the present invention;

[0047] Figure 2 This is an overall framework diagram of a small sample image classification method that integrates global and adaptive local information in the present invention;

[0048] Figure 3 This is the training flow chart of the DCCP-Net network in the pre-training stage;

[0049] Figure 4 This is the training flow chart of the QSA-TAN network in the meta-learning stage;

[0050] Figure 5 This is the training flow chart of the GLKC network in the joint training phase;

[0051] Figure 6 The figure is a schematic structural block diagram of an electronic device according to an embodiment of the present invention.

[0052] Icon: 101, memory; 102, processor; 103, communication interface. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0055] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0056] It should be noted that, in this article, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0057] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0058] Example 1

[0059] See also Figure 1 , Figure 1 The figure shows a step diagram of a small sample image classification method that integrates global and adaptive local information provided by an embodiment of the present application.

[0060] In a first aspect, the present application provides a small sample image classification method that integrates global and adaptive local information, which comprises the following steps:

[0061] S1, obtain a small sample image to build a data set, and divide the data set into a support set and a query set;

[0062] S2, input the data set into the global perception network DCCP-Net for pre-training;

[0063] S3, input the support set and query set into the task-aware network QSA-TAN for meta-training;

[0064] S4. Input the results of pre-training and meta-training into the global-local knowledge collaborative network GLKC for joint training to output the probability distribution difference of small sample images, and classify the small sample images according to the probability distribution difference to obtain the classification results.

[0065] It should be noted that the present invention is mainly divided into three stages: Figure 2 As shown in the figure, in the pre-training stage, the global perception network DCCP-Net is used to deeply learn the information of the entire base class data space, so as to fully explore the associations between different categories in order to discover the uniqueness between classes. At the same time, through dense classification and category proxy, the network can better understand the differences between categories, improve the perception of different categories, and provide more universal feature representations for subsequent tasks; in the meta-learning stage, the classifier is meta-learned mainly through the task perception network QSA-TAN, and a classifier reweighted optimizer CRO based on the multi-head attention mechanism is designed to adapt the classifier to the query set images in each task; in the joint training stage, the positive influence and interaction between the global data space and the local task space are fully considered. Through the global-local knowledge collaboration GLKC, from the perspective of feature distribution consistency, the output category posterior probabilities of DCCP-Net and QSA-TAN are made to converge to achieve implicit bidirectional knowledge enhancement.

[0066] As a preferred implementation, step S2 includes:

[0067] Input the images of the dataset into the Resnet network for feature extraction;

[0068] The classifier is used to calculate the similarity of the image after feature extraction, and normalized to obtain the classification prediction probability of the data set;

[0069] A loss function is constructed according to the classification prediction probability for pre-training; wherein the expression of the loss function includes:

[0070]

[0071] Where, L SL1 represents the loss function of the global perception network DCCP-Net, x i is the input image, y i is the real image label of the image, E is the average loss calculation for all training samples in the task, w is the horizontal dimension of the feature map, h is the vertical dimension of the feature map, p is the row index, q is the column index, σ(·) is the classification prediction probability, is the feature point of the input image, φ θ is a tensor, W is the classifier, and T is the transposed matrix.

[0072] Specifically, Figure 3The figure shows the training process of the DCCP-Net network in the pre-training stage. The process is divided into three steps: image feature extraction, similarity calculation, model training and optimization. First, the image is input into the Resnet network to map it into a high-dimensional space. Since this stage pays special attention to the spatial properties of the input image, the embedded representation of the input image after feature extraction is a tensor. Secondly, the image after feature extraction will pass through the classifier to calculate the similarity between the feature embedding representation and the category agent. This stage introduces a method based on dense classification, which does not perform dimensional compression on the extracted feature map, but performs classification prediction for the feature vector corresponding to each point in the space. Therefore, the tensor output by the last layer of feature extraction will be split into multiple feature vectors and input into the classifier. The similarity between each feature vector and the category agent is calculated respectively, and then the results are normalized by the softmax function to obtain the classification prediction probability, and finally the predicted category of the image is output. Finally, when training and optimizing the model, inspired by the ternary loss based on neighbor component analysis in the field of metric learning, the category agent is introduced for optimization, and the mean of the feature vectors of samples of the same category is used as the agent of the category, and the loss function of the network is finally constructed.

[0073] As a preferred implementation, step S3 includes:

[0074] The images of the support set and the images of the query set are subjected to feature extraction through the Resnet network to obtain the support set image features and the query set image features;

[0075] Calculate the cross entropy loss of the support set image features to initialize the classifier;

[0076] Input the initialized classifier and query set image features into the classifier reweighted optimizer CRO to learn the classifier weights to update the classifier weights;

[0077] Use the updated classifier weights to calculate the similarity of the query set image features and output the classification prediction probability of the query set;

[0078] The cross entropy loss between the query set image features and the classification prediction probability is used as the loss function for meta-training.

[0079] Specifically, Figure 4 The figure shows the training process of QSA-TAN in the meta-learning stage. The overall training method adopts an internal and external double-loop episodic training method, which is divided into three steps: image feature extraction, classifier weight learning, and classifier training and optimization:

[0080] First, the support set images and query set images in the small sample data set are feature extracted through the Resnet network. The feature representation of the support set images is input into the classifier for initialization by calculating the cross entropy loss. At the same time, the feature extractor in this stage will also share some parameters with the feature extractor in the pre-training stage. These parameters will be frozen in subsequent steps and only serve as weights for classifier initialization, completing the inner loop in the double loop.

[0081] Secondly, the outer loop uses the initialized classifier and the query set image feature representation as the input of the classifier reweighted optimizer CRO. The task-specific knowledge obtained in the inner loop is used in the outer loop to learn the classifier weights. The input of the classifier reweighted optimizer CRO is a triple (Query, Key, Value), where Query is the feature vector of the support set, and Key and Value are the feature vectors of the query set. These vectors will be used as the input of the attention head in the multi-head attention mechanism. The learnable parameters in each attention head will map the input vector to a different vector space and apply it to the multi-head attention mechanism after linear transformation. The multi-head attention mechanism focuses on the key information in the input feature representation, and each attention head will produce an independent output head. i , indicating the degree of attention paid by the attention head to different positions or channels in the input information. Finally, these outputs are integrated to update the weights of the adaptive classifier. The formula is:

[0082]

[0083] In the formula, w * is the updated adaptive classifier weight, w is the horizontal dimension of the feature map, is a linear layer, concat is a connection function, head n is the output of the nth attention head;

[0084] Finally, to train the classifier reweighted optimizer, the cross entropy loss between the true label and the predicted result of the query set image is used as the loss function of the meta-training stage, and its formula is:

[0085]

[0086] Where, L SL2 represents the loss function of the task-aware network QSA-TAN, x i is the input image, y i is the true category label of the image, E is the average loss calculation for all training samples in the task, Q is the query set, σ(·) is the classification prediction probability, φ θ is a tensor, w * is the horizontal dimension of the feature map, and T is the transposed matrix.

[0087] As a preferred implementation, in step S4, the process of classifying small sample images according to probability distribution differences includes:

[0088] The final loss function of the global perception network DCCP-Net and the task perception network QSA-TAN is calculated based on the probability distribution difference;

[0089] A small sample image classification model is constructed based on the final loss function, and the small sample images are classified by the small sample image classification model to obtain the classification results.

[0090] It should be noted that the global perception network DCCP-Net and the task perception network QSA-TAN not only receive their own traditional supervised learning training, but also conduct joint training through the global-local knowledge collaboration module GLKC, achieving mutual guidance and correction. The joint stage is carried out simultaneously with the above two stages, fully considering the interaction between networks, realizing two-way transfer of knowledge, and further improving the classification performance and generalization ability of the small sample graph classification model;

[0091] Specifically, Figure 5 As shown in Figure 1, the working principle of global-local knowledge collaboration in the joint training phase is demonstrated:

[0092] First, based on the two stages mentioned above, the small sample task is simultaneously input into the DCCP-Net and QSA-TAN networks, which learn to predict the true labels of the training samples respectively. The output prediction results will be used as the input of the joint training. At the same time, the two networks will also calculate the corresponding supervision loss L according to their respective training methods. SL1 and L SL2 ,In these two stages, the supervised learning of the two networks ensures the basic accuracy of classification prediction;

[0093] Secondly, in the simultaneous joint training phase, since the feature representations learned in the two phases are different, the class probability estimates will also be different, so the KL divergence is used to measure the matching degree of the predicted probability distribution of the two networks for the input samples, and the probability distribution difference D is calculated. KL (p QSA ||p DCCP ) and D KL (p DCCP ||p QSA ), and its calculation formula is:

[0094]

[0095] Where D KL (p QSA ||p DCCP ) is p QSA and pDCCP Probability distribution difference, D KL (p DCCP ||p QSA ) is p DCCP and p QSA The probability distribution difference, p DCCP and p QSA They represent the classification prediction outputs of the global perception network DCCP-Net and the task perception network QSA-TAN, respectively. i is the input image, N is the total number of samples;

[0096] The final loss function of the global perception network DCCP-Net and the task perception network QSA-TAN is calculated based on the probability distribution difference, and its expression is:

[0097] Loss DCCP =λ1L SL1 +λ2D KL (p QSA ||p DCCP )

[0098] Loss QSA =λ3L SL2 +λ4D KL (p DCCP ||p QSA )

[0099] In the formula, Loss DCCP is the final loss function of the global perception network DCCP-Net, Loss QSA is the final loss function of the task-aware network QSA-TAN, D KL (p QSA ||p DCCP ) is p QSA and p DCCP Difference in probability distribution, L SL1 represents the loss function of the global perception network DCCP-Net, L SL2 represents the loss function of the task-aware network QSA-TAN, D KL (p DCCP ||p QSA ) is p DCCP and p QSA The probability distribution difference, λ1, λ2, λ3 and λ4 are all hyperparameters;

[0100] A small sample image classification model is constructed based on the final loss function, and the small sample images are classified by the small sample image classification model to obtain the classification results.

[0101] Example 2

[0102] In Example 2 of the present invention, an evaluation dataset is used to evaluate the method proposed in the present invention. The evaluation dataset is divided into two categories: coarse-grained datasets and fine-grained datasets. The coarse-grained datasets used include mini-ImageNet and tiered-ImageNet, which are currently widely used in the field of small sample image classification; the fine-grained dataset used is CUB-200-2011. Compared with the coarse-grained dataset, the difference between the categories of the fine-grained dataset is small, and the classification difficulty is higher.

[0103] Under the standard small sample learning setting, the experiment was conducted on three datasets: mini-ImageNet, tiered-ImageNet, and CUB-200-2011, respectively, with two N-way K-shot experiments under 5-way 1-shot and 5-way 5-shot settings. The feature extraction network of the model uses a four-layer residual network ResNet-12, and uses Dropblock as a regularizer, and uses stochastic gradient descent (SGD) to optimize the training process.

[0104] The experimental results of this method on the mini-ImageNet and tiered-ImageNet datasets are shown in Table 1.

[0105] Table 1 Experimental results on mini-ImageNet and tiered-ImageNet datasets

[0106]

[0107]

[0108] On both datasets, this method shows the best performance. In the mini-ImageNet dataset, the classification accuracy of this method is 5.65% higher in the 5-way 1-shot setting and 4.09% higher in the 5-way 5-shot setting than the second-best CAD method. On the tiered-ImageNet dataset, compared with the CAD method, this method is 6.07% higher in the 5-way 1-shot setting and 1.32% higher in the 5-way 5-shot setting.

[0109] The experimental results of this method on the CUB-200-2011 dataset are shown in Table 2.

[0110] Table 2 Experimental results on the CUB-200-2011 dataset

[0111]

[0112] From the above experimental results, it can be seen that this method still maintains the best performance on fine-grained datasets with smaller differences between categories, which successfully verifies that the proposed method can effectively solve the problem of intra-class changes in images.

[0113] In addition, based on traditional image classification, the experiment further verifies the generalization performance of the proposed method in cross-domain scenarios, aiming to explore the impact of domain migration on small sample image classification methods. The scenario selected for this cross-domain experiment is mini-ImageNet→CUB-200-2011, that is, the mini-ImageNet dataset is used as the source domain, and the CUB-200-2011 dataset is used as the target domain. The experimental results are shown in Table 3.

[0114] Table 3 Classification accuracy of methods in mini-ImageNet→CUB-200-2011 cross-domain

[0115]

[0116] In this experiment, the classification accuracy of this method is higher than all the above comparison methods, showing the best performance. The experimental results show that this method has strong robustness in the face of domain changes, and still shows significant effectiveness and superiority in cross-domain small sample scenarios.

[0117] Example 3

[0118] See also Figure 6 , Figure 6 A schematic structural block diagram of an electronic device provided in Example 3 of the present application.

[0119] An electronic device includes a memory 101, a processor 102 and a communication interface 103, wherein the memory 101, the processor 102 and the communication interface 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0120] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0121] The processor 102 may be an integrated circuit chip with signal processing capability. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0122] It is understood that the structure shown in the figure is only for illustration, and a small sample image classification method integrating global and adaptive local information may also include more or fewer components than shown in the figure, or have a different configuration than shown in the figure. Each component shown in the figure may be implemented by hardware, software or a combination thereof.

[0123] In the embodiments provided in the present application, it should be understood that the disclosed method can also be implemented in other ways. The embodiments described above are merely schematic, for example, the flowchart or block diagram in the accompanying drawings shows the possible implementation architecture, functions and operations of the method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0125] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0126] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0127] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A small sample image classification method integrating global and adaptive local information, characterized in that: The following steps are involved: S1, obtain a small sample image to build a data set, and divide the data set into a support set and a query set; S2, input the data set into the global perception network DCCP-Net for pre-training; S3, input the support set and query set into the task-aware network QSA-TAN for meta-training; S4. Input the results of pre-training and meta-training into the global-local knowledge collaborative network GLKC for joint training to output the probability distribution difference of small sample images, and classify the small sample images according to the probability distribution difference to obtain the classification results.

2. A small sample image classification method integrating global and adaptive local information as claimed in claim 1, characterized in that: Step S2 includes: Input the images of the dataset into the Resnet network for feature extraction; The classifier is used to calculate the similarity of the image after feature extraction, and normalized to obtain the classification prediction probability of the data set; Construct loss function for pre-training based on classification prediction probability; for each task T * The expression of the loss function includes: Where, L SL1 represents the loss function of the global perception network DCCP-Net, x i is the input image, y i is the real image label, T * represents the task, E is the average loss calculation for all training samples in the task, w is the horizontal dimension of the feature map, h is the vertical dimension of the feature map, p is the row index, q is the column index, σ(·) is the classification prediction probability, is the feature point of the input image, φ θ is a tensor, W is the classifier, and T is the transposed matrix.

3. The small sample image classification method integrating global and adaptive local information as claimed in claim 1, characterized in that: Step S3 includes: The images of the support set and the images of the query set are subjected to feature extraction through the Resnet network to obtain the support set image features and the query set image features; Calculate the cross entropy loss of the support set image features to initialize the classifier; Input the initialized classifier and query set image features into the classifier reweighted optimizer CRO to learn the classifier weights to update the classifier weights; Use the updated classifier weights to calculate the similarity of the query set image features and output the classification prediction probability of the query set; Meta-training is performed using the cross entropy loss of the query set image features and the classification prediction probability as the loss function; the expression of the loss function is: Where, L SL2 represents the loss function of the task-aware network QSA-TAN, x i is the input image, y i is the true category label of the image, E is the average loss calculation for all training samples in the task, Q is the query set, σ(·) is the classification prediction probability, φ θ is a tensor, w * is the horizontal dimension of the feature map, and T is the transposed matrix.

4. The small sample image classification method integrating global and adaptive local information as claimed in claim 1, characterized in that: In step S4, the pre-training results and the meta-training results are input into the global-local knowledge collaborative network GLKC for joint training to output the calculation formula of the probability distribution difference of the small sample image: include: Where D KL (p QSA ||p DCCP ) is p QSA and p DCCP Probability distribution difference, D KL (p DCCP ||p QSA ) is p DCCP and p QSA The probability distribution difference, p DCCP and p QSA They represent the classification prediction outputs of the global perception network DCCP-Net and the task perception network QSA-TAN, respectively. i is the input image, and N is the total number of samples.

5. The small sample image classification method integrating global and adaptive local information as claimed in claim 1, characterized in that: In step S4, the process of classifying small sample images according to probability distribution differences includes: The final loss function of the global perception network DCCP-Net and the task perception network QSA-TAN is calculated based on the probability distribution difference; A small sample image classification model is constructed based on the final loss function, and the small sample images are classified by the small sample image classification model to obtain the classification results.

6. A small sample image classification method integrating global and adaptive local information as claimed in claim 5, characterized in that: The final loss function of the global perception network DCCP-Net and the task perception network QSA-TAN is expressed as: Loss DCCP =λ1L SL1 +λ2D KL (p QSA ||p DCCP ) Loss QSA =λ3L SL2 +λ4D KL (p DCCP ||p QSA ) In the formula, Loss DCCP is the final loss function of the global perception network DCCP-Net, Loss QSA is the final loss function of the task-aware network QSA-TAN, D KL (p QSA ||p DCCP ) is p QSA and p DCCP Difference in probability distribution, L SL1 represents the loss function of the global perception network DCCP-Net, L SL2 represents the loss function of the task-aware network QSA-TAN, D KL (p DCCP ||p QSA ) is p DCCP and p QSA The probability distribution difference of , λ1, λ2, λ3 and λ4 are all hyper parameters.

7. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, a small sample image classification method that integrates global and adaptive local information as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a small sample image classification method integrating global and adaptive local information is implemented as described in any one of claims 1 to 6.

Images