Image classification method, device, electronic device and storage medium

Abstract

Embodiments of the present invention disclose an image classification method, apparatus, electronic device, and storage medium. The method includes: obtaining current category prototypes corresponding to each sample category; obtaining an image to be classified, matching the image to be classified with the current category prototypes corresponding to each sample category, and obtaining a target category prototype; and using the sample category corresponding to the target category prototype as the target category of the image to be classified to obtain a classification result for the image to be classified. The technical solutions of the embodiments of the present invention implement image classification processing and improve image classification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image classification method, device, electronic equipment and storage medium. Background Art

[0002] The small sample size image classification problem is a very important and challenging problem in the field of computer vision. The goal of this problem is to improve the generalization ability of the model and the accuracy of image classification when there are only a few training samples for a new class.

[0003] The current problem of small sample image classification is still in the research stage, and there is no more perfect way to solve the above technical problems. Summary of the Invention

[0004] The image classification method, device, electronic device, and storage medium provided by the embodiments of the present invention are intended to solve the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present invention provides an image classification method, the method comprising:

[0006] Get the current category prototype corresponding to each sample category;

[0007] Get the image to be classified,

[0008] Matching the image to be classified with the current category prototypes corresponding to each category to obtain the target category prototype;

[0009] The sample category corresponding to the target category prototype is used as the target category of the image to be classified to obtain a classification result of the image to be classified.

[0010] In a second aspect, an embodiment of the present invention further provides an image classification device, comprising:

[0011] The prototype acquisition module is used to obtain the current category prototype corresponding to each sample category;

[0012] Image acquisition module, used to obtain images to be classified,

[0013] A target type prototype acquisition module is used to match the image to be classified with the current category prototype corresponding to each category to obtain the target category prototype;

[0014] The target category acquisition module is used to use the sample category corresponding to the target category prototype as the target category of the image to be classified to obtain a classification result of the image to be classified.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image classification method according to any embodiment of the present invention.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the image classification method as in any embodiment of the present invention.

[0020] The technical solution of the embodiment of the present invention obtains the current category prototype corresponding to each sample category, obtains the image to be classified, matches the image to be classified with the current category prototype corresponding to each sample category, obtains the target category prototype, and uses the sample category corresponding to the target category prototype as the target category of the image to be classified, thereby obtaining the classification result of the image to be classified. The technical solution of the embodiment of the present invention improves the accuracy of image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] in:

[0023] Figure 1 1 is a flow chart of an image classification method according to an embodiment;

[0024] Figure 2 Schematic diagram of the structure of an image classification device in one embodiment;

[0025] Figure 3 FIG. 4 is a schematic structural diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0027] In one embodiment of the present invention, an image classification method is provided, such as Figure 1 As shown, the image classification method of the embodiment of the present invention can achieve the classification of images. The image classification method of the embodiment of the present invention can be performed by an image classification device. The image classification device can be implemented by software and / or hardware.

[0028] like Figure 1 As shown, the image classification method of the embodiment of the present invention includes the following steps:

[0029] S110: Obtain the current category prototype corresponding to each sample category.

[0030] The category prototype refers to the mean of the feature vectors of multiple samples in each sample category. The current category prototype refers to the category prototype corresponding to the current sample category. It should be understood that the feature vectors mentioned in the embodiments of the present invention refer to image feature vectors.

[0031] In the embodiment of the present invention, preparations are made for subsequent image classification by obtaining current category prototypes corresponding to various sample categories.

[0032] S120: Obtain an image to be classified.

[0033] The image to be classified in the embodiment of the present invention refers to an image that needs to be classified.

[0034] Specifically, obtain the image to be classified and prepare for subsequent image classification.

[0035] S130 , matching the image to be classified with the current category prototype corresponding to each category to obtain the target category prototype.

[0036] In the embodiment of the present invention, the image to be assigned is matched with each current category prototype, and the current category prototype with the highest matching degree is used as the target category prototype.

[0037] Optionally, matching the image to be classified with the current category prototypes corresponding to each category to obtain the target category prototype includes: obtaining the feature vector to be classified of the image to be classified; calculating the distance between the feature vector to be classified and each current category prototype; and determining the current category prototype corresponding to the minimum distance as the target category prototype.

[0038] In the embodiment of the present invention, a feature vector to be classified of an image to be classified is obtained, the distance between the feature vector to be classified and each current category prototype is calculated respectively, and the current category prototype corresponding to the minimum distance is determined as the target category prototype.

[0039] Optionally, the distance may be a Euclidean distance.

[0040] S140: Using the sample category corresponding to the target category prototype as the target category of the image to be classified to obtain a classification result of the image to be classified.

[0041] In an embodiment of the present invention, the sample category corresponding to the target category prototype is used as the target category of the image to be classified, thereby obtaining a classification result for the image to be classified. For example, each sample category is represented by a category identifier, and each category identifier has a corresponding relationship with the image category. Based on the category identifier of the target category and the corresponding relationship, the image category of the image to be classified, that is, the classification result of the image to be classified, can be obtained. The technical solution of the embodiment of the present invention can achieve accurate image classification.

[0042] The technical solution of the embodiment of the present invention obtains the current category prototype corresponding to each sample category, obtains the image to be classified, matches the image to be classified with the current category prototype corresponding to each sample category, obtains the target category prototype, and uses the sample category corresponding to the target category prototype as the target category of the image to be classified, thereby obtaining the classification result of the image to be classified. The technical solution of the embodiment of the present invention improves the accuracy of image classification.

[0043] In another embodiment of the present invention, a current category prototype corresponding to each sample category is obtained, including: obtaining a target support set through an initial support set and an initial sample set, wherein each second sample in the initial sample set is marked with a pseudo label, and the pseudo label is associated with the sample category of the first sample in the initial support set, and the number of the second samples is greater than the number of the first samples; obtaining a category prototype to be compared for each sample category based on the mean of the feature vectors of each sample category in the target support set; measuring each of the category prototypes to be compared based on the test feature vectors of the test samples in the query set, and if the total measurement result does not meet the preset conditions, updating the target support set and obtaining each new category prototype to be compared to obtain a new total measurement result; the total measurement result is the sum of the measurement results corresponding to each test sample; if the total measurement result meets the preset conditions, the category prototype to be compared is used as the current category prototype of the sample category.

[0044] The preset condition can refer to a preset threshold, a preset threshold range, or whether a loss function converges. For example, if the measurement result is a loss function, the preset condition is that the sum of the loss values calculated by each loss function is within a preset loss range. If the sum of the loss values is within the preset range, the overall measurement result meets the preset condition. If the sum of the loss values is not within the preset range, the overall measurement result does not meet the preset condition.

[0045] In this embodiment of the present invention, a target support set is derived from an initial support set and an initial sample set. The mean of the feature vectors of the target samples included in each sample category in the target support set is used as the category prototype to be compared for each sample category. The distance between the feature vector of each test sample in the query set and each category prototype to be compared is measured, and a measurement result is obtained. The sum of the measurement results corresponding to each test sample is used as the total measurement result. A determination is made as to whether the total measurement result meets preset conditions. If so, the category prototype to be compared is used as the current category prototype for the corresponding sample category. If the measurement result does not meet the preset conditions, the target support set is updated. A new category prototype to be compared is derived based on the mean of the feature vectors of each sample category in the new target support set. Then, a measurement is performed based on the test feature vectors of the test samples in the query set and each of the new category prototypes to be compared to obtain a new total measurement result, and a determination is again made as to whether the new total measurement result meets the preset conditions. This technical solution of this embodiment of the present invention can update the category prototype to be compared based on the total measurement result, thereby making the current category prototype more consistent with the characteristics of the sample category, thereby improving the accuracy of image classification in subsequent image classification.

[0046] In another embodiment of the present invention, the target support set is obtained through the initial support set and the initial sample set, including: labeling the second sample in the initial sample set with a pseudo label through the sample category of the first sample in the initial support set to obtain several third samples; for each third sample, obtaining the attribute vector of the third sample, the attribute vector including the semantic information of the sample category to which the third sample belongs; for each third sample, processing the feature vector and the corresponding attribute vector of the third sample through a preset algorithm to obtain a reconstructed feature vector corresponding to the third sample; and obtaining the target support set from the several reconstructed feature vectors and the feature vectors of the several first samples.

[0047] It should be understood that the third sample refers to the second sample with a pseudo label added. The reconstructed sample is the third sample with the semantic information of the sample category added.

[0048] In an embodiment of the present invention, the second sample in the initial sample set is annotated with a pseudo label based on the sample category of the first sample in the initial support set. It should be understood that each sample category has a corresponding sample label. The second sample is annotated with a pseudo label based on the sample category, including annotating the second sample with a pseudo label based on the label of the sample category, to obtain a plurality of third samples. For each third sample, an attribute vector with semantic information of the third sample is obtained, and the feature vector and the corresponding attribute vector of the third sample are processed by a preset algorithm to obtain a reconstructed feature vector corresponding to the third sample. With all the reconstructed feature vectors and the feature vectors of all the first samples, the target support set is obtained.

[0049] Optionally, the processing of the feature vector and the corresponding attribute vector of the third sample by a preset algorithm to obtain the reconstructed feature vector corresponding to the third sample includes: processing the feature vector and the corresponding attribute vector of the third sample by a variational encoder, and obtaining the reconstructed feature vector when the loss value of the variational encoder meets the first preset loss condition, the loss value of the variational encoder refers to the sum of the loss values corresponding to each sample, and the expression of each loss function can be: L = KL (q (z | x i ,a n )||p(z|a n ))-logp(x i |z,a n ) where x i represents the third sample of i, z represents the hidden space, a n Represents the semantic information related to the n sample categories, KL(q(z|x i ,a n )||p(z|a n )) represents the posterior q(z|x i ,a n ) and the prior p(z|a n ), logp(x i |z,a n ) represents the reconstruction loss.

[0050] The variational encoder may be a variational autoencoder (VAE). In the embodiment of the present invention, the priori refers to a priori probability, and the posterior refers to a posterior probability.

[0051] In the embodiment of the present invention, the feature vector and corresponding attribute information of the third sample are encoded by a variational encoder, and a reconstructed feature vector is output by a variational decoder. The reconstructed feature vector is the third sample with semantic information of the sample category.

[0052] When the loss value of the variational encoder meets the preset conditions, a reconstructed feature vector is obtained. When the loss value of the variational encoder does not meet the preset conditions, the parameters of the variational encoder are adjusted to obtain the current variational encoder. The feature vector and corresponding attribute information of the third sample are then processed by the current variational encoder, and the current reconstructed feature vector is output by the variational decoder, and the loss value is obtained again based on the loss function.

[0053] In another embodiment of the present invention, the second sample in the initial sample set is annotated with a pseudo label by the sample category of the first sample in the initial support set to obtain several third samples, including: clustering the feature vectors of several second samples based on the sample category through a clustering algorithm to obtain several target classes; for each target class, calculating the distance between the initial category prototype of the sample category corresponding to the target class and the feature vector of the second sample in the target class, and annotating the feature vector of the second sample that meets the preset distance condition with a pseudo label based on the sample category.

[0054] In the embodiment of the present invention, the clustering algorithm includes but is not limited to the Gaussian mixture model (GMM). The distance calculation can be Euclidean distance calculation. Of course, other distance calculation methods can also be used, such as the Manhattan distance calculation method, the Chebyshev distance calculation method, and the cosine distance calculation method. No specific limitation is made here.

[0055] Specifically, GMM is used to cluster the feature vectors of several second samples based on sample type to obtain several target classes. For each target class, a distance calculation is performed between the initial class prototype of the sample class corresponding to the target class and the feature vectors of the second samples in the target class. The feature vectors of the second samples that meet the preset distance conditions are annotated with pseudo labels based on the sample class. In this way, several second samples with pseudo labels can be obtained for each sample class.

[0056] In another embodiment of the present invention, the measurement result is a cross-entropy loss function, and if the total measurement result does not meet the preset conditions, the target support set is updated, and each new prototype of the category to be compared is obtained, including: if the sum of the loss values of several cross-entropy loss functions does not meet the preset conditions, the parameters of the feature extractor are adjusted to obtain the current feature extractor; based on the current feature extractor, the current sample feature vector of the first sample and the current sample feature vector of the second sample are extracted; for each second sample, based on the current sample feature vector of the second sample, the current feature vector of the third sample is obtained; for each third sample, the current feature vector of the third sample and the corresponding attribute vector are processed by a preset algorithm to obtain the current reconstructed feature vector corresponding to the third sample; from the several reconstructed feature vectors and the current feature vectors of the several first samples, an updated target support set is obtained to obtain each new prototype of the category to be compared.

[0057] The cross entropy loss function formula is: L1(x) = CE(y, y1), where y is the true label value corresponding to x, y1 is the predicted label value of x, and CE() represents the cross entropy loss. Of course, the embodiments of the present invention are not limited to the use of the cross entropy loss function, and other loss function calculation methods can also be used, which can be adjusted according to actual conditions.

[0058] In an embodiment of the present invention, by calculating the distance between the test feature vector of each test sample and each prototype of the category to be compared, a number of to-be-determined test label values can be obtained for each test sample based on the distance. Each test sample has a true category label and a true label value corresponding to the true category label. A test label value corresponding to the true category label of the test sample is obtained from the number of to-be-determined test label values, and a cross-entropy loss function is calculated based on the test label value and the true label value. A loss value corresponding to each test sample is thereby obtained. It should be understood that each prototype of the category to be compared corresponds one-to-one to the category label. All loss values are summed. If the sum does not meet a preset condition, the parameters of the feature extractor are readjusted, and the current sample features of the first sample and the current sample features of the second sample are extracted based on the updated feature extractor. Pseudo-labels are then annotated based on the current sample features of the second sample to obtain a current feature vector for the third sample. The current feature vector of the third sample and the corresponding attribute vector are then processed by a variational encoder to obtain a current reconstructed feature vector corresponding to the third sample. The multiple reconstructed feature vectors and the multiple current feature vectors of the first sample are used to obtain an updated target support set, thereby obtaining each new prototype of the category to be compared.

[0059] Optionally, in the embodiment of the present invention, the feature extractor may be a convolutional neural network, represented as f a , where a represents the parameters of the convolutional neural network f. By updating the parameters of the feature extractor, the cross-entropy loss function is optimized, and the cross-entropy loss function converges to obtain a target feature extractor. A feature vector of an image sample is obtained based on the target feature extractor. The image sample includes a first sample, a second sample, a third sample, a test sample, and an image to be classified.

[0060] In one embodiment of the present invention, a feature extractor obtained through training performs feature extraction on an image to be classified, and matches a feature vector of the image to be classified with each current category prototype, and uses the current category prototype with the highest matching degree as the target category prototype, and uses the sample category corresponding to the target category prototype as the target category of the image to be classified, thereby obtaining the image category of the image to be classified.

[0061] In another embodiment of the present invention, an image classification method is provided, which is a preferred embodiment of the above embodiment. Technical terms that are the same or similar to those in the above embodiment will not be repeated.

[0062] In this embodiment of the present invention, the initial support set contains a small number of labeled first samples, each of which includes several sample categories. Because each sample category contains a small number of first samples, the category information contained in each sample category is relatively small, and the initial category prototypes for each sample category are therefore not representative enough. Therefore, an initial sample set containing a large number of unlabeled second samples is introduced.

[0063] First, feature extraction is performed on each first sample in the initial support set and each second sample in the initial sample set to obtain their respective feature vectors. Optionally, feature extraction can be performed using a feature extractor. The feature extractor maps the first sample or the second sample into a feature space. The following description is performed on the feature space.

[0064] First, all the second samples in the initial sample set are clustered by Gaussian mixture model (GMM). Specifically, according to the N initial category prototypes included in the initial support set, it is expressed as P = {p1, p2, ..., p N}, cluster the second sample into N target classes through GMM, expressed as C = {c1,c2,…,c N It should be noted that, in the embodiment of the present invention, the second sample can find a corresponding sample category in the initial support set. It should be understood that clustering here refers to processing the image feature vector of the second sample to obtain N target classes.

[0065] Next, pseudo-label the clustered second samples. Specifically, calculate the Euclidean distance between each second sample in C and the N initial category prototypes. Label the sample closest to a particular initial category prototype with the category label of that initial category prototype. By pseudo-labeling the second samples, an updated second sample is obtained, which is used as the third sample.

[0066] Next, category attribute information is added to each third sample. Category attribute information refers to the semantic information of the category. The attribute vector and the feature vector of the third sample are used as input to the variational encoder, which then outputs a reconstructed feature vector that contains the category semantic information.

[0067] The loss function of the variational encoder is set, which incorporates category semantic information. By optimizing the loss function, the parameters of the variational encoder are adjusted to obtain the optimal variational encoder. This optimal variational encoder is then used to obtain the optimal reconstructed feature vector.

[0068] The mean of several reconstructed feature vectors included in each sample category and the feature vector of the first sample is calculated to obtain the category prototype to be compared of the sample category.

[0069] Finally, the Euclidean distance between the test feature vector of each test sample in the query set and each prototype of the category to be compared is used to determine the sample category of the current test sample, that is, the probability value calculated according to the following formula. The larger the probability value, the greater the possibility that the sample belongs to this category.

[0070]

[0071] Among them, x is the test sample, f a represents the feature extractor, y represents the sample label, p n ∈P,p n Represents the prototype of the category to be compared, n represents the serial number of the sample category, and d<> represents the Euclidean distance between the feature vector of the test sample and the prototype of the category to be compared.

[0072] The predicted label value corresponding to each test sample is calculated, and the cross entropy loss function is formed based on the distance between the predicted label value and the true label value of each test sample.

[0073] Optionally, the loss function of the variational encoder and the cross entropy loss function can be used as the target loss function. When the target loss function does not meet the preset conditions, the parameters of the feature extractor and the variational encoder are adjusted, and then the first sample and the second sample are reprocessed. After further processing, an updated prototype of the category to be compared is obtained. When the target loss function meets the preset conditions, the current prototype of the category to be compared is used as the current category prototype.

[0074] Optionally, the loss function of the feature extractor may also be a regression loss function, etc. In another embodiment of the present invention, an image classification device is provided, such as Figure 2 As shown, the image classification device of the embodiment of the present invention can execute the image classification method of any embodiment of the present invention. The image classification device of the embodiment of the present invention has the corresponding functional modules and beneficial effects of the execution method. The technical terms that are the same or similar to those in the above embodiments will not be repeated.

[0075] like Figure 2As shown, the apparatus of the embodiment of the present invention includes: a prototype acquisition module 410, an image acquisition module 420, a target type prototype acquisition module 430 and a target category acquisition module 440, wherein:

[0076] The prototype acquisition module 410 is used to obtain the current category prototype corresponding to each sample category; the image acquisition module 420 is used to obtain the image to be classified; the target type prototype acquisition module 430 is used to match the image to be classified with the current category prototype corresponding to each category to obtain the target category prototype; the target category acquisition module 440 is used to use the sample category corresponding to the target category prototype as the target category of the image to be classified to obtain the classification result of the image to be classified.

[0077] Furthermore, in the embodiment of the present invention, the prototype acquisition module 410 is further configured to:

[0078] A target support set is obtained through an initial support set and an initial sample set, wherein each second sample in the initial sample set is marked with a pseudo label, and the pseudo label is associated with the sample category of the first sample in the initial support set, and the number of second samples is greater than the number of first samples; based on the mean of the feature vectors of each sample category in the target support set, a category prototype to be compared is obtained for each sample category; based on the test feature vectors of the test samples in the query set and each of the category prototypes to be compared, a measurement is performed; if the total measurement result does not meet the preset conditions, the target support set is updated, and each new category prototype to be compared is obtained to obtain a new total measurement result; the total measurement result is the sum of the measurement results corresponding to each test sample; if the total measurement result meets the preset conditions, the category prototype to be compared is used as the current category prototype of the sample category.

[0079] Furthermore, in an embodiment of the present invention, the prototype acquisition module 410 is also used to: annotate the second sample in the initial sample set with a pseudo-label according to the sample category of the first sample in the initial support set to obtain a plurality of third samples; for each third sample, obtain the attribute vector of the third sample, wherein the attribute vector includes the semantic information of the sample category to which the third sample belongs; for each third sample, process the feature vector and the corresponding attribute vector of the third sample by a preset algorithm to obtain a reconstructed feature vector corresponding to the third sample; and obtain the target support set from the plurality of reconstructed feature vectors and the feature vectors of the plurality of first samples.

[0080] Furthermore, in the embodiment of the present invention, the prototype acquisition module 410 is further configured to:

[0081] The feature vectors of several second samples are clustered based on the sample category through a clustering algorithm to obtain several target categories; for each target category, the distance between the feature vectors of the second samples in the target category and the initial category prototype of the sample category corresponding to the target category is calculated, and the feature vectors of the second samples that meet the preset distance conditions are annotated with pseudo labels based on the sample category.

[0082] Furthermore, in an embodiment of the present invention, the measurement result is a cross entropy loss function.

[0083] The prototype acquisition module 410 is further configured to:

[0084] If the sum of the loss values of several cross entropy loss functions does not meet the preset conditions, the parameters of the feature extractor are adjusted to obtain the current feature extractor; the current sample feature vector of the first sample and the current sample feature vector of the second sample are extracted based on the current feature extractor; for each second sample, the current feature vector of the third sample is obtained based on the current sample feature vector of the second sample; for each third sample, the current feature vector of the third sample and the corresponding attribute vector are processed by a preset algorithm to obtain the current reconstructed feature vector corresponding to the third sample; an updated target support set is obtained from the several reconstructed feature vectors and the current feature vectors of the several first samples to obtain each new prototype of the category to be compared.

[0085] Furthermore, in the embodiment of the present invention, the prototype acquisition module 410 is further configured to:

[0086] The feature vector and the corresponding attribute vector of the third sample are processed by a variational encoder. When a loss value of the variational encoder meets a first preset loss condition, the reconstructed feature vector is obtained. The loss function corresponding to the loss value of the variational encoder includes:

[0087] L=KL(q(z|x i ,a n )||p(z|a n ))-logp(x i |z,a n )

[0088] Among them, x i represents the third sample of i, z represents the hidden space, a n Represents the semantic information related to the sample category with sequence number n, KL(q(z|x i ,a n )||p(z|a n )) represents the posterior q(z|x i ,a n ) and the prior p(z|a n), logp(x i |z,a n ) represents the reconstruction loss.

[0089] Furthermore, in the embodiment of the present invention, the target type prototype acquisition module 430 is further configured to:

[0090] Obtain a feature vector to be classified of the image to be classified; calculate the distance between the feature vector to be classified and each current category prototype respectively; and determine the current category prototype corresponding to the minimum distance as the target category prototype.

[0091] The technical solution of the embodiment of the present invention obtains the current category prototype corresponding to each sample category, obtains the image to be classified, matches the image to be classified with the current category prototype corresponding to each sample category, obtains the target category prototype, and uses the sample category corresponding to the target category prototype as the target category of the image to be classified, thereby obtaining the classification result of the image to be classified. The technical solution of the embodiment of the present invention improves the accuracy of image classification.

[0092] It is worth noting that the various modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the various functional modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0093] In another embodiment of the present invention, an electronic device is provided. Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 3 A block diagram of an exemplary electronic device 50 suitable for implementing exemplary embodiments of the present invention is shown. Figure 3 The electronic device 50 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0094] like Figure 3 As shown, electronic device 50 is a general-purpose computing device. Components of electronic device 50 may include, but are not limited to, one or more processors or processing units 501, system memory 502, and a bus 503 connecting various system components (including system memory 502 and processing unit 501).

[0095] Bus 503 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0096] The electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0097] System memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. Electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 506 may be used to read and write non-removable, non-volatile magnetic media ( Figure 3 Not shown, often called a "hard drive"). Although Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 503 via one or more data medium interfaces. Memory 502 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0098] A program / utility 508 having a set (at least one) of program modules 507 may be stored, for example, in memory 502. Such program modules 507 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 507 generally implement the functions and / or methods of the embodiments described herein.

[0099] The electronic device 50 may also communicate with one or more external devices 509 (e.g., keyboard, pointing device, display 510, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 50, and / or communicate with any device that enables the electronic device 50 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 511. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 512. As shown, the network adapter 512 communicates with other modules of the electronic device 50 via the bus 503. It should be understood that although Figure 3 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 50, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0100] The processing unit 501 executes various functional applications and data processing by running programs stored in the system memory 502 , such as implementing the image classification method provided by the embodiment of the present invention.

[0101] In another embodiment of the present invention, a storage medium containing computer-executable instructions is further provided. When the computer-executable instructions are executed by a computer processor, the computer-executable instructions are used to perform an image classification method, the method comprising:

[0102] Obtain the current category prototype corresponding to each sample category; obtain the image to be classified, match the image to be classified with the current category prototype corresponding to each category, and obtain the target category prototype; use the sample category corresponding to the target category prototype as the target category of the image to be classified to obtain the classification result of the image to be classified.

[0103] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0104] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0105] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0106] The computer program code for performing the operations of the embodiments of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0107] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An image classification method, characterized in that: include: Get the current category prototype corresponding to each sample category; Get the image to be classified, Matching the image to be classified with the current category prototypes corresponding to each category to obtain the target category prototype; Using the sample category corresponding to the target category prototype as the target category of the image to be classified to obtain a classification result of the image to be classified; The step of obtaining the current category prototype corresponding to each sample category includes: annotating the second sample in the initial sample set with a pseudo label by the sample category of the first sample in the initial support set to obtain a plurality of third samples; for each third sample, obtaining an attribute vector of the third sample, wherein the attribute vector includes semantic information of the sample category to which the third sample belongs; for each third sample, processing the feature vector and the corresponding attribute vector of the third sample by a variational encoder, and obtaining a reconstructed feature vector when the loss value of the variational encoder meets the first preset loss condition, and the loss function corresponding to the loss value of the variational encoder includes: L=KL(q(z|x i ,a n )||p(z|a n ))-logp(x i |z,a n ), where x i represents the third sample of i, z represents the hidden space, a n Represents the semantic information related to the n sample categories, KL(q(z|x i ,a n )||p(z|a n )) represents the posterior q(z|x i ,a n ) and the prior p(z|a n ), logp(x i |z,a n ) represents the reconstruction loss; a target support set is obtained by a number of reconstructed feature vectors and a number of feature vectors of the first samples, each second sample in the initial sample set is marked with a pseudo label, the pseudo label is associated with the sample category of the first sample in the initial support set, and the number of the second samples is greater than the number of the first samples; based on the mean of the feature vectors of each sample category in the target support set, a category prototype to be compared is obtained for each sample category; based on the test feature vector of the test sample in the query set and each of the category prototypes to be compared, if the total measurement result does not meet the preset conditions, the target support set is updated, and each new category prototype to be compared is obtained to obtain a new total measurement result; the total measurement result is the sum of the measurement results corresponding to each test sample; if the total measurement result meets the preset conditions, the category prototype to be compared is used as the current category prototype of the sample category.

2. The method according to claim 1, characterized in that The method of labeling the second sample in the initial sample set with a pseudo label by using the sample category of the first sample in the initial support set to obtain a plurality of third samples includes: Clustering the feature vectors of the plurality of second samples based on the sample categories using a clustering algorithm to obtain a plurality of target classes; For each target class, the distance calculation is performed between the initial class prototype of the sample class corresponding to the target class and the feature vector of the second sample in the target class, and the feature vector of the second sample that meets the preset distance condition is annotated with a pseudo label based on the sample class.

3. The method according to claim 2, characterized in that The measurement result is the cross entropy loss function, If the total measurement result does not meet the preset conditions, the target support set is updated and each new prototype of the category to be compared is obtained, including: If the sum of the loss values of several cross entropy loss functions does not meet the preset conditions, the parameters of the feature extractor are adjusted to obtain the current feature extractor; extracting a current sample feature vector of the first sample and a current sample feature vector of the second sample based on the current feature extractor; For each second sample, obtaining a current feature vector of a third sample based on the current sample feature vector of the second sample; For each third sample, processing the current feature vector and the corresponding attribute vector of the third sample by a preset algorithm to obtain a current reconstructed feature vector corresponding to the third sample; An updated target support set is obtained from the plurality of reconstructed feature vectors and the plurality of current feature vectors of the first samples, so as to obtain each new prototype of the category to be compared.

4. The method according to claim 1, wherein The step of matching the image to be classified with the current category prototypes corresponding to the respective categories to obtain the target category prototype includes: Obtaining a feature vector to be classified of the image to be classified; Calculating the distance between the feature vector to be classified and each current category prototype respectively; The current category prototype corresponding to the minimum distance is determined as the target category prototype.

5. An image classification device, characterized in that: include: The prototype acquisition module is used to obtain the current category prototype corresponding to each sample category; An image acquisition module, used to acquire images to be classified; A target type prototype acquisition module is used to match the image to be classified with the current category prototype corresponding to each category to obtain the target category prototype; a target category acquisition module, configured to use the sample category corresponding to the target category prototype as the target category of the image to be classified, so as to obtain a classification result of the image to be classified; Among them, when the prototype acquisition module obtains the current category prototype corresponding to each sample category, it includes: using the sample category of the first sample in the initial support set to annotate the second sample in the initial sample set with a pseudo label to obtain a plurality of third samples; for each third sample, obtaining the attribute vector of the third sample, wherein the attribute vector includes the semantic information of the sample category to which the third sample belongs; for each third sample, processing the feature vector and the corresponding attribute vector of the third sample through a variational encoder, and obtaining a reconstructed feature vector when the loss value of the variational encoder meets the first preset loss condition, and the loss function corresponding to the loss value of the variational encoder includes: L=KL(q(z|x i ,a n )||p(z|a n ))-logp(x i |z,a n ), where x i represents the third sample of i, z represents the hidden space, a n Represents the semantic information related to the n sample categories, KL(q(z|x i ,a n )||p(z|a n )) represents the posterior q(z|x i ,a n ) and the prior p(z|a n ), logp(x i |z,a n ) represents the reconstruction loss; a target support set is obtained by a number of reconstructed feature vectors and a number of feature vectors of the first samples, each second sample in the initial sample set is marked with a pseudo label, the pseudo label is associated with the sample category of the first sample in the initial support set, and the number of the second samples is greater than the number of the first samples; based on the mean of the feature vectors of each sample category in the target support set, a category prototype to be compared is obtained for each sample category; based on the test feature vector of the test sample in the query set and each of the category prototypes to be compared, if the total measurement result does not meet the preset conditions, the target support set is updated, and each new category prototype to be compared is obtained to obtain a new total measurement result; the total measurement result is the sum of the measurement results corresponding to each test sample; if the total measurement result meets the preset conditions, the category prototype to be compared is used as the current category prototype of the sample category.

6. The device according to claim 5, characterized in that The prototype acquisition module annotates the second sample in the initial sample set with a pseudo label according to the sample category of the first sample in the initial support set to obtain a plurality of third samples, including: Clustering the feature vectors of the plurality of second samples based on the sample categories using a clustering algorithm to obtain a plurality of target classes; For each target class, the distance calculation is performed between the initial class prototype of the sample class corresponding to the target class and the feature vector of the second sample in the target class, and the feature vector of the second sample that meets the preset distance condition is annotated with a pseudo label based on the sample class.

7. The device according to claim 5, characterized in that The target type prototype acquisition module matches the image to be classified with the current category prototypes corresponding to the categories to obtain the target category prototype, including: Obtaining a feature vector to be classified of the image to be classified; Calculating the distance between the feature vector to be classified and each current category prototype respectively; The current category prototype corresponding to the minimum distance is determined as the target category prototype.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image classification method according to any one of claims 1 to 4.

9. A storage medium containing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the image classification method according to any one of claims 1 to 4.

Images