Artificial intelligence classification method, system, electronic device and storage medium for images

Abstract

The present application discloses an artificial intelligence classification method, system, electronic device, and storage medium for images. The artificial intelligence classification method for images includes: training a preset classification model according to the direction of increasing inter-class spacing of parent classifications of a first group of training images to obtain a pre-order classification model, wherein the inter-class spacing of parent classifications is the distance between feature vectors corresponding to training images with different parent classifications in the first group of training images; training a preset classification model according to the direction of increasing intra-class spacing of sub-classes of a second group of training images to obtain an image classification model, wherein the intra-class spacing of sub-classes is the distance between feature vectors corresponding to training images with the same parent classification but different sub-classifications in the second group of training images; and classifying the images to be classified using the image classification model to obtain a classification result. The present application is conducive to improving the accuracy of image classification.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and specifically to an artificial intelligence image classification method, system, electronic device and storage medium. Background Art

[0002] Image classification method is an image processing method that distinguishes objects of different categories based on the different features reflected in the image information.

[0003] In some scenarios, AI techniques are used to extract features from images for classification. When the number of image categories is large and detailed, the training set often lacks specific labels for these subcategories. Labeling the images is expensive. Further subclassifying unlabeled categories based on similarity increases the difficulty of classification, making it prone to inaccurate classification. Summary of the Invention

[0004] Based on this, in order to solve or improve the problems of the prior art, the present application provides an artificial intelligence image classification method, system, electronic device and storage medium, which can improve the accuracy of image classification.

[0005] In a first aspect, an artificial intelligence image classification method is provided, comprising:

[0006] Training the preset classification model according to the direction of increasing inter-class spacing of the parent classification of the first group of training images to obtain a pre-order classification model, wherein the inter-class spacing of the parent classification is the distance between the feature vectors corresponding to the training images of different parent classifications in the first group of training images;

[0007] The preceding classification model is trained according to a direction of increasing intra-subclass spacing of the second set of training images to obtain an image classification model, wherein the intra-subclass spacing is the distance between feature vectors corresponding to training images of the second set of training images with the same parent class but different subclasses;

[0008] The image to be classified is classified by the image classification model to obtain a classification result.

[0009] In one embodiment, the preset classification model includes an input layer, a convolutional layer, and a parent classification layer;

[0010] The method of training the preset classification model according to the direction of increasing distance between parent classification classes of the first group of training images to obtain the preceding classification model includes:

[0011] Inputting the first set of training images into the convolution layer through the input layer and performing convolution processing to obtain a plurality of first feature vectors respectively;

[0012] Obtaining the plurality of first eigenvectors through the parent classification layer, and calculating a first loss function based on the plurality of first eigenvectors;

[0013] The parameters of the convolutional layer are updated according to the first loss function, the distance between the first eigenvectors corresponding to the training pictures with different parent categories in the first group of training pictures is increased, and the distance between the first eigenvectors corresponding to the training pictures with the same parent category in the first group of training pictures is reduced to obtain a pre-order classification model.

[0014] In one embodiment, the training of a preset classification model according to the direction of increasing inter-subclass spacing of the second set of training images to obtain an image classification model includes:

[0015] Obtaining a plurality of second feature vectors according to the second set of training images;

[0016] Obtaining a plurality of sub-classification cluster centers according to the plurality of second eigenvectors;

[0017] According to the direction in which the distances between the centers of the multiple sub-classification clusters increase, the parameters of the preset classification model are updated to obtain an image classification model.

[0018] In one embodiment, obtaining a plurality of sub-classification cluster centers according to the plurality of second eigenvectors includes:

[0019] According to the plurality of second feature vectors, obtaining parent categories and corresponding child categories corresponding to the plurality of second feature vectors;

[0020] Clustering the second eigenvectors with the same parent classification to obtain multiple subclassifications with the second eigenvectors;

[0021] The average values ​​of the second eigenvectors of the same sub-classifications are calculated respectively to obtain multiple sub-classification cluster centers.

[0022] In one embodiment, the pre-order classification model includes an input layer, a convolutional layer, a parent classification layer, and a plurality of sub-classification layers;

[0023] The preset classification model is trained in the direction of increasing intra-class spacing of the second set of training images to obtain an image classification model, including:

[0024] Inputting the second set of training images into the convolution layer through the input layer for convolution processing to obtain a plurality of second feature vectors respectively;

[0025] Obtaining a plurality of sub-classification cluster centers according to the plurality of second eigenvectors;

[0026] Obtaining the plurality of second eigenvectors through the parent classification layer, and obtaining a second loss function according to the second eigenvectors corresponding to the parent classification;

[0027] The multiple sub-classification layers respectively obtain corresponding second eigenvectors, and obtain a third loss function according to the second eigenvectors corresponding to the sub-classifications;

[0028] The multiple sub-classification layers respectively obtain corresponding second eigenvectors, and obtain a fourth loss function according to the sub-classification cluster centers;

[0029] According to the direction in which the target loss function decreases, the parameters of the preset classification model are updated to obtain an image classification model, and the target loss function is positively correlated with the second loss function, the third loss function and the fourth loss function, respectively.

[0030] In one embodiment, obtaining the fourth loss function according to the sub-classification cluster centers includes:

[0031] Obtaining parent classification cluster centers according to multiple second eigenvectors;

[0032] The fourth loss function is calculated based on the distance between the sub-classification cluster center and the corresponding parent classification cluster center.

[0033] In one embodiment, the third loss function is a multi-category cross entropy loss function.

[0034] In a second aspect, an image classification system is provided, comprising:

[0035] A first acquisition module is configured to train a preset classification model according to a direction of increasing inter-class spacing of parent classifications of the first set of training images to obtain a preceding classification model, wherein the inter-class spacing of parent classifications is a distance between feature vectors corresponding to training images of different parent classifications in the first set of training images;

[0036] a second acquisition module, configured to train the preceding classification model according to a direction of increasing intra-subclass spacing of the second set of training images to obtain an image classification model, wherein the intra-subclass spacing is a distance between feature vectors corresponding to training images of the second set of training images that have the same parent class but different subclasses;

[0037] The classification module is used to classify the image to be classified using the image classification model to obtain a classification result.

[0038] In a third aspect, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned artificial intelligence image classification method.

[0039] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed on a computer, the computer executes the steps of the above-mentioned artificial intelligence image classification method.

[0040] In the above-mentioned artificial intelligence classification method for images, when the first group of training pictures is used to train the preset classification model, the parameters in the preset classification model are updated according to the direction of increasing spacing between parent classification classes, which is beneficial for the previous classification model to make the feature vectors corresponding to images with different parent classifications have a larger distance when processing images. When the second group of training pictures is used to train the previous classification model, the parameters in the previous classification model are updated according to the direction of increasing spacing within sub-classification classes, which is beneficial for the image classification model to make the feature vectors corresponding to images with the same parent classification but different sub-classifications have a larger distance when processing images, thereby achieving finer image classification and improving the accuracy of image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. It should be understood that the drawings described below are only used to explain the present application and are not used to limit the present application.

[0042] Figure 1 This is a schematic diagram of the internal structure of an electronic device in one embodiment of the present application;

[0043] Figure 2 1 is a flow chart of an artificial intelligence image classification method according to an embodiment of the present application;

[0044] Figure 3 This is a schematic diagram of the relationship between feature vectors obtained by using a pre-order classification model during the image classification process of an embodiment of the present application;

[0045] Figure 4 This is a schematic diagram of the relationship between feature vectors obtained by using an image classification model during the image classification process of an embodiment of the present application;

[0046] Figure 5 This is a structural block diagram of an image classification system according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following, in conjunction with the accompanying drawings, clearly and completely describes the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not intended to limit the present application.

[0048] Figure 1FIG. 1 is a schematic diagram of the internal structure of an electronic device in one embodiment. Figure 1 As shown, the terminal includes a processor, a memory and a network interface connected via a system bus. The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory is used to store data, programs, etc., and at least one computer program is stored on the memory. The computer program can be executed by the processor to implement the wireless network communication method for electronic devices provided in the embodiments of the present application. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement an artificial intelligence classification method for an image provided in the following embodiments. The internal memory provides a cached operating environment for the operating system computer program in the non-volatile storage medium. The network interface can be an Ethernet card or a wireless network card, etc., for communicating with external electronic devices.

[0049] The electronic devices described in this application may include mobile terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0050] The subsequent description will be made by taking a mobile terminal as an example. It will be understood by those skilled in the art that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present application can also be applied to fixed-type terminals.

[0051] See also Figure 2 , Figure 2 1 is a flow chart of an artificial intelligence image classification method according to an embodiment of the present application. The artificial intelligence image classification method includes steps 10 to 30.

[0052] Step 10: Train the preset classification model according to the direction of increasing inter-class spacing of the parent classification of the first group of training images to obtain a pre-order classification model, wherein the inter-class spacing of the parent classification is the distance between the feature vectors corresponding to the training images of different parent classifications in the first group of training images.

[0053] Step 20: Train the previous classification model according to the direction of increasing intra-subclass spacing of the second set of training images to obtain an image classification model, wherein the intra-subclass spacing is the distance between the feature vectors corresponding to training images with the same parent classification but different subclassifications in the second set of training images.

[0054] Step 30: Classify the image to be classified using the image classification model to obtain a classification result.

[0055] In a specific application scenario, for example, in an application scenario of classifying pet images, pet images are divided into three parent categories: cat images, dog images, and horse images; further, cat images can be divided into subcategories such as short-haired cat images, long-haired cat images, tabby cat images, Persian cat images, and Burmese cat images; dog images can be divided into subcategories such as poodle images, husky images, and Samoyed images; and horse images can be divided into subcategories such as grassland horse images, desert horse images, and forest horse images.

[0056] In the process of classifying the pet images in the above scenario, similar pet images need to be clustered together based on their similarity. Specifically, in the process of training a pre-classification model with multiple pet images, after the training set consisting of multiple pet images is input into the pre-classification model, the feature vector output by the feature layer of the pre-classification model is used as the standard for measuring whether the two pet images are similar based on the distance between the feature vectors corresponding to two different pet images. Specifically, each pet image in the training set is labeled with a corresponding parent classification label. The pre-classification model is trained using this training set so that there is a large difference between the feature vectors corresponding to different parent classification labels. Then, based on the similarity threshold, it can be determined whether the classification of the two pet images is the same. For example, when the similarity between the two pet images is lower than the similarity threshold of 0.6, the two pet images are judged to be of different categories, thereby achieving the classification of cat images, dog images, and horse images.

[0057] However, because neither the training set nor the pre-classification model has sub-classification labels, when a finer classification is required, such as when performing intra-class subdivision based on similarity, the accuracy of image classification may be reduced. Specifically, when cat images need to be subdivided, for example, when cat images need to be divided into images of tabby cats and images of British shorthair cats, a similarity threshold can theoretically be selected according to the above method to achieve further subdivision. However, in the process of actually training the pre-classification model using some loss functions such as arcloss, since all cat images are counted as the same cat category (the same parent category), the intra-class distances of pet images of the same parent category will be compressed very small after the training is completed, making the feature vectors corresponding to cat images of different breeds (images of different parent categories and subcategories) not very different. Therefore, it may be difficult to find a suitable similarity threshold to distinguish cat images of different breeds, which will cause the image classification model to have a reduced classification accuracy in the face of personalized sub-classification scenarios.

[0058] During the training process, this embodiment first increases the distance between the feature vectors corresponding to images with different parent categories, and then increases the distance between the feature vectors corresponding to images with the same parent category but different subcategories. This further increases the distance between the feature vectors corresponding to images with the same parent category but different subcategories while ensuring that the feature vectors corresponding to images with different parent categories have a large distance between them. This makes it easier to find a suitable similarity threshold to distinguish images of different subcategories, which in turn helps to improve the accuracy of fine image classification.

[0059] Step 10 is to train the preset classification model using the first set of training images, and by updating the parameters (weights) in the preset classification model, increase the distance between the feature vectors of images with different parent classifications, thereby obtaining a pre-order classification model. The feature vector is the data processing result obtained by convolution of the training image input into the neural network model. The distance between the feature vectors can specifically be the angular spacing of the feature vectors, which is specifically obtained by calculating the cosine similarity between the feature vectors.

[0060] The first set of training images includes multiple training images, each of which is labeled with at least the label corresponding to the parent category. For example, the training images are labeled with only the parent category labels of cat, dog, and horse, but not the sub-category labels of cat, such as shorthair, longhair, and tabby cat; the sub-category labels of dog, such as poodle, husky, and Samoyed; the sub-category labels of dog, such as poodle, husky, and Samoyed; and the sub-category labels of horse, such as steppe horse, forest horse, and desert horse.

[0061] Specifically, during training, multiple training images are divided into multiple groups of training images (multiple batches of training images), and each group of training images is all the training images input into the neural network model at one time. The number of training images in each group of training images is the sample size of training images used in one iteration. For example, the number of training images in the first group of training images is 32, 48, 64 or 128, etc.

[0062] In one of the embodiments, the first group of training pictures specifically includes a first training picture and a second training picture. When the parent categories of the first training picture and the second training picture are different, the parameters in the preset classification model are updated at least according to the direction in which the distance between the feature vectors corresponding to the first training picture and the second training picture increases. When the parent categories of the first training picture and the second training picture are the same, the parameters in the preset classification model are updated at least according to the direction in which the distance between the feature vectors corresponding to the first training picture and the second training picture decreases to obtain a pre-order classification model.

[0063] It should be noted that the parent category of the first training image is obtained according to the category label annotated on the first training image, and the parent category of the second training image is obtained according to the category label annotated on the second training image.

[0064] In one embodiment, the preset classification model includes an input layer, a convolutional layer, and a parent classification layer. The input layer is used to input a training image into the preset classification model. The convolutional layer is used to perform convolution processing on the input training image to obtain a corresponding feature vector. The parent classification layer is used to calculate a classification loss function based on the feature vector obtained after convolution, and then update the parameters of the preset classification model based on the classification loss function.

[0065] In one embodiment, a preset classification model is trained according to the direction of increasing distance between parent classification classes of the first group of training images to obtain a previous classification model, including steps 101 to 103.

[0066] Step 101: Input a first set of training images into a convolutional layer through an input layer and perform convolution processing to obtain a plurality of first feature vectors. For example, input a first training image and a second training image into a convolutional layer through an input layer and perform convolution processing to obtain a feature vector corresponding to the first training image and a feature vector corresponding to the second training image, respectively.

[0067] Step 102: Obtain multiple first eigenvectors through the parent classification layer and calculate a first loss function L1 based on the multiple first eigenvectors. For example, the first loss function L1 can be an arcloss loss function or a softmax loss function calculated based on the multiple first eigenvectors. Arcface loss is a loss function that uses the class margin to expand the distance between different classes.

[0068] Step 103: Update the parameters of the convolutional layer according to the first loss function L1, increase the distance between the first eigenvectors corresponding to training images with different parent categories in the first group of training images, and decrease the distance between the first eigenvectors corresponding to training images with the same parent category in the first group of training images, to obtain a pre-order classification model. For example, update the parameters of the convolutional layer according to the first loss function L1 to at least increase the distance between the eigenvector corresponding to the first training image and the eigenvector corresponding to the second training image, to obtain the pre-order classification model.

[0069] By updating the parameters of the convolutional layer according to the direction in which the first loss function L1 decreases, the distance between the first eigenvectors corresponding to the training pictures with different parent categories in the first group of training pictures is increased, and the distance between the first eigenvectors corresponding to the training pictures with the same parent category in the first group of training pictures is reduced.

[0070] Optionally, step 103 sequentially uses the softmax loss function and the arcloss loss function to train the preset classification model to complete the training of the basic model and obtain the pre-order model.

[0071] Step 10 can be used to make the feature vectors corresponding to images of different parent categories have a larger distance when the previous classification model processes the image. Figure 3 , Figure 3 This is a schematic diagram of the relationship between feature vectors obtained by using a pre-order classification model during the image classification process in one embodiment. Figure 3 It includes three parent categories and corresponds to three parent category feature vectors. Specifically, the first parent category feature vector includes the first child category feature vector 401, the second child category feature vector 402 and the third child category feature vector 403, the second parent category feature vector 50 and the third parent category feature vector 60. Among them, the distance between the first parent category feature vector, the second parent category feature vector 50 and the third parent category feature vector 60 is large, while the distance between the first child category feature vector 401, the second child category feature vector 402 and the third child category feature vector 403 is not large.

[0072] When the preset classification model is trained using the first set of training images, the parameters in the preset classification model can be updated according to the direction of increasing distance between parent classification classes, which is beneficial for the previous classification model to make the feature vectors corresponding to images with different parent classifications have a larger distance when processing images, so that the previous classification model has the ability to preliminarily classify images according to the parent classification, thereby improving the effect of subsequent training; and enabling the previous classification model to generate effective feature vectors, thereby realizing clustering and sub-label allocation.

[0073] In step 20, the pre-order classification model is trained using the second set of training images, and the image classification model is obtained by updating the parameters (weights) in the pre-order classification model and increasing the distance between the feature vectors of images with the same parent classification but different sub-classifications.

[0074] Each training image in the second set of training images has at least a label corresponding to a parent category. The second set of training images includes multiple training images. For example, the second set of training images includes a third training image and a fourth training image. The third and fourth training images have the same parent category but different subcategories. Specifically, the parameters of the previous classification model are updated based on at least the direction in which the feature vectors corresponding to the third and fourth training images increase, thereby obtaining an image classification model.

[0075] Optionally, the training pictures included in the second set of training pictures are the same as the training pictures included in the first set of training pictures. That is, the first set of training pictures also includes the third training picture and the fourth training picture.

[0076] Optionally, during the process of training the preset classification model based on the direction of increasing intra-class spacing of the subclasses of the second set of training images, the subclasses of the training images can be automatically determined using a clustering method. Specifically, training images with the same parent class are clustered to obtain pseudo-labels for the training images, and the subclasses of the training images are determined based on the pseudo-labels. For example, the subclasses of the third and fourth training images are respectively determined by clustering the second set of training images, assigning different pseudo-labels to the third and fourth training images based on the clustering results, and determining the subclasses of the third and fourth training images based on the pseudo-labels.

[0077] In one embodiment, a preset classification model is trained according to the direction of increasing inter-subclass spacing of the second set of training images to obtain an image classification model, including steps 201 to 203.

[0078] Step 201: Perform convolution processing on the second set of training images to obtain multiple second feature vectors. For example, perform convolution processing on the third training image and the fourth training image to obtain feature vectors corresponding to the third training image and feature vectors corresponding to the fourth training image, respectively.

[0079] Step 202: Obtain multiple sub-classification cluster centers based on the multiple second eigenvectors. For example, obtain the first cluster center and the second cluster center based on at least the eigenvector corresponding to the third training image and the eigenvector corresponding to the fourth training image.

[0080] Step 203: Update the parameters of the preset classification model based on the increasing direction of the distance between the multiple sub-classification cluster centers to obtain an image classification model. For example, update the parameters of the preset classification model based on at least the increasing direction of the distance between the first cluster center and the second cluster center to obtain an image classification model.

[0081] In one embodiment, obtaining multiple sub-classification cluster centers based on multiple second eigenvectors includes:

[0082] Step 221: determine the parent categories corresponding to the multiple second eigenvectors respectively;

[0083] Step 222: cluster the second eigenvectors with the same parent classification to obtain multiple subclassifications of the second eigenvectors;

[0084] Step 223: Calculate the average value of the second eigenvectors of the same sub-classifications respectively to obtain multiple sub-classification cluster centers.

[0085] The cluster center is the central feature vector obtained by clustering multiple feature vectors with the same parent category. The clustering process can be performed using the kmeans clustering method. Specifically, the kmeans clustering method divides multiple feature vectors with the same parent category into K subcategories based on the distances between them. The method aims to cluster feature vectors within a subcategory as closely as possible, while maximizing the distances between subcategories.

[0086] Optionally, the feature vectors of each sub-classification obtained after clustering are summed and averaged to obtain a cluster center. Each sub-classification corresponds to a cluster center. For example, due to certain differences between the third and fourth training images, two cluster centers can be obtained after clustering the corresponding feature vectors, where the third feature vector corresponds to the first cluster center and the fourth feature vector corresponds to the second cluster center. It can be understood that the third feature vector is closer to the first cluster center than the second cluster center.

[0087] The distance between cluster centers may be based on the cosine similarity between the cluster centers. For example, the distance between the first cluster center and the second cluster center may be based on the cosine similarity calculated between the first cluster center and the second cluster center.

[0088] In one application scenario, the number of subcategories K corresponding to the parent category is set separately. For example, it is determined based on the number of training images in the first group. Specifically, the number of subcategories is set to 1 / 5 of the number of training images of the corresponding parent category. For example, if there are 200 training images for the dog category, the number of dog subcategories is set to 40. The second eigenvectors are extracted from the 200 dog training images using the pre-order model. The obtained second eigenvectors are clustered using kmeans, where k is the number of subcategories, 40. A pseudo label is assigned to each subcategory obtained by clustering, such as dog category 1, dog category 2, dog category 3, etc. In this way, each pseudo label corresponds to a subcategory under each parent category. At the same time, the average value of the multiple second eigenvectors corresponding to each subcategory is calculated to obtain the cluster center Fc corresponding to each subcategory. The average value of the multiple second eigenvectors corresponding to the subcategory can be calculated by performing a positional addition and then calculating the average value of the second eigenvectors with the same subcategory.

[0089] In one embodiment, the process of updating the parameters in the preceding classification model may further include: step 204, updating the parameters in the preceding classification model based on the direction in which the distance between the feature vector of the same subclassification and the cluster center of the corresponding subclassification decreases. For example, the parameters of the preset classification model are updated based on the direction in which the distance between the third feature vector and the first cluster center decreases, and the direction in which the distance between the fourth feature vector and the second cluster center decreases.

[0090] The distance between the third eigenvector and the first cluster center may be a cosine similarity calculated based on the third eigenvector and the first cluster center, and the distance between the fourth eigenvector and the second cluster center may be a cosine similarity calculated based on the fourth eigenvector and the second cluster center.

[0091] In one embodiment, the process of updating the parameters in the preceding classification model may further include: step 205, updating the parameters in the preceding classification model based on the direction in which the distance between the cluster center of the subclassification and the preset cluster center of the subclassification decreases. For example, a first preset cluster center and a second preset cluster center are obtained, and the parameters of the preset classification model are updated based on at least the direction in which the distance between the first cluster center and the first preset cluster center decreases, and the direction in which the distance between the second cluster center and the second preset cluster center decreases, wherein the distance between the first preset cluster center and the second preset cluster center is greater than the preset distance.

[0092] Among them, the preset cluster centers of the subcategories may be preset centers distributed around the corresponding parent category center according to preset rules, for example, the distance between the preset cluster centers of each subcategory and the corresponding parent category center is a preset distance.

[0093] By updating the parameters of the pre-order classification model based on the direction in which the distance between the sub-classification cluster center and the preset sub-classification cluster center decreases, the sub-classification cluster center is facilitated to approach, or even overlap, the preset sub-classification cluster center, thereby controlling the distance between the cluster centers of each sub-classification. In one embodiment, the pre-order classification model includes an input layer, a convolutional layer, a parent classification layer, and multiple sub-classification layers.

[0094] The parent classification layer calculates the corresponding loss function based on the parent classification of the training image. The child classification layer clusters the feature vectors belonging to the same parent classification and calculates the corresponding loss function based on the clustering results. Both the parent classification layer and the child classification layer can be softmax classification layers, but the parameters in the parent classification layer and the first child classification layer are different. The feature vectors input to the parent classification layer can be all feature vectors, while the feature vectors input to the child classification layer are the feature vectors corresponding to the subclass and do not include feature vectors of other subclasses.

[0095] Optionally, the parent classification layer and each sub-classification layer are arranged in parallel. For example, the convolutional layer is connected to the parent classification layer and multiple sub-classification layers respectively.

[0096] In one embodiment, a preset classification model is trained in the direction of increasing intra-subclass spacing of the second set of training images to obtain an image classification model, including steps 21 to 26.

[0097] Step 21: Input the second set of training images into the convolution layer through the input layer for convolution processing, and obtain multiple second eigenvectors. For example, input the third and fourth training images into the convolution layer through the input layer for convolution processing, and obtain two corresponding second eigenvectors.

[0098] Step 22: Obtain multiple sub-classification cluster centers based on the multiple second eigenvectors.

[0099] Due to the training in step 10, the preceding classification model can determine the corresponding parent classification according to the plurality of second eigenvectors. Specifically, the plurality of second eigenvectors can be analyzed through the parent classification layer to obtain the parent classifications corresponding to the plurality of second eigenvectors.

[0100] Since the second eigenvectors corresponding to training images of different sub-categories also have certain differences, multiple second eigenvectors of the same parent category can be clustered to obtain multiple sub-categories and corresponding sub-category cluster centers. Specifically, multiple second eigenvectors of the same parent category can be clustered at the sub-category level to obtain corresponding sub-categories and sub-category cluster centers.

[0101] Step 23: Obtain multiple second eigenvectors through the parent classification layer, and obtain the second loss function L2 based on the second eigenvectors corresponding to the parent classification.

[0102] The direction in which the second loss function L2 decreases is a direction in which the distance between second eigenvectors of different parent categories increases, and / or a direction in which the distance between second eigenvectors of the same parent category decreases.

[0103] Optionally, the second loss function L2 can be calculated based on the multiple second eigenvectors by an arcloss algorithm (arcfaceloss algorithm), wherein the multiple second eigenvectors obtained by the parent classification layer are all the second eigenvectors generated by the second set of training images.

[0104] Step 24: Multiple sub-classification layers respectively obtain corresponding second eigenvectors, and obtain a third loss function L3 based on the second eigenvectors corresponding to the sub-classifications.

[0105] The direction in which the third loss function L3 decreases is the direction in which the distance between the second eigenvectors of different subcategories increases, and / or the direction in which the distance between the second eigenvectors of the same subcategories decreases. Optionally, the third loss function L3 can be calculated based on the multiple second eigenvectors using a multi-category cross entropy algorithm (such as a softmax cross entropy algorithm).

[0106] Step 25: Multiple sub-classification layers respectively obtain corresponding second eigenvectors, and obtain a fourth loss function L4 based on the sub-classification cluster center.

[0107] Optionally, the fourth loss function L4 is related to the distance between the centers of each sub-classification cluster, wherein the direction in which the fourth loss function L4 decreases can be the direction in which the distance between the centers of each sub-classification cluster decreases, or the direction in which the absolute value of the difference between the distance between the centers of each sub-classification cluster and the preset distance decreases.

[0108] In one embodiment, a fourth loss function is obtained based on the sub-classification cluster centers, including:

[0109] Step 251: Obtain the parent classification cluster center according to the multiple second eigenvectors;

[0110] Step 252: Calculate a fourth loss function based on the distance between the sub-classification cluster center and the corresponding parent classification cluster center.

[0111] The parent classification cluster center may be an average value of multiple second eigenvectors with the same parent classification.

[0112] The fourth loss function L4 is related to the distance between each sub-classification cluster center and the corresponding parent classification cluster center. For example, the direction in which the fourth loss function L4 decreases can be the direction in which the distance between each sub-classification cluster center and the corresponding parent classification cluster center decreases. Specifically, the fourth loss function L4 is the absolute value of the difference between each sub-classification cluster center and the corresponding parent classification cluster center.

[0113] Step 26: Update the parameters of the preset classification model according to the direction in which the target loss function decreases to obtain an image classification model, where the target loss function is positively correlated with the second loss function, the third loss function, and the fourth loss function, respectively.

[0114] The calculation formula of the objective loss function can be:

[0115] L t =L2+αL3+βL4

[0116] Among them, L t is the target loss function, L2 is the second loss function, L3 is the third loss function, L4 is the fourth loss function, α is the first loss adjustment parameter, and β is the second loss adjustment parameter.

[0117] In the formula, the direction in which L2 decreases is the direction in which the difference between the feature vectors corresponding to training images of different parent categories increases and the direction in which the difference between the feature vectors corresponding to training images of the same parent category decreases; the direction in which αL3 decreases is the direction in which the difference between the feature vectors corresponding to training images of different subcategories increases and the direction in which the difference between the feature vectors corresponding to training images of the same subcategories decreases; the direction in which βL4 decreases is the direction in which the distance between each subcategory cluster center and the corresponding parent category cluster center decreases.

[0118] Optionally, L4 is the normalized cosine distance between each sub-classification cluster center and the corresponding parent classification cluster center, which can be calculated by ||Fc_f-Fc_s|| 2 , Fc_f is the parent classification cluster center, and Fc_s is the child classification cluster center.

[0119] Optionally, the value of the second loss function is ten or dozens of times greater than the value of the third loss function and the value of the fourth loss function, which helps to reduce the negative impact of the classification performance caused by the excessive size of the third loss function and the fourth loss function, wherein the fourth loss function can constrain the feature vector offset that may be caused during fine-tuning.

[0120] In one application scenario, multiple pet training images are grouped and fed into a pre-order classification model. Each group contains 48 pet training images, with 16 cat, 16 dog, and 16 horse images. For each of the 48 training images, 48 ​​corresponding second eigenvectors are obtained. The arcloss loss function is derived from the differences between the 48 second eigenvectors, yielding the second loss function. The softmax multi-classification cross entropy is then calculated for each of the 48 second eigenvectors, yielding the third loss function. For example, the second eigenvectors of the 16 parent classes of the cat class are input into the cat subclassification layer to calculate the softmax multi-class cross entropy of the cat subclass. Similarly, the softmax multi-class cross entropy of the dog subclass and the softmax multi-class cross entropy of the horse subclass are calculated. The softmax multi-class cross entropy of all subclasses is added together to obtain the third loss function. At the same time, clustering is performed according to the kmeans clustering method, and the cluster center Fc_s of each subclass and the cluster center Fc_f of each parent class are obtained. The difference between each subclass cluster center Fc_s and the corresponding parent class cluster center Fc_f is used as an additional loss function, that is, the fourth loss function. This fourth loss function is used to ensure that the eigenvector distribution of the overall model does not change. After each round of training, it is necessary to regenerate pseudo labels for multiple second eigenvector clusters and assign pseudo labels, and recalculate the subclass cluster centers.

[0121] Step 30 is to classify the image to be classified by the image classification model, which can make the parent classification classes have a larger distance and increase the distance within the sub-classification class, thereby improving the classification accuracy.

[0122] The artificial intelligence classification method for images in this embodiment obtains a pre-order classification model by inputting a first group of training images into a preset classification model, and updating the parameters in the preset classification model according to the direction of increasing spacing between parent classification classes. This is beneficial to ensuring that the pre-order classification model makes the feature vectors corresponding to images with different parent classifications have large differences when processing images. Then, the second group of training images is input into the pre-order classification model, and the parameters in the pre-order classification model are updated according to the direction of increasing spacing within sub-classification classes. This is beneficial to obtaining an image classification model that makes the feature vectors corresponding to images with different parent classifications have large differences when processing images, and further increases the difference between the feature vectors corresponding to images with the same parent classification but different sub-classifications on the premise that the feature vectors corresponding to images with different parent classifications have large differences when processing images, thereby achieving finer image classification and improving the accuracy of image classification.

[0123] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0124] Figure 5 FIG. 1 is a structural block diagram of an image classification system according to an embodiment. Figure 5 As shown, the image classification system includes:

[0125] A first acquisition module 70 is configured to train a preset classification model according to a direction in which the inter-class spacing of the parent classification of the first set of training images increases, to obtain a preceding classification model, wherein the inter-class spacing of the parent classification is the distance between feature vectors corresponding to training images of different parent classifications in the first set of training images;

[0126] A second acquisition module 80 is configured to train the previous classification model according to a direction of increasing intra-subclass spacing of the second set of training images to obtain an image classification model, wherein the intra-subclass spacing is the distance between feature vectors corresponding to training images of the second set of training images that have the same parent class but different subclasses;

[0127] The classification module 90 is used to classify the image to be classified using an image classification model to obtain a classification result.

[0128] In one embodiment, the first acquisition module 70 is also used to input the first group of training pictures into the convolution layer through the input layer and perform convolution processing to obtain multiple first eigenvectors respectively; obtain multiple first eigenvectors through the parent classification layer, and calculate the first loss function based on the multiple first eigenvectors; update the parameters of the convolution layer according to the first loss function, increase the distance between the first eigenvectors corresponding to the training pictures with different parent classifications in the first group of training pictures, and reduce the distance between the first eigenvectors corresponding to the training pictures with the same parent classification in the first group of training pictures, so as to obtain a pre-order classification model.

[0129] In one embodiment, the second acquisition module 80 is also used to perform convolution processing on the second group of training images to obtain multiple second eigenvectors respectively; obtain multiple sub-classification cluster centers based on the multiple second eigenvectors; update the parameters of the preset classification model according to the direction of increasing distance between the multiple sub-classification cluster centers to obtain an image classification model.

[0130] In one embodiment, the second acquisition module 80 is also used to respectively determine the parent categories corresponding to multiple second eigenvectors; cluster the second eigenvectors with the same parent category to obtain subcategories of multiple second eigenvectors; and calculate the average values ​​of the second eigenvectors with the same subcategories to obtain multiple subcategories cluster centers.

[0131] In one embodiment, the second acquisition module 80 is also used to input the second group of training images into the convolution layer through the input layer for convolution processing, and obtain multiple second eigenvectors respectively; according to the multiple second eigenvectors, obtain the parent categories and corresponding subcategories corresponding to the multiple second eigenvectors; obtain multiple second eigenvectors through the parent category layer, and obtain the second loss function according to the difference between the second eigenvectors of different parent categories; obtain the corresponding second eigenvectors of multiple subcategories respectively, and obtain the third loss function according to the difference between the second eigenvectors of different subcategories; obtain the corresponding second eigenvectors of multiple subcategories respectively, calculate multiple subcategories cluster centers, and obtain the fourth loss function according to the subcategories cluster centers; according to the direction of reduction of the target loss function, update the parameters of the preset classification model to obtain the image classification model, and the target loss function is positively correlated with the second loss function, the third loss function and the fourth loss function respectively.

[0132] In one embodiment, the second acquisition module 80 is further configured to acquire a parent classification cluster center based on multiple second eigenvectors; and calculate a fourth loss function based on a distance between a child classification cluster center and a corresponding parent classification cluster center.

[0133] The image classification system can update the parameters in the preset classification model according to the direction of increasing spacing between parent classification classes to obtain a pre-order classification model, which is conducive to ensuring that the pre-order classification model makes the feature vectors corresponding to images with different parent classifications have large differences when processing images, and then update the parameters in the pre-order classification model according to the direction of increasing spacing within the sub-classification class, which is conducive to ensuring that the feature vectors corresponding to images with different parent classifications have large differences when processing images, and further increase the difference between the feature vectors corresponding to images with the same parent classification but different sub-classifications, thereby achieving finer image classification and improving the accuracy of image classification.

[0134] The division of the modules in the above image classification system is only for illustration. In other embodiments, the image classification system may be divided into different modules as needed to complete all or part of the functions of the above image classification system.

[0135] For the specific definition of the image classification system, please refer to the definition of the artificial intelligence classification method for images above, which will not be repeated here. Each module in the above image classification system can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0136] Each module in the image classification system provided in the embodiments of the present application may be implemented in the form of a computer program. The computer program may be executed on a terminal or server. The program modules comprising the computer program may be stored in a memory on the terminal or server. When the computer program is executed by a processor, the steps of the method described in the embodiments of the present application are implemented.

[0137] This embodiment provides an electronic device including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the above-mentioned artificial intelligence image classification method.

[0138] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed on a computer, it causes the computer to perform the above-described artificial intelligence image classification method. Specifically, the computer-readable storage medium comprises one or more non-volatile computer-readable storage media containing computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the processors perform the steps of the artificial intelligence image classification method.

[0139] A computer program product comprising instructions which, when executed on a computer, causes the computer to perform an artificial intelligence image classification method.

[0140] As used herein, any reference to memory, storage, database, or other medium may include nonvolatile and / or volatile memory. Nonvolatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0141] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. An artificial intelligence classification method for images, characterized in that: include: The preset classification model is trained according to the direction of increasing inter-class spacing of the parent classification of the first group of training pictures to obtain a pre-order classification model, including: inputting the first group of training pictures into the convolution layer of the preset classification model through the input layer of the preset classification model and performing convolution processing to obtain multiple first eigenvectors respectively, obtaining the multiple first eigenvectors through the parent classification layer of the preset classification model, and calculating a first loss function according to the multiple first eigenvectors, and updating the parameters of the convolution layer of the preset classification model according to the first loss function, increasing the distance between the first eigenvectors corresponding to training pictures with different parent classifications in the first group of training pictures, and reducing the distance between the first eigenvectors corresponding to training pictures with the same parent classification in the first group of training pictures, to obtain the pre-order classification model; wherein the inter-class spacing of the parent classification is the distance between the eigenvectors corresponding to training pictures with different parent classifications in the first group of training pictures; The pre-order classification model is trained according to the direction of increasing distance within the sub-classification class of the second group of training pictures to obtain an image classification model, including: inputting the second group of training pictures into the convolution layer of the pre-order classification model through the input layer of the pre-order classification model for convolution processing, respectively obtaining a plurality of second eigenvectors, and then obtaining a plurality of sub-classification clustering centers according to the plurality of second eigenvectors; obtaining the plurality of second eigenvectors through the parent classification layer of the pre-order classification model, and obtaining a second loss function according to the second eigenvector corresponding to the parent classification, the plurality of sub-classification layers respectively obtain corresponding second eigenvectors, and according to the corresponding second eigenvectors of the sub-classification, obtain a second loss function; a second eigenvector, obtaining a third loss function, the multiple sub-classification layers respectively obtaining corresponding second eigenvectors, calculating multiple sub-classification cluster centers, and obtaining a fourth loss function according to the sub-classification cluster centers, and updating the parameters of the preset classification model according to the direction in which the target loss function decreases, to obtain an image classification model; wherein the target loss function is positively correlated with the second loss function, the third loss function, and the fourth loss function, respectively, the intra-class is the intra-class of the parent classification, and the intra-sub-class spacing is the distance between the eigenvectors corresponding to the training pictures with the same parent classification but different sub-classifications in the second group of training pictures; The image to be classified is classified by the image classification model to obtain a classification result.

2. The artificial intelligence classification method for images according to claim 1, characterized in that: The step of training the preset classification model according to the direction of increasing inter-subclass spacing of the second set of training images to obtain an image classification model includes: Obtaining a plurality of second feature vectors according to the second set of training images; Obtaining a plurality of sub-classification cluster centers according to the plurality of second eigenvectors; According to the direction in which the distances between the centers of the multiple sub-classification clusters increase, the parameters of the preset classification model are updated to obtain an image classification model.

3. The artificial intelligence classification method for images according to claim 2, characterized in that: The step of obtaining a plurality of sub-classification cluster centers according to the plurality of second eigenvectors includes: respectively determining parent categories corresponding to the plurality of second feature vectors; Clustering the second eigenvectors with the same parent classification to obtain multiple subclassifications with the second eigenvectors; The average values ​​of the second eigenvectors of the same sub-classifications are calculated respectively to obtain multiple sub-classification cluster centers.

4. The artificial intelligence classification method for images according to claim 1, characterized in that: The fourth loss function is obtained according to the sub-classification cluster center, including: Obtaining parent classification cluster centers according to multiple second eigenvectors; The fourth loss function is calculated based on the distance between the sub-classification cluster center and the corresponding parent classification cluster center.

5. The artificial intelligence classification method for images according to claim 1, characterized in that: The third loss function is a multi-class cross entropy loss function.

6. An image classification system, characterized in that include: A first acquisition module is used to train the preset classification model according to the direction of increasing inter-class spacing of the parent classification of the first group of training pictures to obtain a pre-order classification model, including: inputting the first group of training pictures into the convolution layer of the preset classification model through the input layer of the preset classification model and performing convolution processing to obtain multiple first eigenvectors respectively, obtaining the multiple first eigenvectors through the parent classification layer of the preset classification model, and calculating a first loss function according to the multiple first eigenvectors, and updating the parameters of the convolution layer of the preset classification model according to the first loss function, increasing the distance between the first eigenvectors corresponding to the training pictures with different parent classifications in the first group of training pictures, and reducing the distance between the first eigenvectors corresponding to the training pictures with the same parent classification in the first group of training pictures, to obtain the pre-order classification model; wherein the inter-class spacing of the parent classification is the distance between the eigenvectors corresponding to the training pictures with different parent classifications in the first group of training pictures; The second acquisition module is used to train the pre-order classification model according to the direction of increasing intra-sub-class spacing of the second group of training pictures to obtain an image classification model, including: inputting the second group of training pictures into the convolution layer of the pre-order classification model through the input layer of the pre-order classification model for convolution processing, respectively obtaining a plurality of second eigenvectors, and then obtaining a plurality of sub-classification cluster centers according to the plurality of second eigenvectors; obtaining the plurality of second eigenvectors through the parent classification layer of the pre-order classification model, and obtaining a second loss function according to the second eigenvector corresponding to the parent classification, the plurality of sub-classification layers respectively obtain corresponding second eigenvectors, and obtain a second loss function according to the sub-classification The second eigenvector corresponding to the class is obtained to obtain a third loss function, the multiple sub-classification layers respectively obtain the corresponding second eigenvectors, calculate multiple sub-classification cluster centers, and obtain a fourth loss function according to the sub-classification cluster centers, and according to the direction in which the target loss function decreases, the parameters of the preset classification model are updated to obtain an image classification model; wherein the target loss function is positively correlated with the second loss function, the third loss function and the fourth loss function, respectively, the intra-class is the intra-class of the parent class, and the intra-sub-class spacing is the distance between the eigenvectors corresponding to the training pictures with the same parent class but different sub-classes in the second group of training pictures; The classification module is used to classify the image to be classified using the image classification model to obtain a classification result.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the artificial intelligence image classification method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed on a computer, the computer is caused to perform the steps of the artificial intelligence classification method for images according to any one of claims 1 to 5.

Images