Picture classification model training method, classification method, device, equipment and medium

Abstract

The present disclosure provides a picture classification model training method, a classification method, an apparatus, a device and a medium, the method comprising: inputting picture samples in a pre-acquired training data set into a picture classification model to obtain predicted categories of the picture samples; wherein the picture classification model comprises multiple expert sub-networks, and the step of inputting the picture samples into the picture classification model for category prediction comprises: performing feature extraction on the picture samples respectively through the multiple expert sub-networks to obtain picture features extracted by each expert sub-network; performing feature classification according to the picture features extracted by the multiple expert sub-networks to obtain the predicted categories of the picture samples; constructing a target loss function based on the predicted categories, labeled categories of the picture samples and N angles; and adjusting parameters of the picture classification model according to the target loss function to obtain a trained picture classification model. According to the embodiments of the present disclosure, the prediction accuracy of the model for picture classification can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a training method, classification method, apparatus, device, and medium for an image classification model. Background Technology

[0002] In the field of computer technology, the datasets used to train models often exhibit imbalance, meaning that images of a few classes contain a large number of samples (defined as head classes), while images of most classes contain only a small number of samples (defined as tail classes). In this case, the dataset can be called long-tailed data, or a dataset with a long-tailed distribution.

[0003] Due to this imbalance in long-tailed data, the trained model tends to perform well in classifying head-type images but has lower accuracy in classifying tail-type images, thus affecting the overall accuracy of image classification. Summary of the Invention

[0004] This disclosure provides a training method, classification method, apparatus, device, and medium for an image classification model, which can improve the model's prediction accuracy for image classification.

[0005] In a first aspect, this disclosure provides a training method for an image classification model. The training method includes: inputting image samples from a pre-acquired training dataset into the image classification model to obtain the predicted category of the image samples; wherein the image classification model includes multiple expert sub-networks, and the step of inputting the image samples into the image classification model for category prediction includes: extracting features from the image samples through the multiple expert sub-networks respectively to obtain image features extracted by each expert sub-network; performing feature classification based on the image features extracted by the multiple expert sub-networks to obtain the predicted category of the image samples; constructing a target loss function based on the predicted category, the labeled category of the image samples, and N angles, wherein the N angles are determined based on the image features extracted by each expert sub-network, and N is an integer greater than or equal to 1; and adjusting the parameters of the image classification model according to the target loss function to obtain the trained image classification model.

[0006] Secondly, this disclosure provides an image classification method, which includes: acquiring an image to be classified; inputting the image to be classified into an image classification model for category prediction to obtain a predicted category of the image to be classified; wherein the image classification model includes multiple expert sub-networks, and the step of inputting the image to be classified into the image classification model for category prediction includes: extracting features from the image sample through the multiple expert sub-networks respectively to obtain image features extracted by each expert sub-network; and performing feature classification based on the image features extracted by the multiple expert sub-networks to obtain a predicted category of the image sample.

[0007] Thirdly, this disclosure provides an image classification device, which includes: an acquisition module for acquiring an image to be classified; and a obtaining module for inputting the image to be classified into an image classification model for category prediction to obtain a predicted category of the image to be classified; wherein the image classification model includes multiple expert sub-networks, and the step of inputting the image to be classified into the image classification model for category prediction includes: extracting features from the image sample through the multiple expert sub-networks respectively to obtain image features extracted by each expert sub-network; and performing feature classification based on the image features extracted by the multiple expert sub-networks to obtain a predicted category of the image sample.

[0008] Fourthly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described image classification model training method or image classification method.

[0009] Fifthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor / processor core, implements the above-described image classification model training method or image classification method.

[0010] The embodiments provided in this disclosure can train an image classification model using image samples from an image training dataset. During model training, the difference between the predicted and labeled categories of the image samples is used to determine the prediction accuracy of the classification. The angle determined based on the image features extracted by each expert sub-network represents the degree of difference in the image features learned by different expert sub-networks. During model training, a loss function is constructed based on the predicted category, the labeled category of the image sample, and the angle between the image features extracted by different expert sub-networks. This can address the problem of uneven distribution of feature differences learned in a multi-expert network architecture by using the angle between features while ensuring image classification accuracy. This results in a well-balanced image classification model, improving the model's classification performance for all image categories and thus improving the model's prediction accuracy for image classification.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0013] Figure 1 A flowchart illustrating a training method for an image classification model provided in this embodiment of the disclosure;

[0014] Figure 2 This diagram illustrates feature learning during model training via an expert network in related technologies.

[0015] Figure 3 A flowchart illustrating a training method for an image classification model provided in this embodiment of the disclosure;

[0016] Figure 4 A schematic diagram illustrating the training process of the image classification model provided in this embodiment of the disclosure;

[0017] Figure 5 A schematic diagram illustrating the training process of an image classification model provided in another embodiment of this disclosure;

[0018] Figure 6 A flowchart of an image classification method provided in this embodiment of the disclosure;

[0019] Figure 7A block diagram of a training device for an image classification model provided in an embodiment of this disclosure;

[0020] Figure 8 A block diagram of an image classification device provided in an embodiment of this disclosure;

[0021] Figure 9 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0024] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, they specify the presence of features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0026] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0027] In this embodiment of the disclosure, long-tail distribution refers to a small subset of categories in the dataset that constitute the majority of the sample size, while the remaining majority of categories in the dataset constitute a small subset of the sample size; head category refers to a small subset of categories that constitute the majority of the sample size in a long-tail distribution task; tail category refers to a majority of categories that constitute a small subset of the sample size in a long-tail distribution task; long-tail ratio refers to the ratio of the maximum to the minimum sample size of a category.

[0028] In some embodiments, a data sample is defined as (x, y), representing that data sample x has a corresponding classification label y, the total number of categories in the dataset is defined as N, and the number of images for each category is n. i The total number of images across all categories is SUM(n). i If we sum the results, then the number of images in each category is {n1, n2, ..., n}. N The long-tail ratio is defined as max(n). i ) / min(n i ), where max(n) i ) represents the maximum number of samples for each category, min(n) i The long-tail ratio represents the minimum sample size for each category. A higher long-tail ratio indicates a more severe imbalance in the data distribution within a long-tailed dataset.

[0029] For example, suppose there are 10 categories. In a dataset, 80% of the images cover only 2 categories, while 20% of the images cover 8 categories. If we directly use such a data distribution to train a neural network, it will often overfit the head data and ignore the tail categories when making predictions. Therefore, it is necessary to effectively use the imbalanced long-tail data to train a neural network with balanced capabilities.

[0030] Figure 1 This diagram illustrates a set of image data exhibiting a long-tailed distribution. Figure 1 In this approach, classes with a larger sample size are defined as head classes in long-tail tasks, while classes with a smaller sample size are defined as tail classes.

[0031] Figure 1 The horizontal axis represents the different categories of the long-tail distribution; smaller numbers on the horizontal axis represent head categories, and larger numbers represent tail categories. The vertical axis represents the number of image samples for the corresponding category. Figure 1 It can be seen that the vast majority of samples in the image dataset belong to the head category, a small number of categories correspond to a high number of image samples, and most categories correspond to a low number of image samples. The image samples in the image dataset have a long tail distribution, that is, there is a class imbalance phenomenon.

[0032] In related technologies, model fusion methods are widely used in machine learning. By fusing multiple models for prediction, better results can be achieved than with a single model. Through model fusion, learning curves can be studied to determine if the model is overfitting or underfitting, and corresponding adjustments can be made. Furthermore, model weight parameters can be analyzed; features with high / low absolute weights can be further refined, and feature combinations can be performed. Therefore, model fusion methods are often used for training models on long-tailed datasets.

[0033] Traditional model fusion methods involve linearly fusing the results from multiple models, typically combining models with significant differences. To obtain models with substantial differences, three common approaches are used: First, using the same network architecture with different training datasets or the same dataset after different preprocessing. Second, training on the exact same dataset but using models with different network architectures. Third, using the same network architecture and training set but with different initialization parameters during training.

[0034] Traditional model fusion and multi-expert learning networks differ primarily in the following aspects: First, regarding data, model fusion uses different data in specific iterations during training, with each branch learning samples in a different order and undergoing different preprocessing. In contrast, multiple branches of a multi-expert network use the same sample, learning samples in the same order, and with identical data processing methods. Second, regarding networks, model fusion typically uses different networks, even when using the same network, with different parameters or initialization parameters. Multi-expert networks use the same network architecture, sharing only the front-end convolutional network; the rear convolutional networks and classification layers are separate, each optimizing its own parameters, with only a small portion of the network's front end being jointly optimized. Third, regarding fusion, traditional model fusion often involves simple joint operations, while multi-expert networks employ various strategies to ensure significant differences in the knowledge learned by different experts, thereby ensuring the entire network learns sufficient knowledge. In the image classification model training method of this disclosure embodiment, a multi-expert learning network can be used for model training.

[0035] Figure 2 This diagram illustrates feature learning during model training using an expert network in related technologies. Figure 2 In the diagram, the horizontal axis represents the distribution range of the sample data, and the vertical axis represents the feature values ​​learned for different sample data.

[0036] like Figure 2As shown, the expert network includes multiple expert subnetworks, such as expert 1, expert 2 and expert 3, which represent the first expert subnetwork, the second expert subnetwork and the third expert subnetwork, respectively.

[0037] In some embodiments, each expert subnetwork can be used to train a model on the sample data first, and then the trained expert subnetworks can be fused. Since the model parameters of multiple expert subnetworks are initially randomly initialized, and then a loss function is used to widen the gap between the features learned by each expert subnetwork, this can lead to significant differences in the distance between the features learned by multiple expert subnetworks. When the distance between the features learned by some expert subnetworks (expert 1 and expert 2) is too close, the knowledge learned by these expert subnetworks will overlap, resulting in the entire expert network learning less knowledge and still failing to train a network model with balanced capabilities.

[0038] The training method for the image classification model according to embodiments of this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Servers can include independent physical servers, server clusters consisting of multiple servers, or cloud servers capable of cloud computing.

[0039] Figure 3 A flowchart illustrating a training method for an image classification model provided in this embodiment of the disclosure. (Refer to...) Figure 3 The training method for this image classification model may include the following steps.

[0040] S310, input the image samples from the pre-acquired training dataset into the image classification model to obtain the predicted category of the image samples; wherein, the image classification model includes multiple expert sub-networks, and the steps of inputting the image samples into the image classification model for category prediction include: extracting features from the image samples through multiple expert sub-networks respectively to obtain the image features extracted by each expert sub-network; performing feature classification based on the image features extracted by multiple expert sub-networks to obtain the predicted category of the image samples.

[0041] In some embodiments, the image samples in the image training dataset exhibit a long-tailed distribution. A long-tailed distribution is a type of skewed distribution. For the image training dataset of this disclosure, if it exhibits other types of skewed distribution or a normal distribution, it can also be used to train the image classification model of this disclosure. Therefore, the training method for the image classification model of this disclosure is not limited by the type of sample distribution.

[0042] In this step, multiple numbers represent integers greater than or equal to 2; that is, the multiple expert subnetworks include at least two expert subnetworks. In some embodiments, each expert subnetwork may have the same network structure, and each expert subnetwork may perform feature extraction through a feature extraction network, which can be any network structure with feature extraction capabilities. For example, a Deep Residual Network (ResNet).

[0043] In this step, image features extracted by multiple expert subnetworks can be input into a feature classification network. The feature classification network processes the image features extracted by the multiple expert subnetworks to obtain the predicted category of the image sample. The feature classification network can be any network structure with feature classification functionality, and this embodiment does not impose any specific limitations.

[0044] S320 constructs a target loss function based on the predicted category, the labeled category of the image sample, and N angles. The N angles are determined based on the image features extracted by each expert sub-network, and N is an integer greater than or equal to 1.

[0045] In this embodiment, the loss function can be used to estimate the gap between the model training result and the model training objective. In this step, based on the gap between the predicted category obtained by the image classification model in each training iteration and the labeled category of the input image sample, the loss between the predicted category and the true labeled category can be determined. In this step, based on the angle between the features formed by the image features extracted by each expert sub-network, the differences between the image features learned by different expert sub-networks can be determined.

[0046] S330: Adjust the parameters of the image classification model according to the loss function to obtain the trained image classification model.

[0047] In this embodiment of the disclosure, the objectives of parameter adjustment for the image classification model may include the following two aspects: First, minimizing the loss between the predicted category and the true labeled category, thereby improving the prediction accuracy of the image classification model; Second, by adjusting the angle between the image features extracted by different expert sub-networks, the distance between the features learned by different expert sub-networks can be avoided from being too close, which is conducive to training a network model with balanced capabilities.

[0048] According to the image classification model training method of this disclosure, the image classification model can be trained using image samples in the image training dataset. During the model training process, the difference between the predicted category and the labeled category of the image sample is used to determine the prediction accuracy of the classification. The angle determined based on the image features extracted by each expert sub-network represents the degree of difference in the image features learned by different expert sub-networks. During the model training process, a loss function is constructed based on the predicted category, the labeled category of the image sample, and the angle between the image features extracted by different expert sub-networks. This can address the problem of uneven distribution of feature differences learned in the multi-expert network architecture by using the angle between features while ensuring image classification accuracy. This results in a well-balanced image classification model, improving the model's classification performance for all image categories and thus improving the model's prediction accuracy for image classification.

[0049] In some embodiments, the image samples are obtained in the following manner:

[0050] S11, loads image samples from the image training dataset using a random strategy via the data loader.

[0051] In this step, training the network model often involves a large amount of complex data, and each network model's framework usually has a specific data format. A data loader can process the image samples in the image training dataset into the data format required by the image classification model's network framework. For example, a data loader can load and normalize a large number of image samples in the image training dataset in batches, and can also extract corresponding image samples from the image training dataset using a pre-established image index.

[0052] S12, perform data augmentation on the loaded image samples according to at least one data augmentation method to obtain the augmented image samples corresponding to each data augmentation method.

[0053] In this step, image enhancement refers to processing image features, such as edges, contours, and contrast, using a certain image processing method to improve the visual effect of the image, enhance its clarity, or highlight certain information in the image, transforming the image into a form more suitable for human or computer analysis and processing.

[0054] In some embodiments, data augmentation may include at least one of the following: random cropping, random occlusion, and illumination enhancement.

[0055] Random cropping refers to randomly selecting a point within the image's pixel range and cropping a fixed volume of the image. Random occlusion refers to randomly occluding a portion of a pattern (such as an animal or plant) in an image, leaving that part blank. Random cropping and occlusion can remove irrelevant patterns from image samples, such as removing interference areas. Illumination enhancement is used to improve image samples with uneven lighting. It helps address issues such as low overall pixel intensity, low contrast, and the loss of detail caused by insufficient light intensity or short exposure time, thus improving model training performance.

[0056] In other embodiments, data augmentation may also include at least one of rotating, flipping, or scaling the image sample. Rotation refers to randomly rotating the image sample around the z-axis by a certain angle. Flipping refers to rotating the image around the x-axis or y-axis by 180°. Scaling refers to interpolating the image to enlarge or reduce it.

[0057] In this embodiment, enhanced image samples (hereinafter referred to as enhanced samples) are obtained through data augmentation processing, and each enhanced sample can be used as an independent image sample. The more types of augmentation methods applied to the image samples, the more enhanced samples are obtained. The expert sub-network can learn more information from the enhanced samples, thus enabling the fusion of more feature information when fusing the features output by each expert sub-network in the subsequent process. For example, assuming that the image classification model includes three expert sub-networks and there are three data augmentation methods, then for an image sample, data augmentation processing is performed using the three augmentation methods respectively, resulting in three enhanced samples. The three enhanced samples are then input into each of the three expert sub-networks for feature extraction, resulting in nine different image features.

[0058] In the embodiments of this disclosure, data augmentation methods can be selected according to actual needs to improve the quality of image samples, and also to increase the total number of image samples and the diversity of learned features.

[0059] For ease of understanding, in the following embodiments of this disclosure, in conjunction with Figure 4 and Figure 5 This describes the training process of an image classification model. Figure 4 A schematic diagram illustrating the training process of the image classification model provided in this embodiment of the disclosure; Figure 5 This is a schematic diagram illustrating the training process of an image classification model provided in another embodiment of the present disclosure.

[0060] exist Figure 4 and Figure 5 In the image classification model, a multi-expert network architecture is used, which schematically shows three expert sub-networks: the first expert sub-network, the second expert sub-network, and the third expert sub-network. Each expert sub-network includes a convolutional layer and a classification layer. The convolutional layer of each expert sub-network includes a shared part and an independent part.

[0061] In some embodiments, step S310, which involves extracting features from image samples using multiple expert sub-networks to obtain image features extracted by each expert sub-network, may specifically include: performing convolution processing on image samples using a shared portion to obtain convolutional features of the shared portion; performing convolution processing on the convolutional features of the shared portion using the independent portions of the convolutional layers of each expert sub-network to obtain convolutional features of the independent portions of each expert sub-network; and inputting the convolutional features of the independent portions of each expert sub-network into the classification layer of the corresponding expert sub-network to obtain image features extracted by each expert sub-network.

[0062] Reference Figure 4 and Figure 5 After inputting the image samples into the expert subnetwork of the image feature model, feature extraction can be performed through the shared part of the convolutional layers of multiple expert subnetworks (also known as the shared convolutional layer) to obtain the common image features of the expert subnetworks.

[0063] exist Figure 4 and Figure 5 In this process, the convolutional features output from the shared portion of the convolutional layer can be propagated to the independent portions of the convolutional layers of each subsequent expert sub-network (the unique convolutional layers of each expert sub-network) for further feature extraction.

[0064] Continue to refer to Figure 4 and Figure 5 In some embodiments, the classification layer of each expert sub-network is used to receive convolutional features output from an independent portion of the convolutional layer of the corresponding expert sub-network, and to extract features from the received convolutional features to obtain image features extracted by each expert sub-network, such as the image features extracted by the first expert sub-network (denoted as...). Figure 4 and Figure 5 The image features extracted by the F1 and second expert subnetworks (represented as F1, F2, F3, F4, F5, F6, F7, F8, F9, F1, F1, F1, F2 ...2, F1, F2, Figure 4 and Figure 5 The image features extracted by the F2 subnetwork and the third expert subnetwork (represented as F2) and the third expert subnetwork. Figure 4 and Figure 5 (F3 in the middle).

[0065] In this embodiment of the disclosure, the input image samples are sequentially passed through the shared part of the convolutional layer, the independent part of the convolutional layer, and the classification layer of each sub-network to extract features, thereby obtaining the image features corresponding to each expert sub-network.

[0066] In some embodiments, step S310, which involves performing feature classification based on image features extracted by multiple expert subnetworks to obtain the predicted category of the image sample, may specifically include: fusing the image features output by each expert subnetwork to obtain fused features; and normalizing the fused features to obtain the predicted category of the image sample.

[0067] In this embodiment, the average value of the image features output by each expert subnetwork can be taken as the fusion feature, or the image features output by each expert subnetwork can be weighted and summed according to the pre-set feature weights of each expert subnetwork to obtain the fusion feature.

[0068] Reference Figure 4 and Figure 5 In this embodiment, the fused features are normalized. After normalization, the image category predicted by each expert subnetwork for the image sample can be obtained, that is, the predicted category of the image sample.

[0069] In some embodiments, step S320 may specifically include the following steps.

[0070] S41, construct the first loss function based on the predicted category and the labeled category of the image sample.

[0071] Reference Figure 4 and Figure 5 In this step, the first loss function is used to determine the loss between the predicted class and the true labeled class; therefore, the first loss function is used to calculate the classification loss. For example, the first loss function could be the cross-entropy loss function. Figure 4 and Figure 5 In this context, the first loss function can be denoted as: Classification Loss (L... Classify ).

[0072] S42, construct a second loss function based on N angles; wherein, the N angles are determined as follows: determine the feature center point based on the image features extracted by each expert sub-network, and obtain M included angles by connecting the feature points of the image features extracted by each two adjacent expert sub-networks with the feature center point, and obtain the first predetermined number of included angles from the M included angles in ascending order to obtain N angles, where M is an integer greater than or equal to 1.

[0073] Reference Figure 4 and Figure 5The average value of the image features extracted by the first expert subnetwork (feature F1), the second expert subnetwork (feature F2), and the third expert subnetwork (feature F3) is removed to obtain the central feature F. 中心 For example, feature F1 and central feature F 中心 The connection between feature F2 and central feature F 中心 The lines connecting them form a feature angle α1, and feature F1 and the central feature F 中心 The connection and feature F3 and the central feature F 中心 The lines connecting them form a characteristic angle α2, and feature F2 intersects with the central feature F. 中心 The connection and feature F3 and the central feature F 中心 The lines connecting these features form a feature angle α3. The smaller the feature angle, the smaller the difference between the image features learned by the corresponding expert sub-networks; the larger the feature angle, the larger the difference between the image features learned by the corresponding expert sub-networks.

[0074] In this embodiment, the mean value of the image features extracted by each expert sub-network can be calculated as the central feature. The angle between the image features extracted by each two adjacent expert sub-networks and the central feature is used to represent the difference between the image features extracted by each two adjacent expert sub-networks and the central feature. By adjusting the angle, the features learned by each adjacent expert sub-network can eventually tend to the central feature.

[0075] In some embodiments, the construction of the second loss function can be expressed as the following expression (1):

[0076]

[0077] In the above expression (1), L separate K represents the second loss function used to calculate the angle difference loss. SortedAngle() represents the sorting of the angles formed by each pair of adjacent image features and the center feature in ascending order for the image features extracted by each expert subnetwork; separate K represents the number of included angles obtained after sorting. separate The value of is greater than 1 and less than the total number of expert subnetworks in the image classification model; the predetermined number K separate The value can also be half of the total number of expert subnetworks, rounded up or down; for example, if the total number of expert subnetworks is 3, the predetermined number can be 1 or 2; if the total number of expert subnetworks is 5, the predetermined number can be 2 or 3.

[0078] In some embodiments, the angle K can be the smallest. separate The summation of the included angles is used as the second loss function; through the second loss function, the smallest integer angle, K, is added.separate The sum of the angles can widen (increase) the angles between the features learned by the expert sub-networks with the smallest angles, making the differences in the content learned by the expert sub-networks greater, thus allowing multiple expert sub-networks to learn more content; and widening (increasing) the angles between the features learned by the expert sub-networks with the smallest angles can make the angle between the image features output by each expert sub-network and the central feature tend to be close to 120 degrees, thereby accelerating the convergence of the image classification model and accelerating the optimization of the image classification model.

[0079] Reference Figure 4 and Figure 5 In this step, the second loss function is used to determine the degree of difference between the image features learned by different expert sub-networks; therefore, the second loss function is used to calculate the angle difference loss. For example, the second loss function can be denoted as: Angle Difference Loss (L... Separate ).

[0080] S43, perform a weighted summation of the first loss function and the second loss function to obtain the target loss function.

[0081] In some embodiments, the target loss function is expressed as: L total =L classify +λ S L separate , where L classify It is the first loss function, L separate It is the second loss function, λ S It is L separate hyperparameters, L total It is the target loss function.

[0082] Reference Figure 4 In some embodiments, a target loss function can be constructed using a first loss function and a second loss function, and the model can be trained based on the target loss function.

[0083] In the model training process of this embodiment, the classification loss of the image classification model is represented by a first loss function, and minimizing the classification loss can improve the prediction accuracy of the image classification model; the difference between the features learned by each expert sub-network in the image classification model is represented by a second loss function.

[0084] In some embodiments, step S42 above, the step of determining the feature center point based on the image features extracted by each expert sub-network, may specifically include: taking the feature point corresponding to the feature average as the feature center point, wherein the feature average is calculated by: calculating the average of the feature values ​​based on the feature values ​​of the image features extracted by each expert sub-network to obtain the feature average; constructing a second loss function based on N angles includes: taking the sum of the N angles as the second loss function; wherein N is an integer greater than 1 and less than the total number of expert sub-networks included in the image classification model.

[0085] In this embodiment, the image feature values ​​output by multiple expert sub-networks are first averaged, and the feature point corresponding to the average value is taken as the center feature point. Then, the feature angle formed by the line connecting the feature points of the image features output by each two adjacent expert sub-networks and the center feature point is calculated. Several feature angles with the smallest angles are obtained, and the sum of these several feature angles is taken as the second loss function. The second loss function is used to widen (increase) the angles of the several feature angles with the smallest angles.

[0086] Reference Figure 4 and Figure 5 In the training process of the image classification model in this embodiment, for a predetermined number of angles with the smallest obtained angles, by increasing the angle (i.e., widening the angle), the distance between the image features corresponding to the predetermined number of angles with the smallest obtained angles can be increased, making the differences in the content learned by each expert greater, thereby allowing multiple experts to learn more features; on the other hand, for the predetermined number of angles with the smallest obtained angles, the angle increase adjustment operation can make the image features output by adjacent expert sub-networks tend to be the same as the angle formed by the central feature; such as Figure 4 and Figure 5 As shown, if the image classification model includes three expert subnetworks, then during the training process of the image classification model, the angle between the image features output by each expert subnetwork and the central features tends to be close to 120 degrees, which can speed up the convergence of the image classification model and accelerate the optimization of the image classification model.

[0087] In this embodiment, by adjusting the angle between the image features extracted by different expert sub-networks and the central feature, the distance between the features learned by different expert sub-networks can be avoided from being too close, thereby making the differences in the content learned by each expert greater, which is beneficial for multiple experts to learn more features. Furthermore, by adjusting the angle between the image features extracted by different expert sub-networks and the central feature, the angle between the image features extracted by different expert sub-networks and the central feature can be made to approach the average angle (the ratio of 360 degrees to the total number of expert sub-networks), which accelerates the optimization process of the network and is conducive to training a network model with balanced capabilities.

[0088] The following is combined Figure 5 This describes another method for constructing a target loss function in an embodiment of the present disclosure.

[0089] In some embodiments, step S43 may specifically include: performing a weighted summation based on the first loss function, the second loss function, and the third loss function to obtain the target loss function; wherein, the third loss function is a loss function constructed based on the differences between the image features corresponding to the N included angles.

[0090] For example, in Figure 5 In the process, the feature angles α1, α2, and α3 are sorted from smallest to largest as follows: angle α1, angle α2, and angle α3. A predetermined number of angles with the smallest angles are obtained from the angle sorting results. A third loss function is constructed based on the differences between the image features corresponding to the predetermined number of angles with the smallest angles.

[0091] For example, the target loss function is expressed as: L total =L classify +λL Diversify +λ S L separate , where L classify It is the first loss function, L separate It is the second loss function, λ S It is L separate hyperparameters, L Diversify It is the third loss function, and λ is L Diversify hyperparameters, L total It is the target loss function.

[0092] In this embodiment of the disclosure, the third loss function is used to reflect the differences between image features corresponding to a predetermined number of angles with the smallest angle. Increasing the value of the loss function can increase the gap between the learned features, while decreasing the value of the loss function can reduce the gap between the learned features.

[0093] In this embodiment, a target loss function is constructed based on a first loss function, a second loss function, and a third loss function. While ensuring the image classification accuracy (corresponding to the first loss function), the distance between features learned in the multi-expert network architecture is adjusted by the difference between features (the third loss function), and the uneven distribution of feature differences learned in the multi-expert network architecture is addressed by the angle between features (the second loss function), thereby improving the model's classification performance for all image classifications and thus improving the model's prediction accuracy for image classification.

[0094] In some embodiments, the third loss function may be constructed as follows: based on the image features corresponding to each of a predetermined number of angles, the probability distribution difference of the image features is calculated, and the third loss function is constructed based on the calculated probability distribution difference of the image features.

[0095] In this embodiment, the probability distribution difference can be represented by KL divergence, which stands for Kullback Leibler divergence and is used to measure the "distance" between image features output by two expert subnetworks. The third loss function is represented as L... Diversity , for L Diversity The calculation process includes: first, averaging the feature values ​​extracted by multiple expert sub-networks to obtain the central feature (representing the center point of the feature values); then, calculating the distance between the image feature extracted by each expert sub-network and the central feature; and using a third loss function to make the image feature extracted by each expert sub-network move away from the central feature, thereby maximizing the distribution distance of the image features extracted by multiple expert sub-networks. According to the embodiments of this disclosure, the differences between different experts can be maximized by maximizing the KL divergence between the classification prediction values ​​output by different expert sub-networks.

[0096] In some embodiments, step S330 may specifically include: during the iterative training of the image classification model based on the model parameters, calculating the gradient of the loss function using the gradient descent algorithm; adjusting the image classification model parameters based on the gradient corresponding to the loss function; and obtaining the trained image classification model using the adjusted image classification model parameters.

[0097] In this embodiment of the disclosure, the initial value of the model parameters can be a random parameter or a parameter value set by the user based on experience. In some embodiments, during model training, a gradient descent algorithm can be used to determine the gradient corresponding to the loss function, and the parameters of the image classification model can be adjusted according to the gradient corresponding to the loss function. As an example, the gradient descent algorithm may include backpropagation algorithm based on gradient descent, Adam optimization algorithm, etc.

[0098] In some embodiments, the gradient itself is a vector, which can be called a gradient vector. The gradient vector can indicate the direction of updating the model parameters during the training process of the image classification model. By updating the parameters of the image classification model along the direction of the gradient vector and according to the magnitude of the gradient vector, it can be ensured that the result of each training of the image classification model is closer to the model training target.

[0099] In this embodiment, the first loss function (classification loss), the second loss function (diversity loss), and the third loss function (angle difference loss) are weighted and summed to obtain the final loss. Then, backpropagation is performed to update the gradient, and the network is iterated until convergence to obtain the trained image classification model.

[0100] The image classification model training method according to the embodiments of this disclosure minimizes the classification loss constructed based on the predicted category and the labeled category by adjusting the model parameters of the image classification model, thereby ensuring image classification accuracy. Furthermore, it maximizes the KL divergence between image features output by different expert sub-networks to maximize the differences between image features learned by different experts. Additionally, it addresses the uneven distribution of feature differences learned in the multi-expert network architecture by adjusting the angles between features. This achieves the training of a balanced neural network using imbalanced long-tail data, improving the model's classification performance for all image categories and thus enhancing the model's prediction accuracy for image classification.

[0101] Figure 6 A flowchart illustrating the image classification method provided in an embodiment of this disclosure is shown. Figure 6 As shown, the image classification method includes the following steps.

[0102] S610, Obtain the image to be classified; S620, Input the image to be classified into the image classification model for category prediction to obtain the predicted category of the image to be classified; wherein, the image classification model includes multiple expert sub-networks, and the steps of inputting the image to be classified into the image classification model for category prediction include: extracting features from the image samples through multiple expert sub-networks respectively to obtain the image features extracted by each expert sub-network; performing feature classification based on the image features extracted by multiple expert sub-networks to obtain the predicted category of the image sample.

[0103] The image classification model in this embodiment can be a model trained according to any of the image classification model training methods described in the above embodiments.

[0104] In this embodiment of the disclosure, when it is necessary to classify an image, the image is input into an image classification network to obtain the feature values ​​extracted by each expert subnetwork. The average value of the feature values ​​extracted by each expert subnetwork is then taken, and the image is classified through a fully connected layer to obtain the category of the current image.

[0105] According to the image classification method of this disclosure, since the image classification model used has a relatively balanced image feature learning ability for imbalanced long-tail data, the image classification result output by this image classification method has high accuracy.

[0106] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0107] In addition, this disclosure also provides a training device for an image classification model, an image classification device, an electronic device, and a computer-readable storage medium. All of the above can be used to implement any of the image classification model training methods or image classification methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section, and will not be repeated here.

[0108] Figure 7 A block diagram of a training device for an image classification model provided in an embodiment of this disclosure.

[0109] Reference Figure 7 This disclosure provides a training device for an image classification model, which may include the following modules.

[0110] The determination module 710 is used to input image samples from a pre-acquired training dataset into an image classification model to obtain the predicted category of the image samples. The image classification model includes multiple expert sub-networks. The steps of inputting image samples into the image classification model for category prediction include: extracting features from the image samples through multiple expert sub-networks to obtain the image features extracted by each expert sub-network; and performing feature classification based on the image features extracted by the multiple expert sub-networks to obtain the predicted category of the image samples.

[0111] Module 720 is used to construct the target loss function based on the predicted category, the labeled category of the image sample, and N angles. The N angles are determined based on the image features extracted by each expert sub-network, and N is an integer greater than or equal to 1.

[0112] The adjustment module 730 is used to adjust the parameters of the image classification model according to the target loss function to obtain the trained image classification model.

[0113] In some embodiments, the image samples in the image training dataset exhibit a long-tailed distribution.

[0114] In some embodiments, the image samples are obtained in the following manner, wherein the determining module 710 is specifically used to: load image data in the initial dataset using a random strategy through a data loader; and perform data augmentation on the loaded image data according to at least one data augmentation method to obtain the augmented image samples corresponding to each data augmentation method.

[0115] In some embodiments, each expert subnetwork includes a convolutional layer and a classification layer, and the convolutional layer of each expert subnetwork includes a shared part and an independent part. When the determining module 710 extracts features from image samples through multiple expert subnetworks to obtain image features extracted by each expert subnetwork, it specifically performs the following steps: convolving the image samples using the shared part to obtain the convolutional features of the shared part; convolving the convolutional features of the shared part using the independent part of the convolutional layer of each expert subnetwork to obtain the convolutional features of the independent part of each expert subnetwork; and inputting the convolutional features of the independent part of each expert subnetwork into the classification layer of the corresponding expert subnetwork to obtain the image features extracted by each expert subnetwork.

[0116] When determining the predicted category of an image sample by performing feature classification based on image features extracted from multiple expert sub-networks, the module 710 specifically performs the following steps: fusing the image features output by each expert sub-network to obtain fused features; and normalizing the fused features to obtain the predicted category of the image sample.

[0117] In some embodiments, the construction module 720 is specifically used to: construct a first loss function based on the predicted category and the labeled category of the image sample; construct a second loss function based on N angles; wherein the N angles are determined as follows: based on the image features extracted by each expert sub-network, determine the feature center point; according to the line connecting the feature points of the image features extracted by each two adjacent expert sub-networks and the feature center point, obtain M included angles; obtain the first predetermined number of included angles from the M included angles in ascending order to obtain N angles, where M is an integer greater than or equal to 1; and perform a weighted summation of the first loss function and the second loss function to obtain the target loss function.

[0118] In some embodiments, when the construction module 720 performs a weighted summation of the first loss function and the second loss function to obtain the target loss function, it specifically performs a weighted summation based on the first loss function, the second loss function, and the third loss function to obtain the target loss function; wherein, the third loss function is a loss function constructed based on the differences between the image features corresponding to the N included angles.

[0119] In some embodiments, when the construction module 720 determines the feature center point based on the image features extracted by each expert sub-network, it specifically uses the feature point corresponding to the feature average value as the feature center point, wherein the feature average value is calculated by calculating the average value of the feature values ​​of the image features extracted by each expert sub-network to obtain the feature average value.

[0120] In some embodiments, when constructing a second loss function based on N angles, the construction module 720 is specifically used to: use the sum of the angles of the N angles as the second loss function; where N is an integer greater than 1 and less than the total number of expert subnetworks contained in the image classification model.

[0121] According to the image classification model training apparatus of the embodiments of this disclosure, the classification loss constructed based on the predicted category and the labeled category can be minimized by adjusting the model parameters of the image classification model to ensure image classification accuracy; and the differences between image features learned by different experts can be maximized by maximizing the KL divergence between image features output by different expert sub-networks, and the problem of uneven distribution of feature differences learned in the multi-expert network architecture can be handled by adjusting the angle between features. This realizes the training of a balanced neural network using imbalanced long-tail data, improves the model's classification performance for all image classifications, and thus improves the model's prediction accuracy for image classification.

[0122] Figure 8 A block diagram of an image classification device provided in an embodiment of this disclosure. Figure 8 As shown, the image classification device 800 may include the following modules:

[0123] Module 810 is used to acquire images to be classified.

[0124] The prediction module 820 is used to input the image to be classified into the image classification model for category prediction, and obtain the predicted category of the image to be classified. The image classification model includes multiple expert sub-networks. The steps of inputting the image to be classified into the image classification model for category prediction include: extracting features from the image samples through multiple expert sub-networks to obtain the image features extracted by each expert sub-network; and performing feature classification based on the image features extracted by multiple expert sub-networks to obtain the predicted category of the image sample.

[0125] According to the image classification apparatus of this disclosure, since the image classification model used has a relatively balanced image feature learning ability for imbalanced long-tail data, the image classification result output by the image classification method has high accuracy.

[0126] It should be clarified that the present invention is not limited to the specific configurations and processes described in the above embodiments and shown in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here, and the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0127] Figure 9 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0128] Reference Figure 9 This disclosure provides an electronic device, which includes: at least one processor 901; at least one memory 902; and one or more I / O interfaces 903 connected between the processor 901 and the memory 902; wherein the memory 902 stores one or more computer programs that can be executed by the at least one processor 901, and the one or more computer programs are executed by the at least one processor 901 to enable the at least one processor 901 to execute the above-described image classification model training method or image classification method.

[0129] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the above-described image classification model training method or image classification method. The computer-readable storage medium may be volatile or non-volatile.

[0130] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the training method or image classification method of the image classification model described above.

[0131] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0132] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0133] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0134] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0135] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0136] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0137] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0138] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0140] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A training method for an image classification model, characterized in that, The method includes: Image samples from a pre-acquired training dataset are input into an image classification model to obtain the predicted category of the image samples. The image classification model comprises multiple expert sub-networks. The step of inputting the image samples into the image classification model for category prediction includes: extracting features from the image samples using each of the multiple expert sub-networks to obtain image features extracted by each expert sub-network; and performing feature classification based on the image features extracted by the multiple expert sub-networks to obtain the predicted category of the image samples. Based on the predicted category, the labeled category of the image sample, and N angles, a target loss function is constructed. The N angles are determined as follows: based on the image features extracted by each expert sub-network, a feature center point is determined. Based on the line connecting the feature points of the image features extracted by each two adjacent expert sub-networks with the feature center point, M angles are obtained. From the M angles, the first predetermined number of angles from smallest to largest are obtained to get N angles, where M is an integer greater than or equal to 1 and N is an integer greater than or equal to 1. The parameters of the image classification model are adjusted according to the target loss function to obtain the trained image classification model.

2. The method according to claim 1, characterized in that, The image sample was obtained in the following way: The image data in the initial dataset is loaded using a random strategy via a data loader. Data augmentation is performed on the loaded image data using at least one data augmentation method to obtain augmented image samples corresponding to each data augmentation method.

3. The method according to claim 1, characterized in that, Each expert subnetwork includes convolutional layers and classification layers. The convolutional layers of each expert subnetwork include shared parts and independent parts. The step of extracting features from the image samples through the multiple expert sub-networks to obtain the image features extracted by each expert sub-network includes: The shared portion is used to perform convolution processing on the image samples to obtain the convolutional features of the shared portion; the independent parts of the convolutional layers of each expert sub-network are used to perform convolution processing on the convolutional features of the shared portion to obtain the convolutional features of the independent parts of each expert sub-network; the convolutional features of the independent parts of each expert sub-network are input into the classification layer of the corresponding expert sub-network to obtain the image features extracted by each expert sub-network. The step of classifying the image features extracted from the multiple expert sub-networks to obtain the predicted category of the image sample includes: fusing the image features output by each expert sub-network to obtain a fused feature; and normalizing the fused feature to obtain the predicted category of the image sample.

4. The method according to claim 1, characterized in that, The step of constructing a target loss function based on the predicted category, the labeled category of the image sample, and N angles includes: Based on the predicted category and the labeled category of the image sample, a first loss function is constructed; A second loss function is constructed based on the N angles; The target loss function is obtained by weighted summation of the first loss function and the second loss function.

5. The method according to claim 4, characterized in that, The step of weighted summing of the first loss function and the second loss function to obtain the target loss function includes: The target loss function is obtained by weighted summation of the first loss function, the second loss function, and the third loss function; wherein the third loss function is a loss function constructed based on the differences between the image features corresponding to the N included angles.

6. The method according to claim 4, characterized in that, The determination of feature center points based on image features extracted by each expert sub-network includes: The feature point corresponding to the feature average value is taken as the feature center point. The feature average value is calculated by calculating the average value of the feature values ​​of the image features extracted by each expert sub-network. The construction of the second loss function based on the N angles includes: The sum of the N angles is used as the second loss function; where N is an integer greater than 1 and less than the total number of expert subnetworks in the image classification model.

7. An image classification method, characterized in that, include: Obtain the image to be classified, input the image to be classified into the image classification model trained according to any one of claims 1-6 for category prediction, and obtain the predicted category of the image to be classified; The image classification model includes multiple expert sub-networks. The step of inputting the image to be classified into the image classification model for category prediction includes: extracting features from the image sample through the multiple expert sub-networks to obtain the image features extracted by each expert sub-network; and performing feature classification based on the image features extracted by the multiple expert sub-networks to obtain the predicted category of the image sample.

8. An image classification device, characterized in that, The acquisition module is used to acquire images to be classified. The prediction module is used to input the image to be classified into an image classification model trained according to any one of claims 1-6 for category prediction, thereby obtaining the predicted category of the image to be classified; wherein, the image classification model includes multiple expert sub-networks, and the step of inputting the image to be classified into the image classification model for category prediction includes: extracting features from the image sample through the multiple expert sub-networks respectively, thereby obtaining the image features extracted by each expert sub-network; and performing feature classification based on the image features extracted by the multiple expert sub-networks to obtain the predicted category of the image sample.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the image classification model training method as described in any one of claims 1-6, or the image classification method as described in claim 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image classification model training method as described in any one of claims 1-6, or the image classification method as described in claim 7.

Images