Image classification model training method and device, equipment and storage medium

Abstract

The application provides an image classification model training method and device, equipment and a storage medium. The method comprises the following steps: obtaining basic features of image bags in a sample set and category proportion labels of the image bags; inputting the basic features into a sparse dictionary learning submodel to obtain target sparse features of the image bags; inputting the target sparse features into a classifier to obtain predicted category labels of each image in the image bags; determining predicted category proportion labels of the image bags based on the predicted category labels of each image in the image bags; determining a first loss function value based on the predicted category proportion labels and the category proportion labels; and training the image classification model based on the first loss function value to obtain a trained image classification model. According to the application, discriminative features of each image in the image classification model are effectively mined under the condition that only category proportion labels are available in the data set, so that instance-level classification of the images is realized through bag-level category proportion labels.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to an image classification model training method and device, equipment and a storage medium. BACKGROUND

[0002] In real life, many useful information is often published in the form of group-level statistical data rather than specific individual data, such as the influenza infection rate in a certain area, the support rate of different candidates in a certain area, the education level distribution of a certain city, and the like. Such information plays an important role in data analysis tasks, but traditional instance-based learning methods are difficult to directly use such data. In addition, the success of existing algorithms is largely due to large-scale labeled data, and obtaining such data often requires very high time and labor costs; at the same time, due to privacy issues, some instance-level label information is sometimes difficult or even impossible to obtain directly, and only the proportion of fine-grained labels is provided.

[0003] Therefore, it is necessary to effectively mine discriminative features in fine-grained images under the condition that only class proportion labels are available, and to realize instance-level classification of fine-grained images. SUMMARY

[0004] The application provides an image classification model training method and device, equipment and a storage medium, which can effectively mine discriminative features in images by an image classification model under the condition that only class proportion labels are available in a data set, so as to realize instance-level classification of images by the image classification model.

[0005] An embodiment of the first aspect of the application provides an image classification model training method, comprising:

[0006] obtaining a basic feature of an image bag in a sample set and a class proportion label of the image bag;

[0007] inputting the basic feature into a sparse dictionary learning sub-model of an image classification model to obtain a target sparse feature of the image bag;

[0008] inputting the sparse feature into a classifier of the image classification model to obtain a predicted class label of each image in the image bag;

[0009] determining a predicted class proportion label of the image bag based on the predicted class label of each image in the image bag;

[0010] determining a first loss function value based on the predicted class proportion label of the image bag and the class proportion label of the image bag;

[0011] Based on the first loss function value, the parameters of the image classification model are adjusted, and the training is continued until a preset first training completion condition is met, and a trained image classification model is obtained.

[0012] Embodiments of the second aspect of the application provide an image classification model training device, comprising:

[0013] The acquisition module is configured to acquire a basic feature of an image bag in a sample set and a category proportion label of the image bag.

[0014] The input module is configured to input the basic feature into a sparse dictionary learning submodel of an image classification model to acquire a target sparse feature of the image bag.

[0015] The input module is further configured to input the sparse feature into a classifier of the image classification model to acquire a predicted category label of each image in the image bag.

[0016] The determination module is configured to determine a predicted category proportion label of the image bag based on the predicted category label of each image in the image bag.

[0017] The determination module is configured to determine a first loss function value based on the predicted category proportion label of the image bag and the category proportion label of the image bag.

[0018] The adjustment module is configured to adjust parameters of the image classification model based on the first loss function value, continue the training, and obtain a trained image classification model until a preset first training completion condition is met.

[0019] Embodiments of the third aspect of the application provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of the first aspect.

[0020] Embodiments of the fourth aspect of the application provide a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method of the first aspect.

[0021] The technical solutions provided in the embodiments of the application have at least the following technical effects or advantages:

[0022] The application provides an image classification model training method and device, equipment and a storage medium. The method comprises the following steps: obtaining basic features of an image bag in a sample set and a category proportion label of the image bag; inputting the basic features into a sparse dictionary learning submodel to obtain target sparse features of the image bag; inputting the target sparse features into a classifier to obtain a predicted category label of each image in the image bag; determining a predicted category proportion label of the image bag based on the predicted category label of each image in the image bag; determining a first loss function value based on the predicted category proportion label and the category proportion label; and training the image classification model based on the first loss function value to obtain a trained image classification model. The embodiment of the application effectively mines discriminative features of the image classification model for each image under the condition that only the category proportion label is provided in the data set, so that the instance-level classification of the image is realized through the bag-level category proportion label.

[0023] Additional aspects and advantages will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0024] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the application. Furthermore, the drawings are not necessarily drawn to scale.

[0025] In the drawings:

[0026] Figure 1 A flowchart of an image classification model training method provided by an embodiment of the application is shown;

[0027] Figure 2 A structure diagram of a sparse dictionary learning submodel provided by an embodiment of the application is shown;

[0028] Figure 3 A structure diagram of an image classification model training device provided by an embodiment of the application is shown;

[0029] Figure 4 A structure diagram of an electronic device provided by an embodiment of the application is shown;

[0030] Figure 5 A diagram of a storage medium provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0031] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0032] It should be noted that, unless otherwise specified, technical terms or scientific terms used in the present application should be understood as their common meanings to those skilled in the art to which the present application pertains.

[0033] The design method of the synchronization signal block of the present application can be executed by a computing device, and the computing device can be a server, such as a server, a plurality of servers, a server cluster, a cloud computing platform, etc., and optionally, the computing device can also be a terminal device, such as a mobile phone, a tablet computer, a game console, a portable computer, a desktop computer, an advertising machine, an all-in-one machine, etc., and the present application does not limit the device type and the number of the computing device.

[0034] In view of the above background, fine-grained images and their labels usually contain a large amount of detailed and privacy-sensitive information, and most existing computer vision tasks for fine-grained images are still based on instance-level fine-grained images, and the corresponding privacy protection strategies are limited to masking or encoding instance-level labels, and there is no related research on whether label proportion learning can be used to protect the privacy of fine-grained images. In addition, applications with high privacy requirements may only provide the proportion of fine-grained labels rather than the specific class labels of each instance, in which case directly using label proportion learning can not only effectively utilize existing annotation information, but also achieve privacy protection of fine-grained information. The main challenge of label proportion learning for fine-grained images is how to effectively mine discriminative features from fine-grained images with only class proportion labels and achieve instance-level classification of fine-grained images. A key solution is to fully utilize the bag-level structure under the label proportion learning paradigm, to extract and mine features from fine-grained images by constructing a sparse dictionary, and to train a fine-grained visual classification model based on label proportion learning.

[0035] To solve the above problems, the application provides an image classification model training method, device and equipment and storage medium. The method comprises the following steps: obtaining the basic features of the image bag in the sample set and the category proportion label of the image bag; inputting the basic features into a sparse dictionary learning submodel to obtain the target sparse features of the image bag; inputting the target sparse features into a classifier to obtain the predicted category label of each image in the image bag; determining the predicted category proportion label of the image bag based on the predicted category label of each image in the image bag; determining the first loss function value based on the predicted category proportion label and the category proportion label; training the image classification model by using the first loss function value to obtain the trained image classification model. The embodiment of the application effectively mines the discriminative features of the image classification model for each image under the condition that the data set only has the category proportion label, so as to realize the instance-level classification of the image through the bag-level category proportion label.

[0036] The image classification model training method, device, equipment and storage medium according to the embodiment of the application are described below in combination with the drawings.

[0037] Referring to Figure 1 The method specifically comprises the following steps:

[0038] In S101, the basic features of the image bag in the sample set and the category proportion label of the image bag are obtained.

[0039] The sample set comprises a plurality of image bags, and each image bag comprises a plurality of sample images. Since the application with high privacy requirements may only provide the proportion of fine-grained labels instead of the specific category label of each instance, the sample data set does not provide the label of each image, but provides the category proportion label of the image bag. For example, an image bag comprises 10 images, and the category proportion label of the image bag is 20% for the category dog, 50% for the category cat and 30% for the category mouse. The category proportion label of the image bag indicates that 2 images in the 10 images in the bag belong to the category dog, 5 images belong to the category cat and 3 images belong to the category mouse.

[0040] The image bag in the sample set can be input into a pre-trained neural network model to extract the basic features of the image bag. The neural network model can be a convolutional neural network, a VGG-19 network, a ResNet-50 network or the like.

[0041] The image bag can be represented as The basic features can be represented as

[0042] B i represents the i-th input image bag, which contains |B i | instance images, denotes the jth image instance in the ith input image bag, W denotes the width of the input image, H denotes the height of the input image, and the corresponding base feature is denoted as

[0043] S102, input the base feature into a sparse dictionary learning submodel of the image classification model to obtain a target sparse feature of the image bag.

[0044] The sparse dictionary learning submodel includes a dictionary and a set of sparse coefficient matrices to form a sparse feature. Generally, the base feature can be approximated by the product of the dictionary and the sparse feature. In the process of representing the base feature by the product of the dictionary and the sparse feature, the dictionary is optimized to better adapt to the sample set.

[0045] After the dictionary is optimized, the target sparse feature corresponding to the base feature can be determined by the dictionary and the base feature.

[0046] Since the sparse feature represents a large amount of information in the image with as few elements as possible, it effectively reduces the storage and transmission amount of data. This representation can remove redundant information in the image while retaining key information, so the sparse feature is applied to image classification to improve the efficiency of image classification.

[0047] S103, input the sparse feature into a classifier of the image classification model to obtain a predicted class label of each image in the image bag.

[0048] S104, based on the predicted class label of each image in the image bag, determine a predicted class proportion label of the image bag.

[0049] The classifier can output the predicted image class of each image in the image bag based on the sparse feature of the image bag, and further determine the predicted class proportion label of the image bag based on the image class of each image.

[0050] For example, the image bag includes 10 images, and the classifier outputs the result that the first image is a dog, the second to fifth images are cats, and the remaining images are mice, so that the predicted class proportion label of the image bag can be determined based on the output result as 10% for dogs, 40% for cats, and 50% for dogs.

[0051] S105, determine a first loss function value based on the predicted class proportion label of the image bag and the class proportion label of the image bag.

[0052] S106, based on the first loss function value, adjust the parameters of the image classification model to continue training until a preset first training completion condition is met, and obtain a trained image classification model.

[0053] The first loss function value can be determined based on a difference value between the predicted class proportion label of the image bag and the class proportion label of the image bag, and the parameters of the image classification model are adjusted based on the first loss function value, and the training is continued until a preset first training completion condition is met, and a trained image classification model is obtained.

[0054] The first training completion condition can be that the number of training times reaches a first preset training number threshold or the first loss function value is less than a first loss function threshold, wherein the first preset training number threshold and the first loss function threshold can be flexibly set based on actual conditions.

[0055] The application provides an image classification model training method, device and equipment and a storage medium. The method comprises: obtaining a basic feature of an image bag in a sample set and a class proportion label of the image bag; inputting the basic feature into a sparse dictionary learning sub-model to obtain a target sparse feature of the image bag; inputting the target sparse feature into a classifier to obtain a predicted class label of each image in the image bag; determining a predicted class proportion label of the image bag based on the predicted class label of each image in the image bag; determining a first loss function value based on the predicted class proportion label and the class proportion label; and training the image classification model based on the first loss function value to obtain a trained image classification model. The embodiments of the application effectively mine the discriminative features of the image classification model for each image under the condition that the data set only has a class proportion label, so as to realize instance-level classification of the image through the bag-level class proportion label.

[0056] In some embodiments, inputting the basic deep feature into the sparse dictionary learning sub-model of the image classification model to obtain the target sparse feature of the image bag comprises: determining a first target dictionary of the sparse dictionary learning sub-model; determining a first sparse feature of the image bag based on the basic feature of the image bag and the first target dictionary; determining a second sparse feature of the image bag based on the basic feature of the image bag and a target sub-dictionary related to the label proportion of the image bag in the first target dictionary; constraining the first target dictionary based on the first sparse feature of the image bag and the second sparse feature of the image bag to obtain a second target dictionary; and determining the target sparse feature of the image bag based on the second target dictionary and the basic feature of the image bag.

[0057] After optimizing the dictionary, a first target dictionary is obtained, which can better adapt to the sample set.

[0058] The dictionary can be iteratively optimized by setting an optimization target until the dictionary meets the optimization target to obtain the first target dictionary.

[0059] A plurality of network layers can also be provided, each network layer being provided with an update strategy, and the dictionary is updated through the plurality of network layers until the first target dictionary is output by the last network layer.

[0060] It can be understood that all categories corresponding to the basis functions in the sample set are included in the first target dictionary, and the categories corresponding to the image bag only include part of all categories. In order to enable the sparse dictionary learning sub-model to better capture and distinguish the differences between different categories, the basis features of the image bag can be represented by the basis functions related to the categories corresponding to the image bag in the first target dictionary.

[0061] In some embodiments, the first sparse features of the image bag can be determined based on the basis features of the image bag and the first target dictionary; the second sparse features of the image bag can be determined based on the basis features of the image bag and the target sub-dictionary related to the label proportion of the image bag in the first target dictionary, and the first target dictionary is constrained based on the first sparse features of the image bag and the second sparse features of the image bag to obtain the second target dictionary.

[0062] In some embodiments, the first target dictionary of the sparse dictionary learning sub-model is determined, including: determining the sparse features corresponding to the basis features of the image bag under the current dictionary; determining the reconstruction features of the basis features based on the sparse features and the dictionary; and optimizing the dictionary to obtain the first target dictionary with the optimization target of minimizing the difference value between the reconstruction features and the basis features.

[0063] Wherein, the basis features are represented as The dictionary is represented as D, and the corresponding sparse features are represented as The corresponding optimization target is:

[0064]

[0065] Wherein, is the reconstruction error of the data, and the purpose of this part is to enable the dictionary and the sparse features to reconstruct the basis features as accurately as possible. By minimizing this error, we ensure that the learned dictionary can effectively represent the training data. Wherein, λ||Z i λ in ||Z i ||1 is a regularization parameter that controls the strength of the regularization term. ||Z i represents the L1 norm of the coefficient features, that is, the sum of the absolute values of all elements in Z i L1 regularization promotes sparsity by minimizing the L1 norm of Z i , so that many elements of Z i tend to zero, thereby achieving sparse representation. By adjusting the value of λ, the reconstruction error and sparsity can be balanced. A larger λ value will make the model more sparse, but may increase the reconstruction error; while a smaller λ value will reduce sparsity, but may get more accurate reconstruction.

[0066] In summary, the purpose of this optimization objective is to find a dictionary that can accurately reconstruct the training data, while making the representation of each data as sparse as possible, so as to improve the performance and generalization ability of the model.

[0067] Correspondingly, the update strategy corresponding to the above optimization objective is:

[0068]

[0069] where μ is the step size, t represents the t-th update iteration, is the value of the difference between the basis feature and the dictionary multiplied by the current sparse feature at the t-th iteration, D T is the transpose of D, denotes the gradient with respect to the sparse feature, S λ (v j (=sign(v j )·max{|v j |-λ,0} is an element-wise soft thresholding operator, and max{·} is used to calculate the element-wise maximum value.

[0070] Corresponding to the above update strategy, the sparse feature obtained by each iteration of the update strategy can be represented as:

[0071]

[0072] where,

[0073] Further, the model structure diagram of the sparse dictionary learning sub-model is as shown in Figure 2 , wherein the network layer is stacked L times, i.e. t = 1, 2, …, L, to construct an L-layer unfolded network for sparse dictionary learning, and the dictionary D and the sparse feature representation Z i are learned in a trainable manner. The update strategy corresponding to each single unfolded layer is Thus, the dictionary D and the sparse feature representation Z i can be iteratively learned through multiple single unfolded layers to obtain the first target dictionary.

[0074] In some embodiments, the first target dictionary is constrained based on the first sparse feature of the image bag and the second sparse feature of the image bag to obtain a second target dictionary, including: obtaining a first reconstructed deep feature based on the first sparse feature of the image bag and the first target dictionary; obtaining a second reconstructed deep feature based on the first sparse feature of the image bag and the target sub-dictionary; calculating a second loss function value based on the first reconstructed deep feature and the base deep feature; calculating a third loss function value based on the second reconstructed deep feature and the base deep feature; adjusting the first target dictionary based on the second loss function value and the third loss function until a preset iteration completion condition is reached to obtain the second target dictionary.

[0075] In some embodiments, by constraining the first target dictionary using the reconstructed base feature obtained by using the partial dictionary related to the category of the data set and the corresponding sparse feature, it can be ensured that the sparse feature is more focused on capturing information related to a specific category, thereby improving the category relevance of the feature.

[0076] The second loss function value can be obtained by the following formula:

[0077]

[0078] wherein F i,all = DZ i is the feature representation obtained by feature reconstruction using the entire category-related dictionary D, and d(·,·) is used to measure the reconstruction error.

[0079] The third loss function value can be obtained by the following formula:

[0080]

[0081] wherein, is the feature representation obtained by feature reconstruction using only the sub-dictionary D sub related to each bag label.

[0082] The two reconstruction constraints are weighted and summed to jointly constrain the learning of the dictionary and the sparse feature to be discriminative.

[0083]

[0084] wherein, α is used to control the weight between the two reconstruction loss.

[0085] In some embodiments, based on the first loss function value, the parameters of the image classification model are adjusted for continued training until a preset first training completion condition is met to obtain a trained image classification model, including:

[0086] Based on the first loss function value, the second loss function value and the third loss function value, the parameters of the image classification model are adjusted, and the training is continued until the second preset training completion condition is met, and a trained image classification model is obtained.

[0087] In some embodiments, based on the first loss function value, the second loss function value and the third loss function value, the total loss function value of the image classification model is determined, the parameters of the image classification model are adjusted based on the total loss function value, and the training is continued until the second preset training completion condition is met, and a trained image classification model is obtained.

[0088] The second training completion condition can be that the number of training reaches a second preset training number threshold, or the total loss function value reaches a second loss function threshold, wherein the second preset training number threshold and the second loss function threshold can be flexibly set based on actual conditions.

[0089] The total loss function value can be determined by the following formula:

[0090] Loss=L h_prop +βL bagReconst

[0091] Wherein, Loss is the total loss function, L h_prop is the first loss function, L bagReconst is determined based on the second loss function and the third loss function, and β is used to control the proportion weight between the two loss functions.

[0092] In some embodiments, the classifier includes sub-classifiers of different granularities, and the sparse features are input into the classifier of the image classification model to obtain the predicted class label of each image in the image bag, comprising: inputting the sparse features into a plurality of sub-classifiers of different granularities to output the class prediction result of each image in the image bag under different granularities.

[0093] In some embodiments, the class proportion label of the image bag includes sub-class proportion labels of different granularities, and the first loss function value is determined based on the predicted class proportion label of the image bag and the class proportion label of the image bag, comprising: determining a plurality of sub-loss function values for the image bag based on the respective sub-predicted class proportion labels of the image bag under different granularities and the respective class proportion labels of the image bag under different granularities; and determining the first loss function value based on the plurality of sub-loss function values.

[0094] It can be understood that the class proportion label of the image bag includes sub-class proportion labels of different granularities, for example, an image bag includes four images, and the image bag corresponds to two sub-class proportion labels of different granularities, one of which is 50% cat and 50% dog, and the other of which is 25% Persian cat, 25% Maine cat, 25% Corgi dog, and 25% Dachshund dog.

[0095] To correspond to the sub-class proportion labels of different granularities, the classifier includes sub-classifiers of different granularities to output class prediction results of each image in the image bag at different granularities; and the sub-predicted class proportion labels of the image bag at different granularities are determined based on the class prediction results of each image in the image bag at different granularities.

[0096] The class proportion label can be represented as

[0097] wherein, represents the proportion of images with a class label c in the i-th input image bag, that is, there are images with a label c in the i-th input image bag, c ∈ {1, 2,..., C}, and C is the total number of fine-grained image classes.

[0098] Further, the plurality of sub-loss function values for the image bag are determined based on the sub-predicted class proportion labels of the image bag at different granularities and the sub-class proportion labels of the image bag at different granularities.

[0099] The sub-loss function value can be determined by the following formula:

[0100]

[0101] wherein, is the class proportion loss at the l-th granularity, and there are H granularities, represents the label proportion of class c in the i-th bag at the l-th granularity, represents the predicted value of the corresponding classifier, which is calculated by the classifier f(·) of the corresponding granularity l.

[0102] The first loss function value can be determined by the following formula:

[0103]

[0104] wherein, H is the number of granularities.

[0105] The embodiments of the present application also provide an image classification model training device, which is used to execute the image classification model training method provided by any of the above embodiments. As shown in Figure 3 the device comprises:

[0106] The acquisition module 301 is configured to acquire a base feature of an image bag in a sample set and a category proportion label of the image bag.

[0107] The input module 302 is configured to input the base feature into a sparse dictionary learning submodel of an image classification model to acquire a target sparse feature of the image bag.

[0108] The input module 302 is further configured to input the sparse feature into a classifier of the image classification model to acquire a predicted category label of each image in the image bag.

[0109] The determination module 303 is configured to determine a predicted category proportion label of the image bag based on the predicted category label of each image in the image bag.

[0110] The determination module 303 is further configured to determine a first loss function value based on the predicted category proportion label of the image bag and the category proportion label of the image bag.

[0111] The adjustment module 304 is configured to adjust parameters of the image classification model based on the first loss function value, continue training until a preset first training completion condition is met, and obtain a trained image classification model.

[0112] The present application provides an image classification model training method, device and equipment and storage medium. The method comprises: acquiring a base feature of an image bag in a sample set and a category proportion label of the image bag; inputting the base feature into a sparse dictionary learning submodel to acquire a target sparse feature of the image bag; inputting the target sparse feature into a classifier to acquire a predicted category label of each image in the image bag; determining a predicted category proportion label of the image bag based on the predicted category label of each image in the image bag; determining a first loss function value based on the predicted category proportion label and the category proportion label; and training the image classification model with the first loss function value to obtain a trained image classification model. The present application effectively mines discriminative features of the image classification model for each image in the case that the data set only has a category proportion label, so as to realize instance-level classification of images through bag-level category proportion labels.

[0113] In some embodiments, the input module 302 is specifically configured to:

[0114] determine a first target dictionary of the sparse dictionary learning submodel;

[0115] determine a first sparse feature of the image bag based on the base feature of the image bag and the first target dictionary;

[0116] determine a second sparse feature of the image bag based on the first sparse feature of the image bag and the second sparse feature of the image bag;

[0117] constrain the first target dictionary based on the first sparse feature of the image bag and the second sparse feature of the image bag, to obtain a second target dictionary;

[0118] determine a target sparse feature of the image bag based on the second target dictionary and the base feature of the image bag.

[0119] In some embodiments, the input module 302 is further specifically configured to:

[0120] determine a sparse feature corresponding to the base feature of the image bag under a current dictionary;

[0121] determine a reconstruction feature of the base feature based on the sparse feature and the dictionary;

[0122] optimize the dictionary to obtain a first target dictionary, with an optimization objective of minimizing a difference value between the reconstruction feature and the base feature.

[0123] In some embodiments, the input module 302 is further specifically configured to:

[0124] determine a first reconstruction deep feature based on the first sparse feature of the image bag and the first target dictionary;

[0125] determine a second reconstruction deep feature based on the first sparse feature of the image bag and the target sub-dictionary;

[0126] calculate a second loss function value based on the first reconstruction deep feature and the base deep feature;

[0127] calculate a third loss function value based on the second reconstruction deep feature and the base deep feature;

[0128] adjust the first target dictionary based on the second loss function value and the third loss function until a preset iteration completion condition is reached, to obtain a second target dictionary.

[0129] In some embodiments, the adjustment module 304 is specifically configured to:

[0130] adjust parameters of the image classification model based on the first loss function value, the second loss function value and the third loss function value, and continue training until a preset second training completion condition is met, to obtain a trained image classification model.

[0131] In some embodiments, the classifier includes sub-classifiers of different granularities, and the input module 302 is further specifically configured to:

[0132] input the sparse features into the plurality of sub-classifiers of different granularities to output a class prediction result of each image in the image bag under different granularities.

[0133] In some embodiments, the class proportion label of the image bag includes sub-class proportion labels of different granularities, and the determining module 303 is specifically configured to:

[0134] determine a plurality of sub-loss function values of the image bag based on the respective sub-predicted class proportion labels of the image bag under different granularities and the respective sub-class proportion labels of the image bag under different granularities;

[0135] determine a first loss function value based on the plurality of sub-loss function values.

[0136] The image classification model training apparatus provided by the embodiments of the present application and the image classification model training method provided by the embodiments of the present application have the same beneficial effects as the method adopted, run or implemented by them.

[0137] The embodiments of the present application also provide an electronic device for executing the image classification model training method described above. Please refer to Figure 4 which shows a schematic diagram of an electronic device provided by some embodiments of the present application. As shown in Figure 4 The electronic device 8 includes a processor 800, a memory 801, a bus 802 and a communication interface 803, wherein the processor 800, the communication interface 803 and the memory 801 are connected through the bus 802; the memory 801 stores a computer program which can run on the processor 800, and the processor 800 runs the computer program to execute the image classification model training method provided by any one of the preceding embodiments of the present application.

[0138] The memory 801 can include a high-speed random access memory (RAM: Random Access Memory) and can also include a non-volatile memory such as at least one disk memory. The communication between the apparatus network element and at least one other network element is realized through at least one communication interface 803 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.

[0139] The bus 802 can be an ISA bus, a PCI bus, an EISA bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. The memory 801 is configured to store programs, and the processor 800 executes the programs after receiving execution instructions. The image classification model training method disclosed in any of the embodiments of the present application can be applied to the processor 800 or implemented by the processor 800.

[0140] The processor 800 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor 800 or instructions in the form of software. The processor 800 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), and the like; can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware coding processor for execution, or a combination of hardware and software modules in the coding processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory 801, and the processor 800 reads the information in the memory 801 and combines the hardware to complete the steps of the above method.

[0141] The electronic device provided by the embodiments of the present application and the image classification model training method provided by the embodiments of the present application have the same beneficial effects as the methods they adopt, run or implement.

[0142] The embodiments of the present application also provide a computer readable storage medium corresponding to the image classification model training method provided in the foregoing embodiments. Please refer to Figure 5 The computer readable storage medium shown in the figure is an optical disc 30, and a computer program (i.e. program product) is stored on the optical disc 30. When the computer program is run by a processor, the image classification model training method provided in any of the foregoing embodiments is executed.

[0143] It should be noted that examples of the computer-readable storage medium can also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical, magnetic storage mediums, and the like, which are not listed one by one here.

[0144] The computer-readable storage medium provided by the above embodiments of the present application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, based on the same inventive concept as the image classification model training method provided by the embodiments of the present application.

[0145] It should be noted that:

[0146] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known structures and techniques are not shown in detail in order not to obscure the understanding of the present specification.

[0147] Similarly, it should be appreciated that the individual features of the application described in the above description of example embodiments of the application are sometimes grouped together in a single embodiment, figure or description of related embodiments, for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the individual aspects of the application. The disclosure of a feature in more than one particular group of embodiments does not, however, indicate that the feature is used in each and every embodiment or that the feature is indispensable. More particularly, streams of claims following the detailed description of specific embodiments do not limit the scope of protection to the specific combination of features recited in that claim. The scope of the application is limited only by the terms of the claims.

[0148] Furthermore, those skilled in the art will recognize that, although some of the embodiments described herein comprise other embodiments, not all embodiments comprising some features require the presence of other features. As will be understood by those familiar with the art, the features of the different embodiments can be combined in any combination, and the application includes combinations in which each feature of each of the embodiments is used in combination with one or more features of one or more other embodiments. For example, the application as described below can be used in any one or combination of the embodiments or examples described in this specification, whether or not that embodiment is otherwise described in this specification or associated drawings.

[0149] The above describes only the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for training an image classification model, characterized in that, The method comprises the following steps: obtaining the basic features of the image bag in the sample set and the category proportion label of the image bag; inputting the basic features into a sparse dictionary learning submodel of an image classification model to obtain the target sparse features of the image bag; inputting the sparse features into a classifier of the image classification model to obtain the predicted category label of each image in the image bag; determining the predicted category proportion label of the image bag based on the predicted category label of each image in the image bag; determining a first loss function value based on the predicted category proportion label of the image bag and the category proportion label of the image bag; adjusting the parameters of the image classification model based on the first loss function value, continuing training until a preset first training completion condition is met, and obtaining a trained image classification model; the step of inputting the basic features into the sparse dictionary learning submodel of the image classification model to obtain the target sparse features of the image bag comprises the following steps: determining a first target dictionary of the sparse dictionary learning submodel; determining the first sparse features of the image bag based on the basic features of the image bag and the first target dictionary; determining the second sparse features of the image bag based on the basic features of the image bag and the target sub-dictionary related to the label proportion of the image bag in the first target dictionary; constraining the first target dictionary based on the first sparse features of the image bag and the second sparse features of the image bag to obtain a second target dictionary; determining the target sparse features of the image bag based on the second target dictionary and the basic features of the image bag; the step of determining the first target dictionary of the sparse dictionary learning submodel comprises the following steps: determining the sparse features corresponding to the basic features of the image bag under the current dictionary; determining the reconstruction features of the basic features based on the sparse features and the dictionary; optimizing the dictionary to minimize the difference value between the reconstruction features and the basic features as an optimization target to obtain the first target dictionary; the step of constraining the first target dictionary based on the first sparse features of the image bag and the second sparse features of the image bag to obtain the second target dictionary comprises the following steps: obtaining first reconstruction deep features based on the first sparse features of the image bag and the first target dictionary; obtaining second reconstruction deep features based on the first sparse features of the image bag and the target sub-dictionary; calculating a second loss function value based on the first reconstruction deep features and the basic features; calculating a third loss function value based on the second reconstruction deep features and the basic features; adjusting the first target dictionary based on the second loss function value and the third loss function until a preset iteration completion condition is reached to obtain the second target dictionary.

2. The method of claim 1, wherein, the step of adjusting the parameters of the image classification model based on the first loss function value, continuing training until a preset first training completion condition is met, and obtaining a trained image classification model comprises the following steps: adjusting the parameters of the image classification model based on the first loss function value, the second loss function value and the third loss function value, continuing training until a preset second training completion condition is met, and obtaining a trained image classification model.

3. The method of claim 1, wherein, The classifier includes sub-classifiers of different granularities, and the classifier inputs the sparse features into the image classification model to obtain a predicted class label of each image in the image bag, including: inputting the sparse features into multiple sub-classifiers of different granularities to output class prediction results of each image in the image bag at different granularities.

4. The method of claim 3, wherein, The class proportion label of the image bag includes sub-class proportion labels of different granularities, and the first loss function value is determined based on the predicted class proportion label of the image bag and the class proportion label of the image bag, including: determining multiple sub-loss function values for the image bag based on the respective sub-predicted class proportion labels of the image bag at different granularities and the respective sub-class proportion labels of the image bag at different granularities; determining the first loss function value based on the multiple sub-loss function values.

5. An image classification model training apparatus, characterized by comprising: including: an acquisition module configured to acquire a base feature of an image bag in a sample set and a class proportion label of the image bag; an input module configured to input the base feature into a sparse dictionary learning sub-model of an image classification model to obtain a target sparse feature of the image bag; the inputting of the base feature into the sparse dictionary learning sub-model of the image classification model to obtain the target sparse feature of the image bag includes: determining a first target dictionary of the sparse dictionary learning sub-model; determining a first sparse feature of the image bag based on the base feature of the image bag and the first target dictionary; determining a second sparse feature of the image bag based on the base feature of the image bag and a target sub-dictionary related to the label proportion of the image bag in the first target dictionary; constraining the first target dictionary based on the first sparse feature of the image bag and the second sparse feature of the image bag to obtain a second target dictionary; determining the target sparse feature of the image bag based on the second target dictionary and the base feature of the image bag; the determination of the first target dictionary of the sparse dictionary learning sub-model includes: determining a sparse feature corresponding to the base feature of the image bag under a current dictionary; determining a reconstructed feature of the base feature based on the sparse feature and the dictionary; optimizing the dictionary to obtain the first target dictionary, with the optimization objective being to minimize a difference value between the reconstructed feature and the base feature; the constraining of the first target dictionary based on the first sparse feature of the image bag and the second sparse feature of the image bag to obtain the second target dictionary includes: obtaining a first reconstructed deep feature based on the first sparse feature of the image bag and the first target dictionary; obtaining a second reconstructed deep feature based on the first sparse feature of the image bag and the target sub-dictionary; calculating a second loss function value based on the first reconstructed deep feature and the base feature; calculating a third loss function value based on the second reconstructed deep feature and the base feature; adjusting the first target dictionary based on the second loss function value and the third loss function value until a preset iteration completion condition is reached to obtain the second target dictionary; The input module is further configured to input the sparse features into a classifier of the image classification model to obtain a predicted class label of each image in the image bag; The determination module is configured to determine a predicted class proportion label of the image bag based on the predicted class label of each image in the image bag; The determination module is configured to determine a first loss function value based on the predicted class proportion label of the image bag and the class proportion label of the image bag; The adjustment module is configured to adjust parameters of the image classification model based on the first loss function value, continue training until a preset first training completion condition is met, and obtain a trained image classification model.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1-4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-4.

Images