Image classification methods, devices, electronic equipment and storage media

Abstract

This disclosure relates to an image classification method, apparatus, electronic device, and storage medium. The method includes: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain an image classification result corresponding to the image to be classified. The image classification model is obtained by training a preset model based on a target sample set. The target sample set is obtained by: dividing the preset sample set into multiple first sample subsets; for each first sample subset, determining the resampling frequency corresponding to sample images of each category based on a first quantity and a second quantity; determining a second sample subset corresponding to each first sample subset based on the resampling frequency and the second quantity; and obtaining the target sample set based on the first sample subsets and the second sample subsets. This disclosure can avoid underfitting of the image classification model for any category of image, improve the image classification effect of the image classification model, and thus ensure the accuracy of the image classification result.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image classification method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous development of artificial intelligence, image classification, as one of the important applications of AI, has received widespread attention. Currently, the main approach is to use sample sets to train image classification models, and then use these models to perform multi-label image classification on the images to be classified, thus obtaining the classification results. However, the diversity and quality of images in the sample set directly affect the classification performance of the model. In reality, sample sets often exhibit a long-tail distribution, meaning that a small portion of image categories (common classes) contain a large amount of data, while most categories (rare classes) contain only a small amount. This imbalance in data distribution can lead to underfitting of the rare classes by the image classification model, reducing its classification performance and ultimately affecting the accuracy of the classification results. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides an image classification method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of the present disclosure, an image classification method is provided, the method comprising:

[0005] Obtain the image to be classified;

[0006] The image to be classified is input into a pre-trained image classification model to obtain the image classification result corresponding to the image to be classified.

[0007] The image classification model is obtained by training a preset model based on a target sample set, which is obtained in the following way:

[0008] The preset sample set is divided into multiple first sample subsets; wherein each first sample subset includes multiple sample images and a category label corresponding to each sample image, the category label being used to indicate the category of the sample image, and the same first sample subset includes sample images of multiple categories;

[0009] For each of the first sample subsets, the resampling frequency corresponding to the sample images of each category is determined according to a first quantity and a second quantity; wherein, the first quantity is used to represent the total number of sample images in the first sample subset, the second quantity is used to represent the number of sample images of each category in the first sample subset, and the resampling frequency is used to represent the sampling frequency corresponding to the sample images of each category when the first sample subset is resampled according to the first quantity and the second quantity.

[0010] Based on the resampling frequency and the second quantity, determine the second sample subset corresponding to each first sample subset;

[0011] The target sample set is obtained based on the first sample subset and the second sample subset.

[0012] Optionally, determining the resampling frequency corresponding to the sample images of each category based on the first quantity and the second quantity includes:

[0013] The resampling frequency is determined using a frequency calculation formula based on the number of categories of sample images included in each of the first sample subsets, the first number, and the second number.

[0014] The frequency calculation formula includes:

[0015]

[0016] Among them, Pa (i) Let c be the resampling frequency corresponding to the i-th category, N be the number of categories, and c be the resampling frequency. i Let C be the second quantity of the i-th category, C be the first quantity, and α be a preset coefficient, where 0 ≤ α ≤ 1.

[0017] Optionally, determining the second sample subset corresponding to each first sample subset based on the resampling frequency and the second quantity includes:

[0018] Based on the target resampling frequency corresponding to the high-frequency image category in each of the first sample subsets, and the second number of the high-frequency image category, the number of subset sample images corresponding to the first sample subset is determined, wherein the high-frequency image category is the category with the largest second number in the first sample subset;

[0019] Based on the number of sample images in the subset and the resampling frequency, determine the number of category sample images corresponding to each category in the first sample subset;

[0020] The second sample subset is determined based on the number of sample images of the stated category and the second quantity.

[0021] Optionally, determining the second sample subset based on the number of sample images of the category and the second quantity includes:

[0022] For each of the first sample subsets, the difference between the number of category sample images corresponding to each category and the second number of categories is taken as the difference number corresponding to the category. Based on the difference number corresponding to the category, the number of sample images corresponding to the category is selected from the second number of sample images of the category to obtain the second sample subset.

[0023] Optionally, obtaining the target sample set based on the first sample subset and the second sample subset includes:

[0024] Each first sample subset and the corresponding second sample subset are concatenated to obtain the combined sample subset corresponding to the first sample subset;

[0025] Each combined sample subset corresponding to the first sample subset is added to a preset dataset to obtain the target sample set.

[0026] Optionally, the image classification model is trained in the following way:

[0027] The preset model is trained sequentially based on each of the combined sample subsets in the target sample set to obtain the image classification model.

[0028] According to a second aspect of the present disclosure, an image classification apparatus is provided, the apparatus comprising:

[0029] The acquisition module is configured to acquire images to be classified.

[0030] The processing module is configured to input the image to be classified into a pre-trained image classification model to obtain the image classification result corresponding to the image to be classified.

[0031] The image classification model is obtained by training a preset model based on a target sample set, which is obtained in the following way:

[0032] The preset sample set is divided into multiple first sample subsets; wherein each first sample subset includes multiple sample images and a category label corresponding to each sample image, the category label being used to indicate the category of the sample image, and the same first sample subset includes sample images of multiple categories;

[0033] For each of the first sample subsets, the resampling frequency corresponding to the sample images of each category is determined according to a first quantity and a second quantity; wherein, the first quantity is used to represent the total number of sample images in the first sample subset, the second quantity is used to represent the number of sample images of each category in the first sample subset, and the resampling frequency is used to represent the sampling frequency corresponding to the sample images of each category when the first sample subset is resampled according to the first quantity and the second quantity.

[0034] Based on the resampling frequency and the second quantity, determine the second sample subset corresponding to each first sample subset;

[0035] The target sample set is obtained based on the first sample subset and the second sample subset.

[0036] Optionally, the processing module is configured to:

[0037] The resampling frequency is determined using a frequency calculation formula based on the number of categories of sample images included in each of the first sample subsets, the first number, and the second number.

[0038] The frequency calculation formula includes:

[0039]

[0040] Among them, Pa (i) Let c be the resampling frequency corresponding to the i-th category, N be the number of categories, and c be the resampling frequency. i Let C be the second quantity of the i-th category, C be the first quantity, and α be a preset coefficient, where 0 ≤ α ≤ 1.

[0041] Optionally, the processing module is configured to:

[0042] Based on the target resampling frequency corresponding to the high-frequency image category in each of the first sample subsets, and the second number of the high-frequency image category, the number of subset sample images corresponding to the first sample subset is determined, wherein the high-frequency image category is the category with the largest second number in the first sample subset;

[0043] Based on the number of sample images in the subset and the resampling frequency, determine the number of category sample images corresponding to each category in the first sample subset;

[0044] The second sample subset is determined based on the number of sample images of the stated category and the second quantity.

[0045] Optionally, the processing module is configured to:

[0046] For each of the first sample subsets, the difference between the number of category sample images corresponding to each category and the second number of categories is taken as the difference number corresponding to the category. Based on the difference number corresponding to the category, the number of sample images corresponding to the category is selected from the second number of sample images of the category to obtain the second sample subset.

[0047] Optionally, the processing module includes:

[0048] The connection submodule is configured to connect each first sample subset and the second sample subset corresponding to the first sample subset to obtain a combined sample subset corresponding to the first sample subset;

[0049] The determination submodule is configured to add the combined sample subset corresponding to each of the first sample subsets to a preset dataset to obtain the target sample set.

[0050] Optionally, the image classification model is trained in the following way:

[0051] The preset model is trained sequentially based on each of the combined sample subsets in the target sample set to obtain the image classification model.

[0052] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0053] processor;

[0054] Memory used to store processor-executable instructions;

[0055] The processor is configured to perform the steps of the image classification method provided in the first aspect of this disclosure.

[0056] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the image classification method provided in the first aspect of the present disclosure.

[0057] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: This disclosure obtains an image to be classified and inputs it into an image classification model to obtain an image classification result. The image classification model is obtained by training a preset model based on a target sample set. The target sample set is obtained as follows: first, the preset sample set is divided into multiple first sample subsets; then, for each first sample subset, a resampling frequency corresponding to the sample images of each category is determined based on a first quantity and a second quantity; and a second sample subset corresponding to each first sample subset is determined based on the resampling frequency and the second quantity. Finally, the target sample set is obtained based on the first sample subsets and the second sample subsets. This disclosure constructs a target sample set by utilizing the first sample subsets divided from the preset sample set and the second sample subsets determined from the first sample subsets. The constructed target sample set can ensure a balanced distribution of sample images of different categories, avoiding underfitting of the image classification model for any category of image, improving the image classification effect of the image classification model, and thus ensuring the accuracy of the image classification result.

[0058] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0060] Figure 1 This is a flowchart illustrating an image classification method according to an exemplary embodiment.

[0061] Figure 2 This is a block diagram illustrating an image classification apparatus according to an exemplary embodiment.

[0062] Figure 3 It is based on Figure 2 The illustrated embodiment shows a block diagram of a processing module.

[0063] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0064] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0065] Before introducing the image classification method, apparatus, electronic device, and storage medium provided in this disclosure, the application scenarios involved in the various embodiments of this disclosure are first described. These application scenarios can be any scenario requiring image classification using an image classification model, such as intelligent video analysis, traffic scene recognition, image retrieval, and album categorization. Furthermore, the execution entity of the image classification method provided in this disclosure can be a server (i.e., the image classification process is performed on the server) or a terminal device (i.e., the image classification process is performed on the terminal device). The server can include, but is not limited to, physical servers, server clusters, or cloud servers. The terminal can be, for example, a smartphone, tablet, smartwatch, smart bracelet, PDA (Personal Digital Assistant), or a fixed terminal such as a desktop computer.

[0066] Figure 1 This is a flowchart illustrating an image classification method according to an exemplary embodiment. Figure 1 As shown, the method may include the following steps:

[0067] In step 101, the image to be classified is obtained.

[0068] In step 102, the image to be classified is input into a pre-trained image classification model to obtain the image classification result corresponding to the image to be classified.

[0069] The image classification model is obtained by training a pre-defined model on a target sample set. The target sample set is obtained as follows: First, the pre-defined sample set is divided into multiple first sample subsets. Each first sample subset includes multiple sample images and a category label corresponding to each sample image. The category label indicates the category of the sample image. The same first sample subset includes sample images from multiple categories. Then, for each first sample subset, a resampling frequency corresponding to each category of sample images is determined based on a first quantity and a second quantity. The first quantity represents the total number of sample images in the first sample subset, the second quantity represents the number of sample images of each category in the first sample subset, and the resampling frequency represents the sampling frequency corresponding to each category of sample images when resampling the first sample subset according to the first and second quantities. Then, based on the resampling frequency and the second quantity, a second sample subset corresponding to each first sample subset is determined, and the target sample set is obtained based on the first and second sample subsets.

[0070] For example, when classifying images, an image classification model needs to be pre-trained. First, a large number of images can be collected from the internet and classified into a predetermined number of categories. Then, the images in each category are used as sample images for that category, generating a pre-set sample set containing the specified number of categories. Each category represents a particular type of image; for example, a category could be "cat" images, meaning all images containing the word "cat" belong to that category. In reality, because some categories of images appear less frequently, the number of images that can be collected for these categories is also smaller. This results in fewer sample images for these categories in the pre-set sample set, leading to a long-tailed distribution phenomenon. For example, when a specified number of categories include four categories: "cat", "dog", "grassland" and "lemur", the number of images belonging to the "lemur" category that can be collected is relatively small because the "lemur" category appears less frequently (the "lemur" category can be considered a rare category, while the "cat", "dog" and "grassland" categories can be considered common categories). This will result in fewer sample images of the "lemur" category in the preset sample set.

[0071] To avoid the impact of long-tail distribution on the image classification performance of the image classification model, a balanced sample set with different categories of sample images can be constructed based on the preset sample set to train the image classification model and ensure its image classification performance. Specifically, the preset sample set can be randomly shuffled and then evenly divided to obtain multiple first sample subsets (each first sample subset contains the same number of sample images). For example, when the preset sample set includes 810,000 sample images, it can be randomly shuffled and evenly divided into 10 first sample subsets, with each first sample subset containing approximately 81,000 sample images. Then, based on the number of categories in each first sample subset, the first number of sample images in the first sample subset, and the second number of sample images for each category in the first sample subset, the resampling frequency corresponding to the sample images for each category in the first sample subset can be set. The resampling frequency corresponding to the sample images for each category can be understood as the proportion of sample images of that category among all sample images to ensure the image classification performance of the image classification model.

[0072] Then, for each first sample subset, a second sample subset can be determined based on the resampling frequency of sample images corresponding to each category in the first sample subset and the second quantity of sample images for that category. The second sample subset is actually obtained by adjusting the second quantity of sample images for each category based on the resampling frequency of sample images corresponding to each category in the first sample subset. The distribution of sample images of different categories in the first sample subset is the same as the distribution of sample images of different categories in the preset sample set, which makes the first sample subset more focused on training the feature learning ability of the image classification model. The balanced distribution of sample images of different categories in the second sample subset makes the second sample subset more focused on training the classification ability of the image classification model. Finally, each first sample subset and its corresponding second sample subset can be concatenated to obtain a combined sample subset corresponding to the first sample subset. This combined sample subset corresponding to each first sample subset is then added to the preset dataset (which is empty before adding the combined sample subset) to obtain the target sample set.

[0073] After obtaining the target sample set, a preset model can be trained using the target sample set, resulting in an image classification model. A combined sample subset obtained by directly concatenating the first and second sample subsets enhances the image classification model's classification ability beyond its feature learning capabilities, thus ensuring the image classification performance of the model trained on the target sample set composed of the combined sample subsets. Then, after acquiring the image to be classified, it can be used as input to the image classification model to obtain the output category corresponding to the image to be classified, and this category can be used as the image classification result.

[0074] In summary, this disclosure obtains image classification results by acquiring an image to be classified and inputting it into an image classification model. The image classification model is trained on a preset model using a target sample set. The target sample set is obtained as follows: first, the preset sample set is divided into multiple first sample subsets; then, for each first sample subset, a resampling frequency corresponding to the sample images of each category is determined based on a first quantity and a second quantity; and a second sample subset corresponding to each first sample subset is determined based on the resampling frequency and the second quantity. Finally, the target sample set is obtained based on the first and second sample subsets. This disclosure constructs the target sample set using the first sample subsets divided from the preset sample set and the second sample subsets determined from the first sample subsets. The constructed target sample set ensures a balanced distribution of sample images of different categories, avoids underfitting of the image classification model for any category of image, improves the image classification effect of the image classification model, and thus ensures the accuracy of the image classification results.

[0075] Optionally, determining the resampling frequency for each category of sample images based on the first and second quantities is achieved in the following way:

[0076] The resampling frequency is determined using a frequency calculation formula based on the number of categories, the first number, and the second number of sample images included in each first sample subset.

[0077] The frequency calculation formula includes:

[0078]

[0079] Pa (i) Let c be the resampling frequency corresponding to the i-th category, N be the number of categories, and c be the frequency of resampling. i Let C be the second quantity of the i-th category, C be the first quantity, and α be a preset coefficient, where 0 ≤ α ≤ 1.

[0080] For example, after dividing a preset sample set into multiple first sample subsets, the resampling frequency corresponding to the sample images of each category in the first sample subset can be determined using a frequency calculation formula based on the total number of categories of the sample images included in each first sample subset, the first number of sample images in the first sample subset, and the second number of sample images of each category in the first sample subset. For example, when a first sample subset includes 4 categories, the first number is 80,000, and the second numbers of the 4 categories included in the first sample subset are 10,000, 10,000, 20,000, and 40,000 respectively, if α is 0.5, then the resampling frequency corresponding to the category with a second number of 20,000 in the first sample subset is:

[0081] It should be noted that when α is 0, the frequency calculation formula becomes The resampling frequency for each category obtained at this point is actually the ratio of the number of sample images of that category in the preset sample set to the total number of sample images in the preset sample set. The distribution of sample images of different categories in the determined second sample subset is the same as that in the first sample subset, i.e., there is a long-tail distribution phenomenon, which will reduce the classification accuracy of the image classification model for the second category with a smaller number of samples. When α is 1, the frequency calculation formula becomes At this point, the resampling frequency for each category is the same, and the distribution of sample images of different categories in the determined second sample subset is the same. Using such a second sample subset to train the image classification model will reduce the classification accuracy of the image classification model for the second category with a larger number of samples. Therefore, to avoid the above two situations, α can be selected between 0.4 and 0.6.

[0082] Optionally, determining the second sample subset corresponding to each first sample subset based on the resampling frequency and the second quantity may include the following steps:

[0083] In step a, the number of subset sample images corresponding to the first sample subset is determined based on the target resampling frequency corresponding to the high-frequency image category in each first sample subset and the second number of high-frequency image categories.

[0084] Among them, the high-frequency image category is the second largest category in the first sample subset.

[0085] For example, for each first sample subset, the category with the second largest number of images can be selected from all categories in the first sample subset as the high-frequency image category. Based on the high-frequency image category and its corresponding target resampling frequency, the number of subset sample images corresponding to that first sample subset can be determined using a first formula. The first formula includes: all_num is the number of sample images in the subset, Pa (w) c is the target resampling frequency. w This is the second number for the high-frequency image category.

[0086] In step b, the number of category sample images corresponding to each category in the first sample subset is determined based on the number of subset sample images and the resampling frequency.

[0087] Furthermore, based on the number of subset sample images corresponding to each first sample subset and the resampling frequency of each category in the first sample subset, the number of category sample images corresponding to each category in the first sample subset can be determined using the second formula. The second formula includes: cNum (i) =Pa (i)·all_num, cNum (i) denoted as the number of category sample images corresponding to the i-th category.

[0088] In step c, a second sample subset is determined based on the number of category sample images and the second quantity.

[0089] For example, for each first sample subset, the difference between the number of category sample images corresponding to each category in the first sample subset and the second number of images for that category can be used as the difference number for that category, which can be expressed as: extraNum (i) =cNum (i) -c i Among them, extraNum (i) Let be the number of differences corresponding to the i-th category. Then, based on the number of differences corresponding to that category, select a number of sample images corresponding to that category from the second number of sample images of that category to obtain the second sample subset.

[0090] For example, for the i-th category in this first sample subset, if extraNum (i) If ≤0, then cNum can be randomly selected from the second number of sample images of the i-th category. (i) One sample image is used as the sample image in the second sample subset. If extraNum (i) If the value is greater than 0, then the first target quantity and the second target quantity can be calculated using the third and fourth formulas respectively. Then, the second number of sample images of the i-th category are repeatedly extracted from the first target quantity to form the sample images in the second sample subset. Finally, the second target quantity of sample images is randomly selected from the second number of sample images of the i-th category to form the sample images in the second sample subset. The third formula includes: rpz = extraNum (i) / / c (i) The fourth formula includes: rpy = extraNum (i) %c (i) rpz represents the first target quantity, rpy represents the second target quantity, " / / " represents the integer calculation, and "%" represents the remainder calculation.

[0091] Alternatively, the image classification model is trained in the following way:

[0092] The preset model is trained sequentially based on each combined sample subset in the target sample set to obtain the image classification model.

[0093] For example, each subset of combined samples in the target sample set can be used to train the preset model in turn. This allows the image classification model to switch between feature learning ability and classification ability repeatedly, avoiding the negative impact of forgetting in the image classification model itself and the long-tail distribution phenomenon. This enables the image classification model to take into account both feature learning ability and classification ability at the same time, thereby improving the image classification effect of the image classification model.

[0094] In summary, this disclosure obtains image classification results by acquiring an image to be classified and inputting it into an image classification model. The image classification model is trained on a preset model using a target sample set. The target sample set is obtained as follows: first, the preset sample set is divided into multiple first sample subsets; then, for each first sample subset, a resampling frequency corresponding to the sample images of each category is determined based on a first quantity and a second quantity; and a second sample subset corresponding to each first sample subset is determined based on the resampling frequency and the second quantity. Finally, the target sample set is obtained based on the first and second sample subsets. This disclosure constructs the target sample set using the first sample subsets divided from the preset sample set and the second sample subsets determined from the first sample subsets. The constructed target sample set ensures a balanced distribution of sample images of different categories, avoids underfitting of the image classification model for any category of image, improves the image classification effect of the image classification model, and thus ensures the accuracy of the image classification results.

[0095] Figure 2 This is a block diagram illustrating an image classification apparatus according to an exemplary embodiment. Figure 2 As shown, the device 200 may include an acquisition module 201 and a processing module 202.

[0096] The acquisition module 201 is configured to acquire images to be classified.

[0097] The processing module 202 is configured to input the image to be classified into a pre-trained image classification model to obtain the image classification result corresponding to the image to be classified.

[0098] The image classification model is obtained by training a pre-defined model using a target sample set, which is obtained in the following way:

[0099] The preset sample set is divided into multiple first sample subsets, where each first sample subset includes multiple sample images and a category label corresponding to each sample image. The category label is used to indicate the category of the sample image. The same first sample subset includes sample images of multiple categories.

[0100] For each first sample subset, the resampling frequency corresponding to the sample images of each category is determined according to the first quantity and the second quantity. The first quantity is used to represent the total number of sample images in the first sample subset, the second quantity is used to represent the number of sample images of each category in the first sample subset, and the resampling frequency is used to represent the sampling frequency corresponding to the sample images of each category when the first sample subset is resampled according to the first quantity and the second quantity.

[0101] Based on the resampling frequency and the second quantity, determine the second sample subset corresponding to each first sample subset.

[0102] The target sample set is obtained based on the first and second sample subsets.

[0103] Optionally, the processing module 202 is configured as follows:

[0104] The resampling frequency is determined using a frequency calculation formula based on the number of categories, the first number, and the second number of sample images included in each first sample subset.

[0105] The frequency calculation formula includes:

[0106]

[0107] Among them, Pa (i) Let c be the resampling frequency corresponding to the i-th category, N be the number of categories, and c be the frequency of resampling. i Let C be the second quantity of the i-th category, C be the first quantity, and α be a preset coefficient, where 0 ≤ α ≤ 1.

[0108] Optionally, the processing module 202 is configured as follows:

[0109] Based on the target resampling frequency corresponding to the high-frequency image category in each first sample subset, and the second number of high-frequency image categories, the number of subset sample images corresponding to the first sample subset is determined, where the high-frequency image category is the category with the largest second number in the first sample subset.

[0110] The number of category sample images corresponding to each category in the first sample subset is determined based on the number of sample images in the subset and the resampling frequency.

[0111] The second sample subset is determined based on the number of sample images of each category and the second number.

[0112] Optionally, the processing module 202 is configured as follows:

[0113] For each first sample subset, the difference between the number of category sample images corresponding to each category and the second number of categories is taken as the difference number corresponding to that category. Based on the difference number corresponding to that category, a number of sample images corresponding to that category are selected from the second number of sample images of that category to obtain the second sample subset.

[0114] Figure 3 It is based on Figure 2 The illustrated embodiment shows a block diagram of a processing module. For example... Figure 3 As shown, the processing module 202 includes:

[0115] The connection submodule 2021 is configured to connect each first sample subset and the second sample subset corresponding to the first sample subset to obtain the combined sample subset corresponding to the first sample subset.

[0116] The determination submodule 2022 is configured to add the combined sample subset corresponding to each first sample subset to a preset dataset to obtain the target sample set.

[0117] Alternatively, the image classification model is trained in the following way:

[0118] The preset model is trained sequentially based on each combined sample subset in the target sample set to obtain the image classification model.

[0119] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0120] In summary, this disclosure obtains image classification results by acquiring an image to be classified and inputting it into an image classification model. The image classification model is trained on a preset model using a target sample set. The target sample set is obtained as follows: first, the preset sample set is divided into multiple first sample subsets; then, for each first sample subset, a resampling frequency corresponding to the sample images of each category is determined based on a first quantity and a second quantity; and a second sample subset corresponding to each first sample subset is determined based on the resampling frequency and the second quantity. Finally, the target sample set is obtained based on the first and second sample subsets. This disclosure constructs the target sample set using the first sample subsets divided from the preset sample set and the second sample subsets determined from the first sample subsets. The constructed target sample set ensures a balanced distribution of sample images of different categories, avoids underfitting of the image classification model for any category of image, improves the image classification effect of the image classification model, and thus ensures the accuracy of the image classification results.

[0121] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the image classification steps provided in this disclosure.

[0122] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0123] Reference Figure 4 The electronic device 400 may include one or more of the following components: processing component 402, memory 404, power component 406, multimedia component 408, audio component 410, input / output (I / O) interface 412, sensor component 414, and communication component 416.

[0124] Processing component 402 typically controls the overall operation of electronic device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the image classification method described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0125] Memory 404 is configured to store various types of data to support the operation of electronic device 400. Examples of this data include instructions for any application or method operating on electronic device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0126] Power component 406 provides power to various components of electronic device 400. Power component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.

[0127] Multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0128] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0129] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0130] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 may detect the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0131] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other devices. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0132] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image classification method described above.

[0133] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to complete the image classification method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0134] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the image classification method described above when executed by the programmable device.

[0135] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0136] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image classification method, characterized in that, The method includes: Obtain the image to be classified; The image to be classified is input into a pre-trained image classification model to obtain the image classification result corresponding to the image to be classified. The image classification model is obtained by training a preset model based on a target sample set, which is obtained in the following way: The preset sample set is divided into multiple first sample subsets; wherein each first sample subset includes multiple sample images and a category label corresponding to each sample image, the category label being used to indicate the category of the sample image, and the same first sample subset includes sample images of multiple categories; For each of the first sample subsets, the resampling frequency corresponding to the sample images of each category is determined according to a first quantity and a second quantity; wherein, the first quantity is used to represent the total number of sample images in the first sample subset, the second quantity is used to represent the number of sample images of each category in the first sample subset, and the resampling frequency is used to represent the sampling frequency corresponding to the sample images of each category when the first sample subset is resampled according to the first quantity and the second quantity. Based on the resampling frequency and the second quantity, determine the second sample subset corresponding to each first sample subset; The target sample set is obtained based on the first sample subset and the second sample subset; The step of obtaining the target sample set based on the first sample subset and the second sample subset includes: Each first sample subset and the corresponding second sample subset are concatenated to obtain the combined sample subset corresponding to the first sample subset; Each combined sample subset corresponding to the first sample subset is added to a preset dataset to obtain the target sample set.

2. The method according to claim 1, characterized in that, The step of determining the resampling frequency corresponding to the sample images of each category based on the first quantity and the second quantity includes: The resampling frequency is determined using a frequency calculation formula based on the number of categories of sample images included in each of the first sample subsets, the first number, and the second number. The frequency calculation formula includes: in, Let c be the resampling frequency corresponding to the i-th category, N be the number of categories, and c be the resampling frequency. i Let C be the second quantity of the i-th category, and C be the first quantity. For preset coefficients, .

3. The method according to claim 1, characterized in that, The step of determining the second sample subset corresponding to each first sample subset based on the resampling frequency and the second quantity includes: Based on the target resampling frequency corresponding to the high-frequency image category in each of the first sample subsets, and the second number of the high-frequency image category, the number of subset sample images corresponding to the first sample subset is determined, wherein the high-frequency image category is the category with the largest second number in the first sample subset; Based on the number of sample images in the subset and the resampling frequency, determine the number of category sample images corresponding to each category in the first sample subset; The second sample subset is determined based on the number of sample images of the stated category and the second quantity.

4. The method according to claim 3, characterized in that, The step of determining the second sample subset based on the number of sample images of the category and the second number includes: For each of the first sample subsets, the difference between the number of category sample images corresponding to each category and the second number of categories is taken as the difference number corresponding to the category. Based on the difference number corresponding to the category, the number of sample images corresponding to the category is selected from the second number of sample images of the category to obtain the second sample subset.

5. The method according to claim 1, characterized in that, The image classification model was trained in the following way: The preset model is trained sequentially based on each of the combined sample subsets in the target sample set to obtain the image classification model.

6. An image classification device, characterized in that, The device includes: The acquisition module is configured to acquire images to be classified. The processing module is configured to input the image to be classified into a pre-trained image classification model to obtain the image classification result corresponding to the image to be classified. The image classification model is obtained by training a preset model based on a target sample set, which is obtained in the following way: The preset sample set is divided into multiple first sample subsets; wherein each first sample subset includes multiple sample images and a category label corresponding to each sample image, the category label being used to indicate the category of the sample image, and the same first sample subset includes sample images of multiple categories; For each of the first sample subsets, the resampling frequency corresponding to the sample images of each category is determined according to a first quantity and a second quantity; wherein, the first quantity is used to represent the total number of sample images in the first sample subset, the second quantity is used to represent the number of sample images of each category in the first sample subset, and the resampling frequency is used to represent the sampling frequency corresponding to the sample images of each category when the first sample subset is resampled according to the first quantity and the second quantity. Based on the resampling frequency and the second quantity, determine the second sample subset corresponding to each first sample subset; The target sample set is obtained based on the first sample subset and the second sample subset; The processing module includes: The connection submodule is configured to connect each first sample subset and the second sample subset corresponding to the first sample subset to obtain a combined sample subset corresponding to the first sample subset; The determination submodule is configured to add the combined sample subset corresponding to each of the first sample subsets to a preset dataset to obtain the target sample set.

7. The apparatus according to claim 6, characterized in that, The processing module is configured as follows: The resampling frequency is determined using a frequency calculation formula based on the number of categories of sample images included in each of the first sample subsets, the first number, and the second number. The frequency calculation formula includes: in, Let c be the resampling frequency corresponding to the i-th category, N be the number of categories, and c be the resampling frequency. i Let C be the second quantity of the i-th category, and C be the first quantity. For preset coefficients, .

8. The apparatus according to claim 6, characterized in that, The processing module is configured as follows: Based on the target resampling frequency corresponding to the high-frequency image category in each of the first sample subsets, and the second number of the high-frequency image category, the number of subset sample images corresponding to the first sample subset is determined, wherein the high-frequency image category is the category with the largest second number in the first sample subset; Based on the number of sample images in the subset and the resampling frequency, determine the number of category sample images corresponding to each category in the first sample subset; The second sample subset is determined based on the number of sample images of the stated category and the second quantity.

9. The apparatus according to claim 8, characterized in that, The processing module is configured as follows: For each of the first sample subsets, the difference between the number of category sample images corresponding to each category and the second number of categories is taken as the difference number corresponding to the category. Based on the difference number corresponding to the category, the number of sample images corresponding to the category is selected from the second number of sample images of the category to obtain the second sample subset.

10. The apparatus according to claim 6, characterized in that, The image classification model was trained in the following way: The preset model is trained sequentially based on each of the combined sample subsets in the target sample set to obtain the image classification model.

11. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the steps of the method according to any one of claims 1-5.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1-5.

Images