Image classification method and related apparatus, device

Abstract

The application discloses an image classification method and related devices and equipment, wherein the image classification method comprises: obtaining a to-be-classified image; performing feature extraction on the to-be-classified image to obtain a feature vector of the to-be-classified image; performing first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector to obtain a first classification result, a second classification result, a foreground detection result and a feature classification result of the feature vector; and obtaining a classification result of the to-be-classified image based on the first classification result, the second classification result, the foreground detection result and the feature classification result; wherein the classification type of the second classification processing comprises the classification type of the first classification processing and the corresponding sub-type, and the classification type of the second classification processing is the same as the classification type of the feature classification. The above scheme can improve the image classification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image classification, and in particular to an image classification method and related device and equipment. BACKGROUND

[0002] With the continuous development of science and technology, image classification technology is widely used to solve various problems, for example, image recognition, pedestrian detection, medical diagnosis and other problems are realized through deep learning models.

[0003] Image classification schemes distinguish different categories of images through supervised learning. Compared with detection, segmentation and tracking technology, the classification technology scheme has the advantages of fast processing speed and simple labeling. The current classification task adopts supervised information to divide a single image into a large category, and the feature layer mainly extracts global class information.

[0004] The above classification scheme always focuses on global features when extracting image features. When processing images with complex backgrounds, the classification accuracy is often low. SUMMARY

[0005] The present application provides an image classification method and related device and equipment, which solves the problem of low image classification accuracy in the prior art.

[0006] The present application provides an image classification method, comprising: obtaining a to-be-classified image; performing feature extraction on the to-be-classified image to obtain a feature vector of the to-be-classified image; performing first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector to obtain a first classification result, a second classification result, a foreground detection result and a feature classification result of the feature vector; obtaining a classification result of the to-be-classified image based on the first classification result, the second classification result, the foreground detection result and the feature classification result; wherein the classification type of the second classification processing includes the classification type of the first classification processing and its corresponding sub-type, and the classification type of the second classification processing is the same as the classification type of the feature classification.

[0007] The step of performing first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector to obtain a first classification result, a second classification result, a foreground detection result and a feature classification result of the feature vector comprises: performing foreground detection on the feature vector to obtain a foreground detection result of the feature vector; performing first classification processing, second classification processing and feature classification processing on the foreground detection result of the feature vector respectively to obtain a first classification result of the feature vector, a second classification result of the feature vector and a feature classification result of the feature vector.

[0008] The step of obtaining the classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result comprises: in response to the classification type of the second classification result being the same as the classification type of the feature classification result, and the second classification result being the first classification result or a sub-type corresponding to the first classification result, determining the second classification result and the foreground detection result as the classification result of the image to be classified.

[0009] The first classification result comprises the confidence of each classification type of the first classification processing, the second classification result comprises the confidence of each classification type of the second classification processing, and the feature classification result comprises the confidence of each classification type of the feature classification processing; the step of obtaining the classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result comprises: performing product processing on the confidence of each classification type of the second classification processing, the confidence of each classification type of the corresponding first classification processing and the confidence of each classification type of the corresponding feature classification processing respectively to obtain a plurality of product values; and performing logical combination on the classification type of the second classification processing corresponding to the product value with the largest value and the foreground detection result to obtain the classification result of the image to be classified.

[0010] The step of performing feature extraction on the image to be classified to obtain the feature vector of the image to be classified comprises: performing feature extraction on the image to be classified through the feature extraction network of the image classification model to obtain the feature vector of the image to be classified; the steps of performing first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector respectively to obtain the first classification result of the feature vector, the second classification result of the feature vector, the foreground detection result of the feature vector and the feature classification result of the feature vector comprise: performing foreground detection processing on the feature vector through the foreground detection network of the image classification model to obtain the foreground detection result of the feature vector; performing first classification processing on the feature vector through the first classification network of the image classification model to obtain the first classification result of the feature vector; performing second classification processing on the feature vector through the second classification network of the image classification model to obtain the second classification result of the feature vector; and performing feature classification processing on the feature vector using the feature standard library to obtain the feature classification result of the feature vector; wherein the image classification model at least comprises the feature extraction network, the first classification network, the second classification network and the foreground detection network, and the feature extraction network is cascaded with the first classification network, the second classification network and the foreground detection network.

[0011] The step of performing feature classification processing on the feature vector by using the feature standard library to obtain a feature classification result of the feature vector comprises: performing cosine similarity processing on the feature vector and each standard feature vector in the feature standard library respectively to obtain the similarity between the feature vector and each standard feature vector; and determining the classification type corresponding to the standard feature vector with the largest similarity as the feature classification result of the feature vector.

[0012] The step of obtaining the to-be-classified image comprises: obtaining a plurality of training images; performing feature extraction on each training image by using a feature extraction network of the training image classification model to obtain a training feature vector of each training image; performing first classification processing, second classification processing and foreground detection on the training feature vector by using a classification network of the training image classification model to obtain a first classification result of the training feature vector, a second classification result of the training feature vector and a foreground detection result of the training feature vector; and training the training image classification model by using the first classification result, the second classification result and the foreground detection result of each training image and a standard classification result corresponding to each training image until the image classification model is obtained.

[0013] The step of obtaining the plurality of training images comprises: obtaining a plurality of training images; wherein the standard first classification result, the standard second classification result and the standard foreground result are labeled on each training image; performing feature extraction on the training image to obtain a standard feature vector of the training image, and determining a classification type corresponding to the standard feature vector based on a plurality of standard second classification results of the training image; and obtaining the standard classification result of each training image by using the standard first classification result, the plurality of standard second classification results, the standard foreground result and the standard feature vector of each training image.

[0014] The step of training the training image classification model by using the first classification result, the second classification result, the foreground detection result and the feature classification result of each training image and the standard classification result corresponding to each training image until the image classification model is obtained comprises: training the training image classification model by using the first classification result, the second classification result, the foreground detection result and the feature classification result of each training image and the standard classification result corresponding to each training image based on an overall loss function until the image classification model is obtained.

[0015] The step of training the image classification model in the training based on the overall loss function and the first classification result, the second classification result, the foreground detection result and the feature classification result of each training image and the standard classification result corresponding to each training image comprises: training the image classification model in the training based on the first loss function and the first classification result and the corresponding standard first classification result in the standard classification result of each training feature vector; training the image classification model in the training based on the second loss function and the second classification result and the corresponding standard second classification result in the standard classification result of each training feature vector; training the image classification model in the training based on the third loss function and the foreground detection result and the corresponding standard foreground detection result in the standard classification result of each training feature vector; and training the image classification model in the training based on the fourth loss function and each training feature vector and the corresponding standard feature vector; wherein the overall loss function is determined by the sum of the first loss function, the second loss function, the third loss function and the fourth loss function after weighting processing.

[0016] The image classification method is applied to image pornography classification; the classification type of the first classification result comprises a normal type, a sexy type and a pornography type; the classification type of the second classification result and the feature classification result comprises the normal type, the sub-type of the sexy type and the sub-type of the pornography type; and the foreground detection result comprises position information of a region where a human body is located on each image to be classified.

[0017] The classification result of the image to be classified is determined as the normal type in response to the foreground detection size of the foreground detection result being not greater than a preset size.

[0018] The application further provides an image classification device, comprising: an acquisition module configured to acquire an image to be classified; a feature extraction module configured to perform feature extraction on the image to be classified to obtain a feature vector of the image to be classified; a classification module configured to perform first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector respectively to obtain a first classification result of the feature vector, a second classification result of the feature vector, a foreground detection result of the feature vector and a feature classification result of the feature vector; and a determination module configured to obtain a classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result; wherein the classification type of the second classification processing comprises the classification type of the first classification processing and the corresponding sub-type, and the classification type of the second classification processing is the same as the classification type of the feature classification.

[0019] The application further provides an electronic device comprising a memory and a processor coupled to each other, wherein the processor is configured to execute program instructions stored in the memory to implement any of the above image classification methods.

[0020] The application further provides a computer readable storage medium, which stores program instructions, and the program instructions are executed by a processor to implement any of the image classification methods.

[0021] According to the above scheme, the feature vector of the image to be classified is obtained by feature extraction on the image to be classified, and the first classification result, the second classification result, the foreground detection result and the feature classification result of the feature vector are obtained by first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector. Finally, the classification result of the image to be classified is obtained based on the first classification result, the second classification result, the foreground detection result and the feature classification result. The feature vector can be processed multiple times independently by the above method, which can avoid mutual interference and mutual influence between the processes, thereby improving the accuracy and reliability of the overall classification of the feature vector. The final classification image of the image to be classified is obtained by combining the classification results of the independent processes, which can weaken the excessive attention to invalid background in the image classification of the image to be classified in the complex background to some extent, thereby strengthening the attention to the specific foreground in the image to be classified and improving the accuracy and reliability of the image classification. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a flowchart of an embodiment of the image classification method of the application;

[0023] Figure 2 is a flowchart of another embodiment of the image classification method of the application;

[0024] Figure 3 is Figure 2 is a structural diagram of an embodiment of the image classification model in the embodiment;

[0025] Figure 4 is Figure 2 is a structural diagram of another embodiment of the image classification model in the embodiment;

[0026] Figure 5 is Figure 2 is a flowchart of an embodiment of the training method of the image classification model in the embodiment;

[0027] Figure 6 is a framework diagram of an embodiment of the image classification device of the application;

[0028] Figure 7 is a framework diagram of an embodiment of the electronic device of the application;

[0029] Figure 8 is a framework diagram of an embodiment of the computer readable storage medium of the application. DETAILED DESCRIPTION

[0030] The scheme of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0031] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, interfaces, techniques, in order to provide a thorough understanding of the present application.

[0032] The terms "system" and "network" are often used interchangeably herein. The term "and / or" herein is merely a description of the associated objects, and there can be three relationships, for example, A and / or B, which can be: A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects. In addition, "multiple" herein means two or more.

[0033] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the image classification method of the present application.

[0034] Step S11: obtaining the image to be classified.

[0035] First, the image to be classified is obtained. The classification standard of the image to be classified can be based on any object on the image, for example, the image to be classified can be classified by pornography, human body, gender, color, etc. Different classification standards can also be constructed based on different target objects of the image to be classified, for example: plant classification, animal classification, shape classification, etc. The classification of target objects can be set based on actual needs, which is not limited here.

[0036] The image to be classified obtained in this step can be one or more.

[0037] Step S12: feature extraction is performed on the image to be classified to obtain the feature vector of the image to be classified.

[0038] After obtaining the image to be classified, feature extraction is performed on the image to be classified to obtain the feature vector of the image to be classified. When the image to be classified is multiple, feature extraction is performed on the image to be classified in turn to obtain the feature vector of each image to be classified.

[0039] In one specific implementation, the feature extraction algorithm can include one or more of a Local Binary Patterns (LBP) feature extraction algorithm, a Histogram of Oriented Gradient (HOG) feature extraction algorithm, a Haar feature extraction operator, and a LoG (first-order edge extraction) feature extraction algorithm, without limitation.

[0040] In another specific implementation, the feature extraction model trained can be used to extract features of the image to be classified to obtain a feature vector of the image to be classified. The feature extraction model trained can be constructed based on one or more of a BackPropagation (BP) neural network, a Radial Basis Function (RBF) neural network, a linear neural network, a convolutional neural network, and a recurrent neural network, without limitation.

[0041] Step S13: performing first classification processing, second classification processing, foreground detection, and feature classification processing on the feature vector to obtain a first classification result, a second classification result, a foreground detection result, and a feature classification result of the feature vector.

[0042] After the feature vector is obtained, the first classification processing, the second classification processing, the foreground detection, and the feature classification processing are performed on the feature vector to obtain the first classification result, the second classification result, the foreground detection result, and the feature classification result of the feature vector.

[0043] In one specific application scenario, the four types of processing described above can be performed on the feature vector, specifically: the first classification processing is performed on the feature vector to obtain the first classification result of the feature vector; the second classification processing is performed on the feature vector to obtain the second classification result of the feature vector; the foreground detection is performed on the feature vector to obtain the foreground detection result of the feature vector; and the feature classification processing is performed on the feature vector to obtain the feature classification result of the feature vector.

[0044] The four types of processing in this application scenario are independent of each other and are performed in parallel. Although the classification objects of the four types of processing are the feature vector, the classification means and the classification results of the four types of processing are different. By performing multiple independent processing on the feature vector, the mutual interference and mutual influence between the processing can be avoided, thereby improving the accuracy and reliability of the overall classification of the feature vector.

[0045] In another specific application scenario, the feature vector can be subjected to foreground detection first to obtain a foreground detection result of the feature vector, and then the first classification processing, the second classification processing and the feature classification processing can be respectively performed based on the foreground detection result to obtain the first classification result of the feature vector, the second classification result of the feature vector and the feature classification result of the feature vector.

[0046] The application scenario first subjects the feature vector to foreground detection, and then respectively and independently performs the three kinds of classification processing based on the foreground detection result, so that the subsequent three kinds of classification processing pay more attention to the specific foreground in the image to be classified through the foreground detection in advance, thereby improving the classification accuracy and reliability and reducing the situation that the specific foreground cannot be focused in the image to be classified in a complex background, resulting in paying more attention to the background information.

[0047] In this embodiment, the classification types of the second classification processing include the classification types of the first classification processing and the corresponding subtypes, and the classification types of the second classification processing are the same as the classification types of the feature classification.

[0048] That is, the first classification processing can be a coarse classification of the image to be classified, the second classification processing can be a fine classification of the image to be classified, and the classification types of the second classification processing include the classification types of the first classification processing and the corresponding subtypes. In a specific application scenario, when the image classification is applied to image color classification, the first classification processing can be a coarse color classification of the image to be classified, and the classification types thereof can include white, red and blue. The second classification processing can be a fine color classification of the image to be classified, and the classification types thereof can include white, pink, magenta, blood red, sky blue and dark blue. Among them, the pink, magenta and blood red of the classification types of the second classification processing are the subtypes of the red of the classification types of the first classification processing, the sky blue and dark blue of the classification types of the second classification processing are the subtypes of the blue of the classification types of the first classification processing, and the white of the classification types of the second classification processing is the same as the white of the classification types of the first classification processing. The classification types and the number of the first classification processing, the second classification processing and the feature classification processing can be set based on actual needs, which are not limited herein.

[0049] Although the classification types of the second classification processing include the classification types of the first classification processing and the corresponding subtypes, the second classification of this embodiment is not based on the first classification result of the first classification, but is based on the feature vector of the image to be classified. The three kinds of classification processing of this embodiment are independent of each other to avoid mutual interference and mutual influence between the processes, thereby improving the accuracy and reliability of the overall classification of the feature vector.

[0050] The foreground detection refers to detecting a target object of the image to be classified based on the feature vector, for example, when the image is classified as pornography, the target object of the image to be classified can be a human body, and the foreground detection refers to detecting the human body on the image to be classified based on the feature vector. The foreground detection result can be output by marking a detection frame or position information of a foreground region on the image to be classified.

[0051] The feature classification refers to directly judging the feature similarity based on the feature vector, so as to determine the feature classification result of the feature vector. The object of the feature similarity judgment of the feature vector can be a feature vector in an image that has been classified in the image classification process, or a standard feature library constructed based on each classification type of the image in advance, which is not limited here.

[0052] The first classification result, the second classification result, and the feature classification result are independent of each other, so as to avoid mutual interference and mutual influence between the three classification processes, thereby improving the accuracy of the first classification result, the second classification result, and the feature classification result. However, the classification standards of the three classification processes are associated with each other, so that the first classification result, the second classification result, and the feature classification result can be mutually referenced and assisted in judgment, and finally the classification result of the image to be classified is obtained, thereby further improving the accuracy of the classification result of the image to be classified.

[0053] Step S14: obtaining the classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result, and the feature classification result.

[0054] After obtaining the first classification result, the second classification result, the feature classification result, and the foreground detection result, the classification result of the image to be classified is obtained based on the first classification result, the second classification result, the feature classification result, and the foreground detection result. The first classification result, the second classification result, the feature classification result, and the foreground detection result are combined, so as to obtain the classification result of the image to be classified.

[0055] In a specific application scenario, the first classification result, the second classification result, the feature classification result, and the foreground detection result can be logically combined, and the first classification result, the second classification result, the feature classification result, and the foreground detection result after logical combination are determined as the classification result of the image to be classified.

[0056] In one specific application scenario, the first classification result, the second classification result, and the feature classification result can be compared and judged. In response to the same substantial meaning of the three classification results, the second classification result and the foreground detection result after logical combination are determined as the classification result of the image to be classified, or the first classification result and the foreground detection result after logical combination are determined as the classification result of the image to be classified, or the feature classification result and the foreground detection result after logical combination are determined as the classification result of the image to be classified. The selection is based on the actual situation.

[0057] The form of logical combination can include comprehensive output through text, table, image, and the like.

[0058] The same feature vector is subjected to different classification processing, and the final classification image of the image to be classified is obtained based on the different classification results. This can weaken the excessive attention of image classification to invalid background in the image to be classified in a complex background, thereby strengthening the attention to specific foreground in the image to be classified and improving the image classification accuracy and reliability.

[0059] Through the above steps, the image classification method of the embodiment can obtain the feature vector of the image to be classified through feature extraction on the image to be classified, and obtain the first classification result, the second classification result, the foreground detection result, and the feature classification result of the feature vector through first classification processing, second classification processing, foreground detection, and feature classification processing on the feature vector. Finally, the classification result of the image to be classified is obtained based on the first classification result, the second classification result, the foreground detection result, and the feature classification result. The feature vector can be processed multiple times independently of each other, which can avoid mutual interference and mutual influence between the processes, thereby improving the accuracy and reliability of the overall classification of the feature vector. The classification result after independent processing is combined to obtain the final classification image of the image to be classified, which can weaken the excessive attention of image classification to invalid background in the image to be classified in a complex background, thereby strengthening the attention to specific foreground in the image to be classified and improving the image classification accuracy and reliability.

[0060] Please refer to Figure 2 , Figure 2 is a flowchart of another embodiment of the image classification method of the present application.

[0061] Step S21: An image to be classified is obtained.

[0062] The embodiment is the same as step S11 of the foregoing embodiment. Please refer to the foregoing description, which will not be repeated here.

[0063] Step S22: feature extraction is performed on the image to be classified by a feature extraction network of the image classification model to obtain a feature vector of the image to be classified.

[0064] Feature extraction is performed on the image to be classified by a feature extraction network of the image classification model to obtain a feature vector of the image to be classified. The image classification model at least includes a feature extraction network and a classification network which are cascaded with each other.

[0065] The feature extraction network is a feature extraction network obtained by pre-training based on a classification standard of the image to be classified. The feature extraction network can be constructed based on one or more of a BP (Back Propagation) neural network, a radial basis function (RBF-Radial Basis Function) neural network, a linear neural network, a convolutional neural network, a recurrent neural network, etc. The specific construction is not limited here.

[0066] In this step, the feature extraction network of the image classification model is used to perform feature extraction on the image to be classified. During the feature extraction process of the feature extraction network, after obtaining an intermediate feature map, attention weights can be inferred in turn by combining channel attention and spatial attention, and then multiplied with the original feature map of the image to be classified to adaptively adjust the features, thereby improving the accuracy and reliability of the feature vector.

[0067] In a specific application scenario, before feature extraction is performed on the image to be classified, the image to be classified can also be pre-processed to eliminate irrelevant information in the image as much as possible. The pre-processing at least includes one or more of cropping, denoising, color balance, grayscale processing, stretching, etc.

[0068] Step S23: foreground detection is performed on the feature vector by a foreground detection network of the image classification model to obtain a foreground detection result of the feature vector; first classification is performed on the feature vector by a first classification network of the image classification model to obtain a first classification result of the feature vector; second classification is performed on the feature vector by a second classification network of the image classification model to obtain a second classification result of the feature vector; and feature classification is performed on the feature vector by using a feature standard library to obtain a feature classification result of the feature vector.

[0069] After the feature vector of the image to be classified is obtained, the feature vector is subjected to foreground detection processing by a foreground detection network of the image classification model to obtain a foreground detection result of the feature vector; the feature vector is subjected to first classification processing by a first classification network of the image classification model to obtain a first classification result of the feature vector; the feature vector is subjected to second classification processing by a second classification network of the image classification model to obtain a second classification result of the feature vector; and the feature vector is subjected to feature classification processing by using a feature standard library to obtain a feature classification result of the feature vector.

[0070] The feature classification processing is performed separately and is not within the processing range of the image classification model. Thus, on the basis of image classification by using the image classification model, feature classification processing outside the image classification model is introduced, and finally the processing result of the feature classification processing outside the image classification model is used to assist in judging the classification result of the image classification model, thereby improving the robustness of the classification result of the image to be classified and further improving the accuracy and reliability of the classification result of the image to be classified.

[0071] The image classification model at least includes a feature extraction network, a first classification network, a second classification network and a foreground detection network, and the feature extraction network is mutually cascaded with the first classification network, the second classification network and the foreground detection network.

[0072] In one specific embodiment, the feature vector can be subjected to foreground detection first to obtain a foreground detection result of the feature vector, and then the foreground detection result of the feature vector is subjected to first classification processing, second classification processing and feature classification processing respectively to obtain a first classification result of the feature vector, a second classification result of the feature vector and a feature classification result of the feature vector.

[0073] Please refer to Figure 3 , Figure 3 is Figure 2 A structural schematic diagram of one embodiment of the image classification model in the embodiment.

[0074] The image classification model 10 of the embodiment includes a feature extraction network 11, a foreground detection network 12, a first classification network 13 and a second classification network 14. The feature extraction network 11 and the foreground detection network 12 are mutually cascaded, and the foreground detection network 12 is mutually cascaded with the first classification network 13 and the second classification network 14 respectively.

[0075] Specifically, the feature extraction network 11 is used to extract features of the image to be classified to obtain a feature vector of the image to be classified, the foreground detection network 12 is used to detect foreground of the feature vector to obtain a foreground detection result of the feature vector, the same foreground detection result is respectively processed by the first classification network 13 and the second classification network 14 to obtain a second classification result and a second classification result.

[0076] Meanwhile, the feature standard library is used to perform feature classification processing on the feature vector to obtain a feature classification result of the feature vector. The feature classification processing on the feature vector by the feature standard library is performed independently and is not within the processing range of the image classification model 10.

[0077] The foreground detection is performed in advance in this embodiment, so that the subsequent three classification processes pay more attention to the specific foreground in the image to be classified, thereby improving the classification accuracy and reliability, and reducing the situation that the specific foreground cannot be focused in the image to be classified in a complex background, and more attention is paid to the background information.

[0078] In another specific embodiment, the feature vector can be simultaneously subjected to foreground detection, first classification processing, second classification processing, and feature classification processing to obtain a first classification result of the feature vector, a second classification result of the feature vector, a foreground detection result of the feature vector, and a feature classification result of the feature vector.

[0079] Please refer to Figure 4 , Figure 4 is Figure 2 a structural schematic diagram of another embodiment of the image classification model.

[0080] The image classification model 20 of this embodiment includes a feature extraction network 21, a foreground detection network 22, a first classification network 23, and a second classification network 24. The feature extraction network 21 is respectively cascaded with the foreground detection network 22, the first classification network 23, and the second classification network 24.

[0081] The feature extraction network 11 is used to extract features of the image to be classified to obtain a feature vector of the image to be classified, the foreground detection network 22, the first classification network 23, and the second classification network 24 are used to process the feature vector to obtain a first classification result, a second classification result, and a foreground detection result. Meanwhile, the feature standard library is used to perform feature classification processing on the feature vector to obtain a feature classification result of the feature vector. The feature classification processing on the feature vector by the feature standard library is performed independently and is not within the processing range of the image classification model 10.

[0082] The four processing manners in the embodiment are independent of each other and are performed in parallel, so that the accuracy and reliability of the overall classification of the feature vectors are improved.

[0083] In the step, the feature vectors are classified by using the feature standard library to obtain the feature classification result of the feature vectors, and the step specifically includes: performing cosine similarity processing on the feature vectors and each standard feature vector in the feature standard library to obtain the similarity between the feature vectors and each standard feature vector, and determining the classification type corresponding to the standard feature vector with the maximum similarity as the feature classification result of the feature vector.

[0084] The feature classification processing is different from the second classification processing in that the feature classification processing is performed by using the feature vectors in the feature standard library to perform cosine similarity comparison and classification determination, while the second classification processing is performed by using the label parameters learned by the image classification model to perform classification calculation and determination.

[0085] The classification types of the second classification processing of the embodiment include the classification types of the first classification processing and the corresponding subtypes, that is, the first classification processing can be coarse classification of the image to be classified, and the second classification processing can be fine classification of the image to be classified. The classification types of the second classification processing are the same as the classification types of the feature classification, the first classification processing, the second classification processing and the feature classification processing are performed on the same feature vector, the classification types of the second classification processing include the classification types of the first classification processing and the corresponding subtypes, and the classification types of the second classification processing are the same as the classification types of the feature classification, that is, the processing processes of the three classification processes are independent of each other, so that the mutual interference and mutual influence between the processes are avoided, and the accuracy of the first classification result, the second classification result and the feature classification result is improved. However, the classification standards of the three classification processes are associated with each other, so that the first classification result, the second classification result and the feature classification result can be mutually referenced and assisted to determine the classification result of the image to be classified, thereby further improving the accuracy of the classification result of the image to be classified.

[0086] In a specific application scenario, when the image classification is image pornography classification, the classification types of the first classification result include normal type, sexy type and pornography type; the classification types of the second classification result and the feature classification result include normal type, subtypes of sexy type and subtypes of pornography type; and the foreground detection result includes the position information of the region where the human body is located in each image to be classified.

[0087] Step S24: obtaining the classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result.

[0088] The first classification result, the second classification result, the foreground detection result and the feature classification result are integrated to obtain a classification result of the image to be classified.

[0089] In a specific application scenario, in response to the classification type of the second classification result being the same as the classification type of the feature classification result and the second classification result being the first classification result or a sub-type corresponding to the first classification result, the second classification result and the foreground detection result are determined as the classification result of the image to be classified. Since the classification type of the second classification result is more detailed, the second classification result is output in the case that the three classification results match each other, thereby improving the information content of the classification result of the image to be classified. In other application scenarios, other classification results can be selected for output based on actual needs, which are not limited herein.

[0090] In a specific application scenario, the first classification result includes the confidence of each classification type of the first classification processing, the second classification result includes the confidence of each classification type of the second classification processing, and the feature classification result includes the confidence of each classification type of the feature classification processing. The confidence of each classification type of the second classification processing is multiplied with the confidence of each classification type of the corresponding first classification processing and the confidence of each classification type of the corresponding feature classification processing respectively to obtain a plurality of product values, and the classification type of the second classification processing corresponding to the maximum product value and the foreground detection result are logically combined to obtain the classification result of the image to be classified.

[0091] For example, when the classification type of the first classification result is normal type, pornography type and sex type, the classification types of the second classification result and the feature classification result can be normal, first pornography sub-type, second pornography sub-type, third pornography sub-type, first sex sub-type, second sex sub-type and third sex sub-type respectively. The specific objects of the pornography sub-type and the sex sub-type can be determined based on actual situations.

[0092] When the classification type and the confidence of the first classification result are obtained as follows: normal: 0.2; sex: 0.5; and pornography: 0.3.

[0093] The second classification result includes normal: 0.1; first pornography sub-type: 0.1; second pornography sub-type: 0.2; third pornography sub-type: 0.1; first sex sub-type: 0.3; second sex sub-type: 0.1; and third sex sub-type: 0.1.

[0094] The feature classification result includes normal: 0.2; first pornography sub-type: 0.1; second pornography sub-type: 0.2; third pornography sub-type: 0.1; first sex sub-type: 0.3; second sex sub-type: 0.1; and third sex sub-type: 0.1.

[0095] The confidence of each classification type of the second classification processing is multiplied with the confidence of each classification type of the corresponding first classification processing and the confidence of each classification type of the corresponding feature classification processing respectively to obtain a plurality of product values. For example, the product processing of the normal type of the second classification processing is the confidence of the normal type of the second classification result * the confidence of the normal type of the first classification result * the confidence of the normal type of the feature classification result, which is "0.1*0.2*0.2=0.004", the product processing of the first sex sub-type of the second classification processing is: the confidence of the normal type of the second classification result * the confidence of the sex type of the first classification result * the confidence of the first sex sub-type of the feature classification result, which is "0.3*0.5*0.3=0.045", and the calculation method of the product values of other confidences is similar to the above, which is not limited here.

[0096] The classification type of the second classification processing corresponding to the product value with the largest value and the foreground detection result are logically combined to obtain the classification result of the image to be classified.

[0097] In a specific application scenario, when the image classification is image pornography classification, in response to the foreground detection size of the foreground detection result being not greater than a preset size, the classification result of the image to be classified is determined as the normal type. When the image classification detects that the size of the foreground detection is insufficient to cause adverse effects, the classification result of the image to be classified can be directly determined as the normal type.

[0098] In a specific application scenario, in order to compare the product values with each other, the product values can be normalized and then compared with each other to select the product value with the largest value to obtain the classification result of the image to be classified.

[0099] The classification result of the image to be classified is determined by the above comprehensive method, which can weaken the excessive attention of the image classification to the invalid background in the complex background of the image to be classified to some extent, thereby strengthening the attention to the specific foreground in the image to be classified and improving the image classification accuracy and reliability.

[0100] In the image classification of the embodiment, the image classification model can be trained. Please refer to Figure 5 , Figure 5 is Figure 2 The flowchart of the training method of the image classification model in the embodiment.

[0101] Step S31: Obtain a plurality of training images.

[0102] Obtaining a plurality of training images, wherein the plurality of training images can include all classification types of the first classification, all classification types of the second classification (all types of feature classification), so as to ensure that the image classification model can be trained for classification of each classification type.

[0103] Moreover, each training image is labeled with a standard first classification result, a standard second classification result, and a standard foreground result. In one specific application scenario, the standard first classification result, the standard second classification result, and the standard foreground result of each training image can be labeled by a person. In another specific application scenario, the standard first classification result, the standard second classification result, and the standard foreground result of each training image can be labeled by using the first classification model, the second classification model, and the foreground detection model that have been trained.

[0104] In one specific application scenario, when the image classification is a pornography classification, the plurality of training images can be detected by using a human body detection to obtain a human body foreground region, so as to obtain the standard foreground result.

[0105] In one specific application scenario, the standard feature vector of each training image can also be obtained. The standard feature vector can be obtained based on each training image by using a feature extraction algorithm or a trained feature extraction network. After obtaining the standard feature vector of each training image, the classification type corresponding to the standard feature vector is determined based on the plurality of standard second classification results of the training image, so as to facilitate the construction of the standard feature library in the foregoing embodiments by using the standard feature vector of each training image.

[0106] Step S32: The feature extraction network of the image classification model under training is used to respectively extract features of each training image, so as to obtain a training feature vector of each training image.

[0107] The feature extraction network of the image classification model under training is used to respectively extract features of each training image, so as to obtain a training feature vector of each training image.

[0108] The specific feature extraction process is the same as the feature extraction process in step S22 of the foregoing embodiments. Please refer to the foregoing description, which will not be repeated here.

[0109] Step S33: The classification network of the image classification model under training is used to respectively perform first classification processing, second classification processing, and foreground detection on the training feature vector, so as to obtain a first classification result of the training feature vector, a second classification result of the training feature vector, and a foreground detection result of the training feature vector.

[0110] The specific first classification, the specific second classification, the specific feature classification, and the specific process of foreground detection are the same as those in step S23 of the foregoing embodiment, please refer to the foregoing description, and details are not described herein.

[0111] Step S34: The image classification model in training is trained by using the first classification result, the second classification result, and the foreground detection result of each training feature vector and the standard classification result corresponding to each training image, until the image classification model is obtained.

[0112] After obtaining the first classification result, the second classification result, and the foreground detection result of each training feature vector, the image classification model in training is trained by using the first classification result, the second classification result, and the foreground detection result of each training feature vector and the standard classification result of the corresponding training image, until the image classification model is obtained.

[0113] In a specific application scenario, the image classification model in training can be trained by using the first classification result, the second classification result, and the foreground detection result of each training image, the standard classification result corresponding to each training image, and the standard feature vector based on the overall loss function, until the image classification model is obtained.

[0114] In a specific application scenario, the image classification model in training can be trained by using the first classification result of each training feature vector and the corresponding standard first classification result in the standard classification result based on the first loss function, and trained by using the second classification result of each training feature vector and the corresponding multiple standard second classification results in the standard classification result based on the second loss function, and trained by using the foreground detection result of each training feature vector and the corresponding standard foreground detection result in the standard classification result based on the third loss function, and trained by using each training feature vector and the corresponding standard feature vector based on the fourth loss function, wherein the overall loss function is determined by the sum of the first loss function, the second loss function, the third loss function, and the fourth loss function after weighting processing.

[0115] Through the weighting processing, each loss function can be in the same order of magnitude, so that the optimization accuracy of the multiple branch networks of the classification network and the feature extraction network is similar, and the training balance of the image classification model is achieved.

[0116] In a specific application scenario, the first loss function and the second loss function can adopt the softmax function, the third loss function can adopt the ciou loss function, and the fourth loss function can adopt the am function. In other application scenarios, each loss function can also adopt other types of loss functions based on the characteristics of the training object, which is not limited herein.

[0117] The first classification, the second classification, the foreground detection and the feature extraction processing are respectively optimized and trained by the four loss functions, so that the image classification model in the training can be independently trained and converged based on the four image processing, thereby improving the processing accuracy of each processing of the image classification model, and further improving the classification accuracy of the entire image classification model.

[0118] The training can enable the image classification model to learn how to perform the first classification, the second classification, the foreground detection and the feature extraction during the training process, thereby enabling the trained image classification model to improve the accuracy of the first classification, the second classification, the foreground detection and the feature extraction, and train the training image by the four different processing, so that the trained image classification model can focus more on the foreground area of the image to be classified, reduce the excessive attention to the invalid background area, improve the pertinence of the model in the first classification and the second classification, and further improve the classification accuracy and reliability of the image classification model. Further, the foreground training can enable the image classification model to ignore the background information as much as possible during the learning process and focus on the foreground, thereby enabling the image classification model to be more focused on the foreground in different scene classifications and have stronger adaptability. Further, the image classification process of the image classification model adopts multiple classification processing methods for classification, which can further improve the generalization ability of the image classification model, so that it can be applied to application scenarios that can be adapted to multiple classification processing methods, and improve the application range and robustness of the image classification method of the embodiment.

[0119] Through the above steps, the image classification model of the embodiment extracts features of the to-be-classified image through the feature extraction network of the image classification model, obtains a feature vector of the to-be-classified image, and then respectively performs first classification processing, multiple second classification processing, and foreground detection on the feature vector through the classification network of the image classification model, to obtain a first classification result, a second classification result, and a foreground detection result of the feature vector. Finally, the classification result of the to-be-classified image is obtained based on the first classification result, the second classification result, and the foreground detection result, so that the entire image classification can avoid the situation that a specific foreground cannot be focused on in a to-be-classified image with a complex background, resulting in excessive attention to background information. Through multiple classification processing, the attention to the specific foreground in the to-be-classified image is improved, thereby improving the image classification accuracy and reliability. The embodiment can respectively process the feature vector multiple times independently, avoid the situation that the processing interferes with each other and affects each other, thereby improving the accuracy and reliability of the overall classification of the feature vector. Moreover, the classification results after independent processing are combined to obtain the final classification image of the to-be-classified image, which can weaken the excessive attention to invalid background in the image classification of the to-be-classified image with a complex background to some extent, thereby strengthening the attention to the specific foreground in the to-be-classified image and improving the image classification accuracy and reliability. Moreover, the training process of the image classification model of the embodiment can also enable the image classification model in the training to be independently trained and converged based on different image processing, thereby improving the processing accuracy of each processing of the image classification model, and further improving the classification accuracy of the entire image classification model. Moreover, the image classification model of the embodiment adopts multiple classification processing methods for classification in the image classification process of the image classification model, which can further improve the generalization ability of the image classification model, so that it can be applied to application scenarios that can be adapted to multiple classification processing methods, thereby improving the application range and robustness of the image classification method of the embodiment.

[0120] Please refer to Figure 6 , Figure 6 is a schematic diagram of the framework of an embodiment of the image classification device of the present application. The image classification device 60 comprises an acquisition module 61, a feature extraction module 62, a classification module 63, and a determination module 64. The acquisition module 61 is configured to acquire a to-be-classified image. The feature extraction module 62 is configured to perform feature extraction on the to-be-classified image to obtain a feature vector of the to-be-classified image. The classification module 63 is configured to respectively perform first classification processing, second classification processing, foreground detection, and feature classification processing on the feature vector to obtain a first classification result of the feature vector, a second classification result of the feature vector, a foreground detection result of the feature vector, and a feature classification result of the feature vector. The determination module 64 is configured to obtain a classification result of the to-be-classified image based on the first classification result, the second classification result, the foreground detection result, and the feature classification result.

[0121] The classification module 63 is further configured to perform foreground detection on the feature vector to obtain a foreground detection result of the feature vector, and perform first classification processing, second classification processing, and feature classification processing on the foreground detection result of the feature vector respectively to obtain a first classification result of the feature vector, a second classification result of the feature vector, and a feature classification result of the feature vector.

[0122] The determination module 64 is further configured to, in response to the classification type of the second classification result being the same as the classification type of the feature classification result and the second classification result being the first classification result or a sub-type corresponding to the first classification result, determine the second classification result and the foreground detection result as the classification result of the image to be classified.

[0123] The determination module 64 is further configured to multiply the confidence of each classification type of the second classification processing with the confidence of each classification type of the corresponding first classification processing and the confidence of each classification type of the corresponding feature classification processing respectively to obtain a plurality of product values, and perform logical combination on the classification type of the second classification processing and the foreground detection result corresponding to the product value with the largest value to obtain the classification result of the image to be classified.

[0124] The feature extraction module 62 is further configured to perform feature extraction on the image to be classified through a feature extraction network of the image classification model to obtain a feature vector of the image to be classified, and the steps of performing first classification processing, second classification processing, foreground detection, and feature classification processing on the feature vector respectively to obtain a first classification result of the feature vector, a second classification result of the feature vector, a foreground detection result of the feature vector, and a feature classification result of the feature vector include: performing foreground detection processing on the feature vector through a foreground detection network of the image classification model to obtain the foreground detection result of the feature vector; performing first classification processing on the feature vector through a first classification network of the image classification model to obtain the first classification result of the feature vector; performing second classification processing on the feature vector through a second classification network of the image classification model to obtain the second classification result of the feature vector; and performing feature classification processing on the feature vector by using a feature standard library to obtain the feature classification result of the feature vector; wherein the image classification model at least includes the feature extraction network, the first classification network, the second classification network, and the foreground detection network, and the feature extraction network is cascaded with the first classification network, the second classification network, and the foreground detection network respectively.

[0125] The classification module 63 is further configured to perform cosine similarity processing on the feature vector and each standard feature vector in the feature standard library respectively to obtain the similarity between the feature vector and each standard feature vector, and determine the classification type corresponding to the standard feature vector with the largest similarity as the feature classification result of the feature vector.

[0126] The acquisition module 61 is further configured to acquire a plurality of training images.

[0127] The feature extraction network of the image classification model in training respectively extracts features of each training image to obtain a training feature vector of each training image; the classification network of the image classification model in training respectively performs first classification processing, second classification processing, and foreground detection on the training feature vector to obtain a first classification result of the training feature vector, a second classification result of the training feature vector, and a foreground detection result of the training feature vector; the image classification model in training is trained by using the first classification result, the second classification result, and the foreground detection result of each training image and the standard classification result corresponding to each training image until the image classification model is obtained.

[0128] The acquisition module 61 is further configured to acquire a plurality of training images; wherein the standard first classification result, the standard second classification result, and the standard foreground result are labeled on each training image; the standard feature vector of the training image is obtained by performing feature extraction on the training image, and the classification type corresponding to the standard feature vector is determined based on the plurality of standard second classification results of the training image; the standard classification result of each training image is obtained by using the standard first classification result, the plurality of standard second classification results, the standard foreground result, and the standard feature vector of each training image.

[0129] The acquisition module 61 is further configured to train the image classification model in training by using the first classification result, the second classification result, the foreground detection result, and the feature classification result of each training image and the standard classification result corresponding to each training image based on the overall loss function until the image classification model is obtained.

[0130] The acquisition module 61 is further configured to train the image classification model in training by using the first classification result of each training feature vector and the corresponding standard first classification result in the standard classification result based on the first loss function; train the image classification model in training by using the second classification result of each training feature vector and the corresponding standard second classification result in the standard classification result based on the second loss function; train the image classification model in training by using the foreground detection result of each training feature vector and the corresponding standard foreground detection result in the standard classification result based on the third loss function; and train the image classification model in training by using each training feature vector and the corresponding standard feature vector based on the fourth loss function; wherein the overall loss function is determined by the sum of the first loss function, the second loss function, the third loss function, and the fourth loss function after weighting processing.

[0131] The classification type of the first classification result includes the normal type, the sexy type, and the pornography type; the classification type of the second classification result and the feature classification result includes the normal type, the sub-type of the sexy type, and the sub-type of the pornography type; the foreground detection result includes the position information of the region where the human body is located on each image to be classified.

[0132] The determining module 64 is further configured to determine the classification result of the image to be classified as the normal type in response to the foreground detection size of the foreground detection result being not greater than a preset size.

[0133] The above scheme can improve the image classification accuracy.

[0134] Please refer to Figure 7 , Figure 7 is a schematic diagram of the framework of an embodiment of the electronic device. The electronic device 70 includes a memory 71 and a processor 72 coupled to each other. The processor 72 is configured to execute program instructions stored in the memory 71 to implement the steps of the image classification method of any of the above embodiments. In a specific implementation scenario, the electronic device 70 can include but is not limited to a microcomputer, a server, and in addition, the electronic device 70 can also include a notebook computer, a tablet computer, and other mobile devices, which are not limited herein.

[0135] Specifically, the processor 72 is configured to control itself and the memory 71 to implement the steps of any of the above image classification method embodiments. The processor 72 can also be referred to as a CPU (Central Processing Unit). The processor 72 can be an integrated circuit chip with a processing capability of signals. The processor 72 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 72 can be implemented by an integrated circuit chip together.

[0136] The above scheme can improve the image classification accuracy.

[0137] Please refer to Figure 8 , Figure 8 is a schematic diagram of the framework of an embodiment of the computer readable storage medium. The computer readable storage medium 80 stores program instructions 801 capable of being executed by a processor, and the program instructions 801 are used to implement the steps of the image classification method of any of the above embodiments.

[0138] The above scheme can improve the image classification accuracy.

[0139] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely an example, and the division of the units or components can be different, for example, some units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the display or discussion of the coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., they can be located in one place or distributed on network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0141] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0142] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

Claims

1. An image classification method, characterized by, The image classification method is applied to image pornography classification; the image classification method comprises: obtaining a to-be-classified image; performing feature extraction on the to-be-classified image to obtain a feature vector of the to-be-classified image; performing first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector to obtain a first classification result, a second classification result, a foreground detection result and a feature classification result of the feature vector; wherein the feature classification processing comprises performing feature classification processing on the feature vector by using a feature standard library to obtain the feature classification result; the classification types of the first classification result include a normal type, a sexy type and a pornography type; the classification types of the second classification result and the feature classification result include the normal type, subtypes of the sexy type and subtypes of the pornography type; the foreground detection result includes position information of a region where a human body is located on each of the to-be-classified images; obtaining a classification result of the to-be-classified image based on the first classification result, the second classification result, the foreground detection result and the feature classification result; wherein the confidence of each classification type of the second classification processing is multiplied by the confidence of each classification type of the corresponding first classification processing and the confidence of each classification type of the corresponding feature classification processing, respectively, to obtain a plurality of product values; the classification type of the second classification processing corresponding to the largest product value and the foreground detection result are logically combined to obtain the classification result of the to-be-classified image; in response to the foreground detection size of the foreground detection result being not greater than a preset size, the classification result of the to-be-classified image is determined as the normal type; wherein the classification types of the second classification processing include the classification types of the first classification processing and their corresponding subtypes, and the classification types of the second classification processing are the same as the classification types of the feature classification.

2. The image classification method of claim 1, wherein, The step of obtaining a classification result of the to-be-classified image based on the first classification result, the second classification result, the foreground detection result and the feature classification result comprises: in response to the classification type of the second classification result being the same as the classification type of the feature classification result, and the second classification result being the first classification result or a subtype corresponding to the first classification result, the logically combined second classification result and the foreground detection result are determined as the classification result of the to-be-classified image.

3. The image classification method of claim 1 or 2, characterized in that, The step of performing first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector to obtain a first classification result, a second classification result, a foreground detection result and a feature classification result of the feature vector comprises: performing foreground detection on the feature vector to obtain a foreground detection result of the feature vector; performing first classification processing, second classification processing and feature classification processing on the foreground detection result of the feature vector, respectively, to obtain a first classification result of the feature vector, a second classification result of the feature vector and a feature classification result of the feature vector.

4. The image classification method of claim 1 or 2, characterized in that, The step of performing feature extraction on the image to be classified to obtain a feature vector of the image to be classified comprises: performing feature extraction on the image to be classified by a feature extraction network of the image classification model to obtain a feature vector of the image to be classified; The step of performing first classification processing, second classification processing, foreground detection, and feature classification processing on the feature vector respectively to obtain a first classification result of the feature vector, a second classification result of the feature vector, a foreground detection result of the feature vector, and a feature classification result of the feature vector comprises: performing foreground detection processing on the feature vector by a foreground detection network of the image classification model to obtain a foreground detection result of the feature vector; and performing first classification processing on the feature vector by a first classification network of the image classification model to obtain a first classification result of the feature vector; and performing second classification processing on the feature vector by a second classification network of the image classification model to obtain a second classification result of the feature vector; and performing feature classification processing on the feature vector by using a feature standard library to obtain a feature classification result of the feature vector; The image classification model at least comprises a feature extraction network, a first classification network, a second classification network, and a foreground detection network, and the feature extraction network is respectively cascaded with the first classification network, the second classification network, and the foreground detection network.

5. The image classification method of claim 4, wherein, The step of performing feature classification processing on the feature vector by using a feature standard library to obtain a feature classification result of the feature vector comprises: performing cosine similarity processing on the feature vector and each standard feature vector in the feature standard library respectively to obtain the similarity between the feature vector and each standard feature vector; determining the classification type corresponding to the standard feature vector with the largest similarity as the feature classification result of the feature vector.

6. The image classification method of claim 1, wherein, The step of obtaining an image to be classified comprises: obtaining a plurality of training images; performing feature extraction on each of the training images by a feature extraction network of the image classification model in training to obtain a training feature vector of each of the training images; performing first classification processing, second classification processing, and foreground detection on the training feature vector by a classification network of the image classification model in training to obtain a first classification result of the training feature vector, a second classification result of the training feature vector, and a foreground detection result of the training feature vector; training the image classification model in training by using the first classification result, the second classification result, and the foreground detection result of each of the training images and the standard classification result corresponding to each of the training images until the image classification model is obtained.

7. The image classification method of claim 6, wherein, The step of obtaining a plurality of training images further comprises: obtaining a plurality of training images; wherein each training image is labeled with a standard first classification result, a standard second classification result, and a standard foreground result; performing feature extraction on the training image to obtain a standard feature vector of the training image, and determining the classification type corresponding to the standard feature vector based on a plurality of standard second classification results of the training image; The standard classification results of the training images are obtained based on the standard first classification results, the standard second classification results, the standard foreground results and the standard feature vectors of the training images.

8. The image classification method of claim 6, wherein, The step of training the image classification model in training based on the first classification results, the second classification results, the foreground detection results and the feature classification results of the training images and the standard classification results corresponding to the training images by using the overall loss function until the image classification model is obtained comprises: The image classification model in training is trained based on the first classification results, the second classification results, the foreground detection results and the feature classification results of the training images and the standard classification results corresponding to the training images by using the overall loss function until the image classification model is obtained.

9. The image classification method of claim 8, wherein, The step of training the image classification model in training based on the first classification results, the second classification results, the foreground detection results and the feature classification results of the training images and the standard classification results corresponding to the training images by using the overall loss function until the image classification model is obtained comprises: The image classification model in training is trained based on the first classification results of the training feature vectors and the corresponding standard first classification results in the standard classification results by using the first loss function; and The image classification model in training is trained based on the second classification results of the training feature vectors and the corresponding standard second classification results in the standard classification results by using the second loss function; and The image classification model in training is trained based on the foreground detection results of the training feature vectors and the corresponding standard foreground detection results in the standard classification results by using the third loss function; and The image classification model in training is trained based on the training feature vectors and the corresponding standard feature vectors by using the fourth loss function. The sum of the first loss function, the second loss function, the third loss function and the fourth loss function after weighting processing determines the overall loss function.

10. An image classification apparatus characterized by comprising: The image classification device is applied to image pornography classification, comprising: An acquisition module is configured to acquire a to-be-classified image. A feature extraction module is configured to perform feature extraction on the to-be-classified image to obtain a feature vector of the to-be-classified image. A classification module is configured to perform first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector respectively to obtain a first classification result of the feature vector, a second classification result of the feature vector, a foreground detection result of the feature vector and a feature classification result of the feature vector. The feature classification processing comprises performing feature classification processing on the feature vector by using a feature standard library to obtain the feature classification result. The classification types of the first classification result include a normal type, a sexy type and a pornography type. The classification types of the second classification result and the feature classification result include the normal type, subtypes of the sexy type and subtypes of the pornography type. The foreground detection result includes position information of a region where a human body is located on each of the to-be-classified images. The determining module is configured to obtain a classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result, and the feature classification result; wherein, the confidence of each classification type of the second classification processing is multiplied with the confidence of each classification type of the first classification processing and the confidence of each classification type of the feature classification processing respectively to obtain a plurality of product values; the classification type of the second classification processing corresponding to the product value with the largest value and the foreground detection result are logically combined to obtain the classification result of the image to be classified; and in response to the foreground detection size of the foreground detection result being not greater than a preset size, the classification result of the image to be classified is determined as a normal type. The classification types of the second classification processing include the classification types of the first classification processing and the corresponding subtypes, and the classification types of the second classification processing are the same as the classification types of the feature classification.

11. An electronic device, comprising: The apparatus comprises a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory to implement the image classification method according to any one of claims 1 to 9.

12. A computer readable storage medium having stored thereon program instructions, wherein, The program instructions are executed by the processor to implement the image classification method according to any one of claims 1 to 9.

Images