Image classification method and device, communication equipment and storage medium

Abstract

The present disclosure is about an image classification method and apparatus, communication equipment and storage medium. The image classification method comprises: extracting semantic features of a target image to be classified using a feature extraction part of a classification model; processing the semantic features using a classification part of the classification model to obtain a first classification result based on M classification, wherein M is a positive integer equal to or greater than 3; based on the first classification result, outputting at least degree classification information indicating that the target image is a vulgar image, wherein the vulgar image includes: pornographic images, horror images and / or violent images; the method has the characteristics of realizing the discrimination of whether the target image is a vulgar image and realizing the degree classification of the image, having the characteristics of high classification accuracy, and can meet the detailed classification requirements in the detailed classification scenario.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to an image classification method and apparatus, a communication device, and a storage medium. Background Art

[0002] With the explosive growth of user-generated content (UGC), information flow products occupy more and more user time, and pictures are the most direct carrier of information. It is a very necessary task to identify and block vulgar content, control risky content, and purify picture carriers. However, with the maturity of image review technology, more and more content creators are intensifying the creation of pornographic content. The boundaries between soft pornography and pornography, normal and soft pornography are becoming increasingly blurred, which also brings great difficulties and challenges to the review of pornographic images. Summary of the invention

[0003] The embodiments of the present application disclose an image classification method and apparatus, a communication device, and a storage medium.

[0004] The present application provides an image classification method, including:

[0005] The feature extraction part of the classification model is used to extract the semantic features of the target image to be classified;

[0006] Processing the semantic feature using the classification part of the classification model to obtain a first classification result based on M classification, wherein M is a positive integer equal to or greater than 3;

[0007] Based on the first classification result, output is the degree classification information indicating at least that the target image is a vulgar image, wherein the vulgar image includes: pornographic images, horror images and / or violent images.

[0008] Based on the above solution, the method further includes:

[0009] Mapping the first classification result to a second classification result based on N classifications, wherein N is a positive integer less than M;

[0010] The second classification result based on N classification is: the classification result based on the parent class;

[0011] The first classification result based on M classification is: a classification result based on subclasses, wherein at least one of the parent classes includes: a plurality of the subclasses.

[0012] Based on the above solution, the outputting, based on the first classification result, classification information indicating at least that the target image is a vulgar image includes:

[0013] Outputting a first classification result of the target image;

[0014] and / or,

[0015] Output a second classification result of the target image.

[0016] Based on the above scheme, the feature extraction part of the classification model extracts the semantic features of the target image to be classified, including at least one of the following:

[0017] For the pornographic image, using the feature extraction part of the classification model, extract one or more of the clothing features, behavior features, exposure degree features of specific parts of the character elements, and overall exposure degree features of the character elements in the target image;

[0018] For the violent image, using the feature extraction part of the classification model, extract one or more of whether the target image contains violent elements, the number of violent elements, the proportion of the image area occupied by violent elements, and the degree of violence of the violent elements;

[0019] For the horror image, the feature extraction part of the classification model is used to extract one or more of whether the target image contains horror elements, the number of horror elements, the proportion of the image area occupied by horror elements, and the vulgarity of the horror elements.

[0020] Based on the above solution, before recognizing the image, the method further includes:

[0021] Obtaining a first sample image and an annotated label, wherein the annotated label is a subclass label of the M classification;

[0022] Using the first sample images and the annotated labels, training the classification model;

[0023] When the training loss of the classification model converges, stop training;

[0024] A mapping layer for mapping the first classification result to the second classification result is added to the output layer of the classification model.

[0025] Based on the above solution, the method further includes:

[0026] Performing morphological transformation on the first sample image to obtain a second sample image, wherein the second sample image has the same annotation label as the first sample image;

[0027] The step of training the classification model by using the first sample image and the annotated label includes:

[0028] The classification model is trained using the first sample image and the annotated label of the first sample image, and the second image sample and the annotated label of the second sample image.

[0029] Based on the above solution, performing morphological transformation on the first sample image to obtain the second sample image includes at least one of the following:

[0030] Performing a linear transformation on the first sample image to obtain the second sample image, wherein the linear transformation includes: image rotation and / or image scaling;

[0031] Performing an image content change on the first sample image to obtain the second sample image, wherein the image content change includes: local mosaic processing, image content replacement in a local area, and / or local blurring processing;

[0032] Performing image parameter transformation on the first sample image to obtain the second sample image, wherein the image parameters include: contrast and / or brightness.

[0033] Based on the above scheme, the classification model is a SEResNeXt model.

[0034] A second aspect of an embodiment of the present application provides an image classification device, comprising:

[0035] An extraction module, used for extracting semantic features of a target image to be classified by using a feature extraction part of a classification model;

[0036] A classification module, used for processing the semantic feature by using the classification part of the classification model to obtain a first classification result based on M classification, wherein M is a positive integer equal to or greater than 3;

[0037] The output module is used to output classification information indicating at least the degree to which the target image is a vulgar image based on the first classification result, wherein the vulgar image includes: pornographic images, horror images and / or violent images.

[0038] Based on the above solution, the device also includes:

[0039] A mapping module, used to map the first classification result to a second classification result based on N classifications, wherein N is a positive integer less than M;

[0040] The second classification result based on N classification is: the classification result based on the parent class;

[0041] The first classification result based on M classification is: a classification result based on subclasses, wherein at least one of the parent classes includes: a plurality of the subclasses.

[0042] Based on the above solution, the output module is used to output the first classification result of the target image; and / or output the second classification result of the target image.

[0043] Based on the above solution, the extraction module is used to perform at least one of the following:

[0044] For the pornographic image, using the feature extraction part of the classification model, extract one or more of the clothing features, behavior features, exposure degree features of specific parts of the character elements, and overall exposure degree features of the character elements in the target image;

[0045] For the violent image, using the feature extraction part of the classification model, extract one or more of whether the target image contains violent elements, the number of violent elements, the proportion of the image area occupied by violent elements, and the degree of violence of the violent elements;

[0046] For the horror image, the feature extraction part of the classification model is used to extract one or more of whether the target image contains horror elements, the number of horror elements, the proportion of the image area occupied by horror elements, and the vulgarity of the horror elements.

[0047] Based on the above solution, the device also includes:

[0048] An acquisition module, used for acquiring a first sample image and a label before recognizing the image, wherein the label is a subclass label of the M classification;

[0049] A training module, used to train the classification model using the first sample image and the annotated label; when the training loss of the classification model converges, stop the training;

[0050] An adding module is used to add a mapping layer to the output layer of the classification model, which maps the first classification result to the second classification result.

[0051] Based on the above solution, the device also includes:

[0052] a transformation module, configured to perform morphological transformation on the first sample image to obtain a second sample image, wherein the second sample image has the same annotation label as the first sample image;

[0053] The training module is used to train the classification model by using the first sample image and the annotated label of the first sample image, and the second image sample and the annotated label of the second sample image.

[0054] Based on the above solution, the transformation module is used to perform at least one of the following:

[0055] Performing a linear transformation on the first sample image to obtain the second sample image, wherein the linear transformation includes: image rotation and / or image scaling;

[0056] Performing an image content change on the first sample image to obtain the second sample image, wherein the image content change includes: local mosaic processing, image content replacement in a local area, and / or local blurring processing;

[0057] Performing image parameter transformation on the first sample image to obtain the second sample image, wherein the image parameters include: contrast and / or brightness.

[0058] Based on the above scheme, the classification model is a SEResNeXt model.

[0059] A third aspect of an embodiment of the present application provides an electronic device, including:

[0060] processor;

[0061] a memory for storing processor-executable instructions;

[0062] Among them, the processor is configured to implement the method provided by any technical solution of the first aspect mentioned above by executing the executable instructions.

[0063] The fourth aspect of the embodiments of the present application provides a non-temporary computer-readable storage medium, which stores executable instructions; after the executable instructions are executed by the processor, the method provided by any technical solution of the first aspect can be implemented.

[0064] The technical solution provided by the embodiment of the present application: when using a classification model to classify whether a target image is a vulgar image, not only will it be determined whether the target image is a vulgar image, but M classifications will be performed on the degree to which the target image is a vulgar image; M is a classification equal to or greater than 3. In this way, while realizing the determination of whether the target image is a vulgar image, a detailed classification of the degree of the image is realized, which has the characteristics of high classification accuracy and can meet the detailed classification requirements in detailed classification scenarios.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0067] Figure 1The figure is a flowchart of an image classification method according to an exemplary embodiment.

[0068] Figure 2 The figure is a flowchart of an image classification method according to an exemplary embodiment.

[0069] Figure 3 The figure is a flowchart of an image classification method according to an exemplary embodiment.

[0070] Figure 4 is a schematic diagram of a normal image according to an exemplary embodiment.

[0071] Figure 5 The figure is a block diagram of an image classification device according to an exemplary embodiment.

[0072] Figure 6 It is a schematic diagram showing the difference between a ResNet model and a SEResNeXt model according to an exemplary embodiment.

[0073] Figure 7 is a schematic diagram of an SE module according to an exemplary embodiment.

[0074] Figure 8 It is a schematic block diagram of a mobile terminal according to an exemplary embodiment.

[0075] Fig. 9 The figure is a schematic block diagram of a service according to an exemplary embodiment. DETAILED DESCRIPTION

[0076] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0077] like Figure 1 As shown, this embodiment provides an image classification method, including:

[0078] S110: extracting semantic features of the target image to be classified using a feature extraction part of the classification model;

[0079] S120: Processing the semantic feature using the classification part of the classification model to obtain a first classification result based on M classification, where M is a positive integer equal to or greater than 3;

[0080] S130: Based on the first classification result, output classification information indicating at least the degree to which the target image is a vulgar image; wherein the vulgar image includes: pornographic images, horror images and / or violent images.

[0081] In some embodiments, the image classification method in the embodiments of the present disclosure may be applied to a user device, wherein the user device includes a mobile terminal and a fixed terminal. Here, the mobile terminal includes a mobile phone, a laptop computer, a tablet computer, a wearable electronic device, a smart speaker, etc., and the fixed terminal includes a personal computer, a television, etc.

[0082] These user terminals are capable of performing image classification, and after completing the image classification, determine whether to display a certain target image or automatically block certain vulgar images based on the classification information.

[0083] For example, the user terminal has an adult mode and a child mode; in the child mode, in order to maintain the physical and mental health of children, the user terminal needs to filter out vulgar images through the image classification method, for example, shielding the display of vulgar images, intercepting the download of vulgar images, etc. In some embodiments, the user terminal can first download a preview image, which is much smaller than the image data of the original image, and then classify the image through the classification method. If the preview image is determined to be a vulgar image, the original image must be a vulgar image, and the download of the original image is refused. When children watch the images downloaded by the user terminal, on the one hand, the problem of downloading and displaying vulgar images that are not conducive to the physical and mental health of children is reduced; on the other hand, unnecessary traffic waste is reduced.

[0084] In some embodiments, the method can also be applied to a server. For example, some users upload images through a user terminal, and the uploaded images are the target images. After receiving the target images, the server can classify the images using the image classification method provided in the embodiment of the present application; then, based on the image classification, determine whether to make these images public on the network, or perform graded processing on these images.

[0085] The pornographic images may involve images with a high level of nudity;

[0086] The horror image may be an image that creates a horror atmosphere.

[0087] The violent images may involve bloody images, etc., specifically, images of fighting scenes and / or images of violent homicide scenes.

[0088] In the embodiment of the present application, in order to achieve accurate and fine-grained classification of such vulgar images, the first classification result based on M classification will be used; in this way, it will not only be simply determined whether the target image is a vulgar image, but also the degree to which it is a vulgar image. For example, the degree determination result is given according to the degree of similarity between the target image and the vulgar image, or the probability value of being classified as a vulgar image.

[0089] For example, taking pornographic images as an example, based on the M classification of pornographic images, it is determined whether the target image is a pornographic image according to the degree of pornographic images, and if it is a pornographic image, the pornographic level of the pornographic image is distinguished, for example, soft pornographic images, pornographic images and normal images. The pornographic level of soft pornographic images is lower than that of pornographic images.

[0090] For another example, taking a violent image as an example, based on the M classification of the violent image, it is determined whether the target image is a violent image, and if it is a violent image, the violence level of the violent image is distinguished, for example, soft violent images, violent images and normal images. Specifically, the value of M can be 3 or any value above 3. For example, the value of M can be 5, 6, 7, 8 or 9, etc. The violence level of a violent image is higher than that of a soft violent image.

[0091] Further, taking a horror image as an example, based on the M classification of the horror image, it is determined whether the target image is a horror image, and if it is a horror image, the horror level of the horror image is distinguished, for example, a soft horror image, a horror image, and a normal image. Specifically, the value of M can be 3 or any value above 3. For example, the value of M can be 5, 6, 7, 8, or 9. The horror level of a soft horror image is lower than that of a horror image.

[0092] In this way, through the first classification result of M classification, the determination of whether the target image is a vulgar image is realized, and the degree of detailed classification of the image is realized. It has the characteristics of high classification accuracy and can meet the detailed classification requirements in the detailed classification scenario.

[0093] In some embodiments, the method further comprises:

[0094] Mapping the first classification result to a second classification result based on N classifications, wherein N is a positive integer less than M;

[0095] The second classification result based on N classification is: the classification result based on the parent class;

[0096] The first classification result based on M classification is: a classification result based on subclasses, wherein at least one of the parent classes includes: a plurality of the subclasses.

[0097] In the embodiment of the present application, the classification model obtains a first classification result based on M classifications, and the first classification result can be further mapped to a second classification result based on N classifications. Here, N is less than M. For example, M is equal to 7, and N is equal to 3. At this time, the classification equivalent to the first classification result is a subclass classification result of the detailed classification, and each subclass has its corresponding parent class, and the second classification result is the parent class corresponding to the subclass.

[0098] By introducing the second classification result, when classifying images, each subclass-based classification can be mapped to the corresponding parent class according to the current classification requirements, thereby satisfying the classification of target images with different classification granularities.

[0099] For example, taking the classification of pornographic images as an example, it can be divided into 7 subcategories and sorted from heavy to light according to the degree of pornography. The first 4 subcategories can be determined as parent categories of pornographic images, the 5th subcategory can be classified as the parent category of soft pornographic images, and the last two subcategories can be determined as normal images, or parent categories of non-pornographic images.

[0100] By mapping the first classification result and the second classification result, the classification of the target image in different classification granularity scenarios is satisfied.

[0101] Furthermore, if Figure 2 As shown, when image classification is performed in S130, the output classification information may include:

[0102] S131: Outputting the first classification result separately;

[0103] S132: Outputting the second classification result separately;

[0104] S133: Output the first classification result and the second classification result simultaneously.

[0105] In this way, whether to output the first classification result or the second classification result separately, or to output the first classification result and the second classification result at the same time, can be differentiated according to the business scenario to meet the classification requirements of different classification scenarios.

[0106] In some embodiments, the step of extracting semantic features of the target image to be classified by using the feature extraction part of the classification model includes:

[0107] For the pornographic image, using the feature extraction part of the classification model, extract one or more of the clothing features, behavior features, exposure degree features of specific parts of the character elements, and overall exposure degree features of the character elements in the target image;

[0108] For the violent image, using the feature extraction part of the classification model, extract one or more of whether the target image contains violent elements, the number of violent elements, the proportion of the image area occupied by violent elements, and the degree of violence of the violent elements;

[0109] For the horror image, the feature extraction part of the classification model is used to extract one or more of whether the target image contains horror elements, the number of horror elements, the proportion of the image area occupied by horror elements, and the vulgarity of the horror elements.

[0110] In the embodiment of the present application, the classification model includes: an encoder and a decoder; the encoder is located before the decoder, the encoder can be used to extract image features, and the decoder is used to classify the image features.

[0111] For different specific images, semantic feature extraction of the target image is performed to extract various features indicating the degree of pornography, violence and / or terror, which is convenient for subsequent decoder classification.

[0112] For example, pornographic images can be classified into M and N categories according to the following table (Table 1).

[0113]

[0114]

[0115] Table 1

[0116] Of course, the above is only an example of the classification of the degree of pornographic images, and the specific implementation is not limited to the above example. Both the N classification and the M classification can be further subdivided.

[0117] If the classification model provided in the embodiment of the present application is used, Figure 4 should be classified as a normal image. Figure 4 The first classification result of the image shown may be sportswear, fitness clothing or thermal underwear. The classification result classified as soft pornography is the second classification result. The first classification result may be light exposure of women.

[0118] Before using the classification model to classify images in the embodiment of the present application, the classification model needs to be trained, and the training of the classification model here includes: online training and / or offline training. The method also includes: model training of the classification model.

[0119] The following provides a classification model training method, which can be used to train the above classification model, but is not limited to the above classification model. Figure 3 As shown, the model training of the classification model includes:

[0120] S210: Acquire a first sample image and an annotated label, wherein the annotated label is a subclass label of the M classification;

[0121] S220: Training the classification model using the first sample image and the annotated label;

[0122] S230: When the training loss of the classification model converges, stop training;

[0123] S240: Adding a mapping layer to the output layer of the classification model, which maps the first classification result to the second classification result.

[0124] When training the model, it is first necessary to obtain sample images, at least a first sample image is obtained here, and the first sample image at this time can be a photo downloaded from the Internet or actually taken.

[0125] These first sample images are annotated by experts or other trained high-precision classification models, that is, the annotated labels of the first sample images are obtained. The model is trained using at least the first sample images and the annotated labels.

[0126] When the training loss of the classification model converges, stop the training of the classification model, otherwise continue training until the training loss converges.

[0127] In some embodiments, the method further comprises:

[0128] Performing transformation processing on the first sample image to obtain a second sample image, wherein the second sample image has the same annotation label as the first sample image;

[0129] The step of training the classification model by using the first sample image and the annotated label includes:

[0130] The classification model is trained using the first sample image and the annotated label of the first sample image, and the second image sample and the annotated label of the second sample image.

[0131] The first image sample that already has an annotated label is subjected to an image transformation that does not change the essential content of the image, that is, does not affect the category to which the image belongs, thereby obtaining a second image sample; in this way, the number of image samples can be increased, and training can be performed by combining the first sample image and the second sample image to improve the generalization ability of the trained classification model.

[0132] For example, performing morphological transformation on the first sample image to obtain the second sample image includes at least one of the following:

[0133] Performing a linear transformation on the first sample image to obtain the second sample image, wherein the linear transformation includes: image rotation and / or image scaling;

[0134] Performing an image content change on the first sample image to obtain the second sample image, wherein the image content change includes: local mosaic processing, image content replacement in a local area, and / or local blurring processing;

[0135] Performing image parameter transformation on the first sample image to obtain the second sample image, wherein the image parameters include: contrast and / or brightness.

[0136] When the linear transformation is performed on different image contents, the first sample image is subjected to various linear transformations in whole or in part. The phenomenon transformations here include: overall image rotation, overall image scaling, partial image rotation and / or partial image scaling.

[0137] For example, the image is rotated by 90 degrees, 180 degrees, or 270 degrees, etc. Image scaling includes, but is not limited to, lengthening or widening the image, or scaling the image in a ratio of length to width.

[0138] The local rotation and / or local scaling of the image may include: cutting out a certain part of the image, for example, the central area of ​​the image, and performing scaling and rotation.

[0139] In some embodiments, the method of changing the first sample image into the second sample image further includes: cropping the first sample image to obtain the second sample image.

[0140] In other embodiments, the local image content is transformed for part or all of the content in the first sample image without affecting the image classification or degree. For example, by adding mosaic processing, the image is replaced with content other than mosaic. The local blur processing may include: Gaussian blur processing or blur processing of other algorithms.

[0141] The second sample image transformed from the first sample image may include:

[0142] A second sample image obtained by blurring the clear first sample image;

[0143] A second sample image obtained by rotating and blurring the video screenshot as the first sample image;

[0144] A second sample image obtained by mosaicking;

[0145] The second sample image is obtained by rotating and adding a QR code (ie, adding a mosaic).

[0146] In some embodiments, the classification model is a SEResNeXt model.

[0147] like Figure 5 As shown, this embodiment provides an image classification device, including:

[0148] An extraction module 110, for extracting semantic features of a target image to be classified using a feature extraction part of a classification model;

[0149] A classification module 120, configured to process the semantic feature using the classification part of the classification model to obtain a first classification result based on M classification, wherein M is a positive integer equal to or greater than 3;

[0150] The output module 130 is used to output classification information indicating at least that the target image is a vulgar image based on the first classification result, wherein the vulgar image includes: pornographic images, horror images and / or violent images.

[0151] In some embodiments, the extraction module 110, the classification module 120 and the output module 130 may all be program modules; after the program modules are executed by the processor, they can realize the extraction of semantic features, the acquisition of the first classification result and the output of degree classification information.

[0152] In other embodiments, the extraction module 110, the classification module 120 and the output module 130 may all be soft-hard combination modules; the soft-hard combination module includes but is not limited to a programmable array; the programmable array includes but is not limited to a complex programmable array or a field programmable array.

[0153] In some other embodiments, the extraction module 110, the classification module 120 and the output module 130 may all be pure hardware modules; the pure hardware modules include but are not limited to: application specific integrated circuits.

[0154] In some embodiments, the apparatus further comprises:

[0155] A mapping module, used to map the first classification result to a second classification result based on N classifications, wherein N is a positive integer less than M;

[0156] The second classification result based on N classification is: the classification result based on the parent class;

[0157] The first classification result based on M classification is: a classification result based on subclasses, wherein at least one of the parent classes includes: a plurality of the subclasses.

[0158] In some embodiments, the output module 130 is used to output a first classification result of the target image; and / or output a second classification result of the target image.

[0159] In some embodiments, the extraction module 110 is configured to perform at least one of the following:

[0160] For the pornographic image, using the feature extraction part of the classification model, extract one or more of the clothing features, behavior features, exposure degree features of specific parts of the character elements, and overall exposure degree features of the character elements in the target image;

[0161] For the violent image, using the feature extraction part of the classification model, extract one or more of whether the target image contains violent elements, the number of violent elements, the proportion of the image area occupied by violent elements, and the degree of violence of the violent elements;

[0162] For the horror image, the feature extraction part of the classification model is used to extract one or more of whether the target image contains horror elements, the number of horror elements, the proportion of the image area occupied by horror elements, and the vulgarity of the horror elements.

[0163] In some embodiments, the apparatus further comprises:

[0164] An acquisition module, used for acquiring a first sample image and a label before recognizing the image, wherein the label is a subclass label of the M classification;

[0165] A training module, used to train the classification model using the first sample image and the annotated label; when the training loss of the classification model converges, stop the training;

[0166] An adding module is used to add a mapping layer to the output layer of the classification model, which maps the first classification result to the second classification result.

[0167] In some embodiments, the apparatus further comprises:

[0168] a transformation module, configured to perform morphological transformation on the first sample image to obtain a second sample image, wherein the second sample image has the same annotation label as the first sample image;

[0169] The training module is used to train the classification model by using the first sample image and the annotated label of the first sample image, and the second image sample and the annotated label of the second sample image.

[0170] In some embodiments, the transformation module is used to perform at least one of the following:

[0171] Performing a linear transformation on the first sample image to obtain the second sample image, wherein the linear transformation includes: image rotation and / or image scaling;

[0172] Performing an image content change on the first sample image to obtain the second sample image, wherein the image content change includes: local mosaic processing, image content replacement in a local area, and / or local blurring processing;

[0173] Performing image parameter transformation on the first sample image to obtain the second sample image, wherein the image parameters include: contrast and / or brightness.

[0174] In some embodiments, the classification model is a SEResNeXt model.

[0175] A specific example is provided below in combination with any of the above embodiments:

[0176] Example 1:

[0177] This solution proposes a multi-granular classification standard for pornographic images, which makes the image semantics clearer, and the classification standards for pornographic, soft pornographic, and normal images clearer. In addition, this application proposes a data enhancement preprocessing method and uses the network results of the more powerful SEResNeXt as a classification model. Experiments show that the pornographic image method of this solution has better results and stronger explanatory power.

[0178] This application divides UGC online images into 3 major categories and 7 subcategories. These 7 major categories include various obvious pornographic displays and pornographic behaviors, as well as the definition of sexy soft pornography. Among them, a clear distinction is made between the degree of exposure of the female body, and the exposure of the male body and the exposure of the female body are distinguished. There are a total of 14 classification standards:

[0179]

[0180] According to the classification criteria in Table (a), 100,000 sample images are annotated, and each annotated sample is divided into one of the seven categories of the subdivision category.

[0181] Step 2: Data preprocessing and data enhancement, which may include:

[0182] After the data is labeled, in order to make the model learn the semantic features of each category of data, data enhancement can make the same semantic image show more forms. This application enhances the image by rotating, locally enlarging, randomly mosaicking small areas, adjusting brightness and contrast, etc.

[0183] Step 3: Construct a classification model for pornographic images and other vulgar images, which may include: Although the ResNet model solves the problem that deep networks are difficult to train due to problems such as gradient disappearance or gradient explosion, the network structure is relatively simple and the performance is limited. This application uses the SEResNeXt model as the classification model. The SEResNeXt model is the development and evolution of ResNet and has better classification performance and robustness. Figure 6 As shown in the figure, in the convolution operation, the SE ResNeXt model replaces the three-layer convolution blocks of the original ResNet model by stacking blocks of the same topological structure in parallel, and adds the SE module - Squeeze to obtain the global features at the channel level, and then performs an excitation operation on the global features to learn the relationship between each channel and obtain the weights of different channels. In this way, the network structure is deepened and strengthened, making the network's feature extraction capability stronger and the model's robustness is also improved. Figure 7 That is, the network structure of the SE module that converges the global features at the channel level to obtain the final output. Figure 6 The left side of the figure shows the three-layer convolution module of the ResNet model; the right side shows the three parallel stacked convolution modules of the SEResNeXt model.

[0184] Step 4: Model training of the classification model. Specifically, the classification model is trained using the following parameters:

[0185]

[0186] Step 5: The classification model predicts the image, which may include: using the classification model obtained in step 4 to predict the new image, obtaining one of the seven classification results, and giving the probability value.

[0187] This solution uses hierarchical fine-grained classification standards to capture the main semantic features of the image, making the semantic representation of different categories of data more significant. At the same time, the fine-grained classification standards enable a better combination of algorithm standards and business standards, allowing the model classification results to be flexibly changed according to the business scale and more interpretable.

[0188] Through data enhancement and SEResNeXt network structure, the algorithm performance is stronger and the classification accuracy is higher. The following table shows the test results of two deep learning algorithms:

[0189]

[0190] Figure 88 is a block diagram of a mobile terminal according to an exemplary embodiment. For example, the mobile terminal 800 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. The mobile terminal may be used to execute the image classification method provided by any of the aforementioned technical solutions.

[0191] Reference Figure 8 The mobile terminal 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0192] The processing component 802 generally controls the overall operation of the mobile terminal 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0193] The memory 804 is configured to store various types of data to support operations on the mobile terminal 800. Examples of such data include instructions for any application or method operating on the mobile terminal 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0194] The power component 806 provides power to the various components of the mobile terminal 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the mobile terminal 800.

[0195] The multimedia component 808 includes a screen that provides an output interface between the mobile terminal 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the mobile terminal 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0196] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the mobile terminal 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0197] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0198] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the mobile terminal 800. For example, the sensor assembly 814 can detect the open / closed state of the mobile terminal 800, the relative positioning of components, such as the display and keypad of the mobile terminal 800, and the sensor assembly 814 can also detect the position change of the mobile terminal 800 or a component of the mobile terminal 800, the presence or absence of user contact with the mobile terminal 800, the orientation or acceleration / deceleration of the mobile terminal 800, and the temperature change of the mobile terminal 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0199] The communication component 816 is configured to facilitate wired or wireless communication between the mobile terminal 800 and other devices. The mobile terminal 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0200] In an exemplary embodiment, the mobile terminal 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0201] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the mobile terminal 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0202] An embodiment of the present application provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute an image classification method, the method comprising: extracting semantic features of a target image to be classified using a feature extraction part of a classification model; processing the semantic features using a classification part of the classification model to obtain a first classification result based on M classification, wherein M is a positive integer equal to or greater than 3; based on the first classification result, outputting classification information indicating at least the degree to which the target image is a vulgar image, wherein the vulgar image includes: pornographic images, horror images and / or violent images.

[0203] In some embodiments, the method further comprises:

[0204] Mapping the first classification result to a second classification result based on N classifications, wherein N is a positive integer less than M;

[0205] The second classification result based on N classification is: the classification result based on the parent class;

[0206] The first classification result based on M classification is: a classification result based on subclasses, wherein at least one of the parent classes includes: a plurality of the subclasses.

[0207] In some embodiments, outputting classification information indicating at least that the target image is a vulgar image based on the first classification result includes:

[0208] Outputting a first classification result of the target image;

[0209] and / or,

[0210] Output a second classification result of the target image.

[0211] In some embodiments, the extracting semantic features of the target image to be classified by using the feature extraction part of the classification model includes at least one of the following:

[0212] For the pornographic image, using the feature extraction part of the classification model, extract one or more of the clothing features, behavior features, exposure degree features of specific parts of the character elements, and overall exposure degree features of the character elements in the target image;

[0213] For the violent image, using the feature extraction part of the classification model, extract one or more of whether the target image contains violent elements, the number of violent elements, the proportion of the image area occupied by violent elements, and the degree of violence of the violent elements;

[0214] For the horror image, the feature extraction part of the classification model is used to extract one or more of whether the target image contains horror elements, the number of horror elements, the proportion of the image area occupied by horror elements, and the vulgarity of the horror elements.

[0215] In some embodiments, before identifying the image, the method further comprises:

[0216] Obtaining a first sample image and an annotated label, wherein the annotated label is a subclass label of the M classification;

[0217] Using the first sample images and the annotated labels, training the classification model;

[0218] When the training loss of the classification model converges, stop training;

[0219] A mapping layer for mapping the first classification result to the second classification result is added to the output layer of the classification model.

[0220] In some embodiments, the method further comprises:

[0221] Performing morphological transformation on the first sample image to obtain a second sample image, wherein the second sample image has the same annotation label as the first sample image;

[0222] The step of training the classification model by using the first sample image and the annotated label includes:

[0223] The classification model is trained using the first sample image and the annotated label of the first sample image, and the second image sample and the annotated label of the second sample image.

[0224] In some embodiments, performing morphological transformation on the first sample image to obtain the second sample image includes at least one of the following:

[0225] Performing a linear transformation on the first sample image to obtain the second sample image, wherein the linear transformation includes: image rotation and / or image scaling;

[0226] Performing an image content change on the first sample image to obtain the second sample image, wherein the image content change includes: local mosaic processing, image content replacement in a local area, and / or local blurring processing;

[0227] Performing image parameter transformation on the first sample image to obtain the second sample image, wherein the image parameters include: contrast and / or brightness.

[0228] The classification model is a SEResNeXt model.

[0229] Fig. 9 19 is a block diagram of a server according to an exemplary embodiment. For example, server 1900 may be provided as a server. Referring to FIG. 19 , server 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method...

[0230] The server 1900 may also include a power supply component 1926 configured to perform power management of the server 1900, a wired or wireless network interface 1950 configured to connect the server 1900 to a network, and an input / output (I / O) interface 1958. The server 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0231] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0232] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image classification method, It is characterized in that include: The feature extraction part of the classification model is used to extract the semantic features of the target image to be classified; The method of extracting the semantic features of the target image to be classified by using the feature extraction part of the classification model includes: extracting the features of the target image to be classified by using a plurality of convolution modules stacked in parallel by the classification model, and aggregating the features output by the plurality of convolution modules to obtain a global feature as the semantic feature; performing an excitation operation on the global feature by using the classification model to obtain weights of different channels; the method of extracting the semantic features of the target image to be classified by using the feature extraction part of the classification model also includes at least one of the following: extracting one or more of the number of violent elements, the proportion of the image area occupied by the violent elements, and the degree of violence of the violent elements of a violent image by using the feature extraction part of the classification model; extracting one or more of the number of terrorist elements, the proportion of the image area occupied by the terrorist elements, and the degree of vulgarity of the terrorist elements of a terrorist image by using the feature extraction part of the classification model; Processing the semantic feature using the classification part of the classification model to obtain a first classification result based on M classification, wherein M is a positive integer equal to or greater than 3; Based on the first classification result, outputting at least degree classification information indicating that the target image is a vulgar image, wherein the vulgar image includes: a pornographic image, the horror image and / or the violent image; Mapping the first classification result to a second classification result based on N classifications using a mapping layer after the output layer of the classification model; wherein N is a positive integer less than M; The second classification result based on N classification is: the classification result based on the parent class; The first classification result based on M classification is: a classification result based on subclasses, wherein at least one of the parent classes includes: a plurality of the subclasses.

2. The method according to claim 1, It is characterized in that The step of outputting classification information indicating at least that the target image is a vulgar image based on the first classification result includes: Outputting a first classification result of the target image; and / or, Output a second classification result of the target image.

3. The method according to claim 1, It is characterized in that Before identifying the image, the method further includes: Obtaining a first sample image and an annotated label, wherein the annotated label is a subclass label of the M classification; Using the first sample images and the annotated labels, training the classification model; When the training loss of the classification model converges, stop training; A mapping layer for mapping the first classification result to the second classification result is added to the output layer of the classification model.

4. The method according to claim 3, It is characterized in that The method further comprises: Performing morphological transformation on the first sample image to obtain a second sample image, wherein the second sample image has the same annotation label as the first sample image; The step of training the classification model by using the first sample image and the annotated label includes: The classification model is trained using the first sample image and the annotated label of the first sample image, and the second image sample and the annotated label of the second sample image.

5. The method according to claim 4, It is characterized in that The step of performing morphological transformation on the first sample image to obtain a second sample image includes at least one of the following: Performing a linear transformation on the first sample image to obtain the second sample image, wherein the linear transformation includes: image rotation and / or image scaling; Performing an image content change on the first sample image to obtain the second sample image, wherein the image content change includes: local mosaic processing, image content replacement in a local area, and / or local blurring processing; Performing image parameter transformation on the first sample image to obtain the second sample image, wherein the image parameters include: contrast and / or brightness.

6. The method according to claim 1, It is characterized in that The classification model is a SEResNeXt model.

7. An image classification device, It is characterized in that include: An extraction module, used for extracting semantic features of a target image to be classified by using a feature extraction part of a classification model; The extraction module is specifically used to extract the features of the target image to be classified through multiple convolution modules stacked in parallel in the classification model, and aggregate the features output by the multiple convolution modules to obtain a global feature as the semantic feature; The classification model is used to perform an excitation operation on the global features to obtain weights of different channels, and the semantic features of the target image to be classified are extracted through the weights; the extraction module is further specifically used to perform at least one of the following: using the feature extraction part of the classification model, extracting one or more of the number of violent elements in the target image, the proportion of the image area occupied by the violent elements, and the degree of violence of the violent elements; using the feature extraction part of the classification model, extracting one or more of the number of terrorist elements in the target image, the proportion of the image area occupied by the terrorist elements, and the degree of vulgarity of the terrorist elements; A classification module, used for processing the semantic feature by using the classification part of the classification model to obtain a first classification result based on M classification, wherein M is a positive integer equal to or greater than 3; an output module, configured to output, based on the first classification result, at least classification information indicating that the target image is a vulgar image, wherein the vulgar image includes: a pornographic image, the horror image and / or the violent image; A mapping module, wherein the mapping layer after the output layer of the classification model maps the first classification result to a second classification result based on N classifications, wherein N is a positive integer less than M; The second classification result based on N classification is: the classification result based on the parent class; The first classification result based on M classification is: a classification result based on subclasses, wherein at least one of the parent classes includes: a plurality of the subclasses.

8. The device according to claim 7, It is characterized in that The output module is used to output a first classification result of the target image; and / or to output a second classification result of the target image.

9. The device according to claim 7, It is characterized in that The device also includes: An acquisition module, used for acquiring a first sample image and a label before recognizing the image, wherein the label is a subclass label of the M classification; A training module, used to train the classification model using the first sample image and the annotated label; when the training loss of the classification model converges, stop the training; An adding module is used to add a mapping layer to the output layer of the classification model, which maps the first classification result to the second classification result.

10. The device according to claim 9, It is characterized in that The device also includes: a transformation module, configured to perform morphological transformation on the first sample image to obtain a second sample image, wherein the second sample image has the same annotation label as the first sample image; The training module is used to train the classification model by using the first sample image and the annotated label of the first sample image, and the second image sample and the annotated label of the second sample image.

11. The device according to claim 10, It is characterized in that The transformation module is used to perform at least one of the following: Performing a linear transformation on the first sample image to obtain the second sample image, wherein the linear transformation includes: image rotation and / or image scaling; Performing an image content change on the first sample image to obtain the second sample image, wherein the image content change includes: local mosaic processing, image content replacement in a local area, and / or local blurring processing; Performing image parameter transformation on the first sample image to obtain the second sample image, wherein the image parameters include: contrast and / or brightness.

12. The device according to claim 7, It is characterized in that The classification model is a SEResNeXt model.

13. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method provided in any one of claims 1 to 6 by executing the executable instructions.

14. A non-transitory computer-readable storage medium storing executable instructions; after the executable instructions are executed by a processor, the method provided in any one of claims 1 to 6 can be implemented.

Images