Image processing method, device, computer equipment and storage medium

Abstract

The present disclosure is about an image processing method, apparatus, computer equipment and storage medium, the method comprising: obtaining a target image to be processed; determining the semantic information of the target image according to a first model and the target image, wherein the first model comprises: a model obtained by performing transfer learning training on a second model based on a first image sample set of a preset type, the second model comprises: a pre-trained model obtained by pre-training using a first image database data set, wherein the proportion of the number of images in the first image sample set of the preset type to the number of images in the first image database data set is less than a first proportion threshold; determining whether the target image is an image of the preset type according to the semantic information of the target image. Therefore, in the case where the distribution of various types of sample data is unbalanced and / or the sample data of the preset type is insufficient, the image recognition rate of the preset type of image is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, computer equipment and storage medium. Background Art

[0002] Computer Vision (CV) is one of the important directions of artificial intelligence software technology. It refers to the use of cameras and computers to replace human eyes to identify, track and measure targets. Computer vision technology refers to the study of certain deep learning models to intelligently classify images input into these deep learning models.

[0003] In the related technologies, for some sample data with unbalanced distribution, especially for image data with small samples or few samples that cannot obtain sufficient sample numbers, the image samples of a certain class or multiple classes are distributed in small numbers and the numbers are not sufficient, which will make these deep learning models have low recognition accuracy for the images corresponding to this class of image samples, or even fail to recognize them. Based on this, how to improve the image recognition accuracy and recognition rate for image data with few or small samples has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present disclosure is to provide an image processing method, apparatus, computer equipment and storage medium, aiming to improve the recognition accuracy and recognition rate of image data for a small number of samples or samples.

[0005] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0007] Obtaining a target image to be processed;

[0008] Determine semantic information of the target image according to the first model and the target image, wherein the first model comprises: a model obtained by performing transfer learning training on a second model based on a first image sample set of a preset type, and the second model comprises: a pre-trained model obtained by training using a first image database data set, wherein the proportion of the number of images of the first image sample set of the preset type in the number of images in the first image database data set is less than a first proportion threshold;

[0009] It is determined whether the target image is an image of the preset type according to the semantic information of the target image.

[0010] In the above scheme, the method further includes:

[0011] Acquire the first image sample set of the preset type from a second image database data set, wherein a ratio of the number of images of the first image samples of the preset type to the number of images in the second image database data set is greater than a second ratio threshold, wherein the second ratio threshold is greater than the first ratio threshold.

[0012] In the above solution, determining the third model according to the first model includes:

[0013] Performing transfer learning training on M fourth models according to the model parameters of the first model to obtain M fifth models, wherein M is a positive integer greater than or equal to 1, and the fourth model includes: a pre-trained model obtained by training with the second image sample set in the cloud database;

[0014] Performing integrated learning on the M fifth models to obtain the third model;

[0015] The target image is processed using the third model to determine the semantic information of the target image.

[0016] In the above solution, the fourth model is trained according to the second image sample set, including:

[0017] Dividing the second image sample set into a first positive image sample set and a first negative image sample set, wherein the images in the positive image sample set are images of the preset type, and the images in the negative image sample set are not images of the preset type;

[0018] Performing data enhancement processing on the first positive image sample set to obtain a second positive image sample set, and performing data distillation processing on the first negative image sample set to obtain a second negative image sample set;

[0019] The fourth model is obtained by training according to the second positive image sample set and the second negative image sample set.

[0020] In the above solution, the fourth model is obtained by training according to the second positive image sample set and the second negative image sample set, including:

[0021] In response to a ratio of the number of images of the second positive image sample set to that of the second negative image sample set being less than a ratio threshold, sampling the second negative image sample set to obtain a third negative image sample set, wherein a ratio of image data of the second image sample set to that of the third negative image sample set is greater than or equal to the ratio threshold;

[0022] The fourth model is obtained by training according to the second positive image sample set and the third negative image sample set.

[0023] In the above solution, the third model can divide the image samples into a correctly classified sample set and an incorrectly classified sample set;

[0024] The method further comprises:

[0025] In response to the third model not satisfying the convergence condition, the third model is adjusted until the adjusted third model satisfies the convergence condition, wherein the convergence condition is: the proportion of the N-th misclassified image sample set obtained by classifying the N-1-th misclassified image sample set using the third model adjusted for the N-1-th time to the N-1-th misclassified image sample set is less than a proportion threshold, wherein N is a positive integer greater than or equal to 1, and when N is equal to 1, the N-1-th misclassified image sample set is the third image sample set;

[0026] The third image sample set is classified by using the third model adjusted for the N-1th time to obtain an Nth correctly classified image sample set, and the Nth correctly classified image sample set is used as the second image sample set.

[0027] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0028] A first acquisition module, used for acquiring a target image to be processed;

[0029] A first determination module is configured to determine semantic information of the target image based on a first model and the target image, wherein the first model comprises: a model obtained by performing transfer learning training on a second model based on a first image sample set of a preset type, and the second model comprises: a pre-trained model obtained by training using a first image database data set, wherein a proportion of the number of images of the first image sample set of the preset type in the number of images in the first image database data set is less than a first proportion threshold;

[0030] The second determination module is used to determine whether the target image is an image of the preset type according to the semantic information of the target image.

[0031] In the above solution, the device further comprises:

[0032] A second acquisition module is used to acquire the first image sample set of the preset type from a second image database data set, wherein a proportion of the number of images in the first image sample set of the preset type to the number of images in the second image database data set is greater than a second proportion threshold, and the second proportion threshold is greater than the first proportion threshold.

[0033] In the above solution, the first determining module includes:

[0034] A first obtaining submodule is used to perform transfer learning training on M fourth models according to the model parameters of the first model to obtain M fifth models, wherein M is a positive integer greater than or equal to 1, and the fourth model includes: a pre-trained model obtained by training with a second image sample set in a cloud database;

[0035] The second submodule is used to perform integrated learning on the M fifth models to obtain the third model.

[0036] The first determination submodule is used to process the target image using the third model to determine the semantic information of the target image.

[0037] In the above solution, the device further includes: a third determination module, configured to determine the fourth model according to the second image sample set;

[0038] The third determining module comprises:

[0039] a division submodule, configured to divide the second image sample set into a first positive image sample set and a first negative image sample set;

[0040] A third obtaining submodule is used to perform data enhancement processing on the first positive image sample set to obtain a second positive image sample set, and to perform data distillation processing on the first negative image sample set to obtain a second negative image sample set;

[0041] The third obtaining submodule is used to train and obtain the fourth model according to the second positive image sample set and the second negative image sample set.

[0042] In the above solution, the fourth obtaining submodule is further used for:

[0043] In response to a ratio of the number of images in the second positive image sample set to the number of images in the second negative image sample set being less than a ratio threshold, sampling the second negative image sample set to obtain a third negative image sample set, wherein the ratio of the number of images in the second image sample set to the third negative image sample set is greater than or equal to the ratio threshold;

[0044] The fourth model is obtained by training according to the second positive image sample set and the third negative image sample set.

[0045] In the above solution, the third model can divide the image samples into a correctly classified sample set and an incorrectly classified sample set;

[0046] The device also includes:

[0047] an adjustment module, configured to adjust the third model in response to the third model not satisfying the convergence condition, until the adjusted third model satisfies the convergence condition, wherein the convergence condition is: the proportion of the N-th misclassified image sample set obtained by classifying the N-1-th misclassified image sample set using the third model adjusted for the N-1-th time to the N-1-th misclassified image sample set is less than a proportion threshold, wherein N is a positive integer greater than or equal to 1, and when N is equal to 1, the N-1-th misclassified image sample set is the third image sample set;

[0048] A processing module is used to classify the third image sample set by using the third model adjusted for the N-1th time to obtain an Nth correctly classified image sample set, and use the Nth correctly classified image sample set as the second image sample set.

[0049] According to a third aspect of an embodiment of the present disclosure, a computer device is provided, including:

[0050] A processor and a memory for storing a computer program that can be run on the processor, wherein the processor is used to implement any of the above-mentioned steps of the image processing method when running the computer program.

[0051] According to a fourth aspect of an embodiment of the present disclosure, there is provided a non-temporary computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement any of the steps of the image processing method described above.

[0052] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0053] In an embodiment of the present disclosure, a target image to be processed is obtained; and based on a first model and the target image, the semantic information of the target image is determined, where the first model includes: a model obtained by performing transfer learning training on the second model based on a first image sample set of a preset type, and the second model includes: a pre-trained model trained using a first image database data set, wherein the proportion of the number of images in the first image sample set of the preset type to the number of images in the first image database data set is less than a first proportion threshold; finally, based on the semantic information of the target image, it is determined whether the target image is an image of the preset type. That is, the embodiment of the present disclosure uses a first image sample set of a preset type to perform transfer learning optimization processing on a pre-trained model pre-trained using a first image database data set, and obtains a first model that has learned sample features of the first image sample set of the preset type. In this way, since the semantic information of the target image is extracted by the first model, and since the first model is a model obtained by introducing a first image sample set of a preset type to perform transfer learning optimization training on the second model, it is possible to make up for the lack of semantic information of the second model for the preset type of image, so that the semantic information of the target image extracted by the first model can be more accurate, so that it can be more accurately determined whether the target image belongs to the preset type of image based on the semantic information of the target image. In this way, when the distribution of sample data of each type is unbalanced and / or the sample data of the preset type is insufficient, the accuracy of image recognition for the preset type of image can be improved, and the failure rate of image recognition for the preset type of image can be reduced to improve the image recognition rate for the preset type of image. This further improves the image classification performance of the computer device.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0056] Figure 1 is a flow chart of a soft image processing method according to an exemplary embodiment;

[0057] Figure 2 is a schematic diagram of a scene of an image processing method according to an exemplary embodiment;

[0058] Figure 3 is another scene schematic diagram of an image processing method according to an exemplary embodiment;

[0059] Figure 4 is another scene schematic diagram of an image processing method according to an exemplary embodiment;

[0060] Figure 5 is a flow chart of an image processing method according to a specific embodiment;

[0061] Figure 6 is a functional module diagram of image processing according to an exemplary embodiment;

[0062] Figure 7 The invention is a block diagram of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0063] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0064] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0065] Machine Learning (ML) is a multi-disciplinary cross-discipline involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and inductive learning. In the image processing method of the present embodiment, an image recognition model is obtained by training through a machine learning method, so that the image to be processed can be better recognized in order to achieve accurate classification.

[0066] Deep learning (DL) is a new research direction in the field of machine learning. It is introduced into machine learning to make it closer to its original goal - artificial intelligence AI. Deep learning is to learn the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds. It is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural speech processing, multimedia learning, speech, recommendation and personalization technology, and other related fields. Deep learning enables machines to imitate human activities such as seeing, hearing and thinking, solves many complex pattern recognition problems, and makes many advances in artificial intelligence related technologies.

[0067] Convolutional Neural Networks (CNN) are a type of feedforward neural network that includes convolutional calculations and has a deep structure. It is one of the representative algorithms of deep learning. Convolutional neural networks have the ability of representation learning and can perform shift-invariant classification on input information according to its hierarchical structure. Therefore, they are also called "Shift-Invariant Artificial Neural Networks (SIANN)".

[0068] It should be noted that with the development of Internet technology, users can obtain or publish various information on the Internet, and more than 70% of the data on the Internet is image or video data. In order to better serve the majority of users, how to better distinguish the categories of distributed or acquired image or video data has become the main topic of current research. In related technologies, deep learning models are introduced to intelligently classify images obtained or published on the Internet. However, the recognition accuracy of existing deep learning models is closely related to the amount of sample data used in the training of deep learning models. It is understandable that if the number of samples of a certain type of image used to train the deep learning model is insufficient, it will inevitably affect the recognition of the deep learning model for that type of image.

[0069] The embodiments of the present disclosure are intended to improve the recognition accuracy and recognition rate of certain types of images in machine learning when samples of certain types are insufficient.

[0070] The embodiments of the present disclosure are applicable to but not limited to recognition scenarios of network image data, for example, the review of image data posted or obtained by users by application manufacturers, or the intelligent classification of images posted by users by application manufacturers for classified display of images, or for intelligent search based on image classification, etc.

[0071] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1 As shown, the method comprises the following steps:

[0072] Step 101: Acquire a target image to be processed;

[0073] Step 102: determining semantic information of the target image according to the first model and the target image, wherein the first model comprises: a model obtained by performing transfer learning training on a second model based on a first image sample set of a preset type, and the second model comprises: a pre-trained model obtained by training using a first image database data set, wherein a proportion of the number of images of the first image sample image set of the preset type to the number of images in the first image database data set is less than a first proportion threshold;

[0074] Step 103: Determine whether the target image is an image of the preset type according to the semantic information of the target image.

[0075] Here, in step 101, the obtaining of the target image to be processed may include: obtaining the target image to be processed from the computer device itself, that is, performing corresponding processing on the target image to be processed stored in the computing device itself. For example, in some scenarios, the computer device may be a server of an application manufacturer, and the method may be used to identify and classify images to be published on the application of the application manufacturer, or to intelligently classify images to be published on the application of the application manufacturer.

[0076] In other embodiments, in step 101, the obtaining of the target image to be processed may also include: obtaining the target image to be processed from a device other than the computer device itself, that is, being able to intelligently identify and classify images of devices other than the computer device itself.

[0077] Here, semantic information refers to the semantic representation that can be used as the target image, that is, it can express the objects contained in the target image. For example, if the target image is a landscape image, the semantic information can be used to represent the landscape information; if the target image is a person image, the semantic information can be used to represent the person information. Here, the semantic information can be represented by numbers or characters, and different semantic information corresponds to different objects contained in the target image to be represented.

[0078] Here, the preset type can be the type of the part of images that are difficult to obtain during sample collection. In other words, there are few images of the preset type during sample collection, and it is difficult to reach the quantity requirement for training a deep learning model. Here, the quantity requirement for training a deep learning model refers to the number of samples required to enable the deep learning model to recognize images of this type with an accuracy rate that reaches the accuracy threshold.

[0079] Here, the proportion of the number of images of the preset type in the first image database data set is less than the first ratio threshold. The first ratio threshold may be the ratio of the number of images of the preset type as samples to the number of all image samples, which meets the minimum ratio requirement for training a deep learning model. Here, the minimum ratio requirement for training a deep learning model refers to the minimum ratio requirement of the number of samples that enables the accuracy of deep learning to recognize images of this type to reach the accuracy threshold.

[0080] That is to say, if the proportion of images of the preset type in the first image database data set is less than the minimum proportion requirement for training a deep learning model, then the model trained using the first image database data set will not be able to correctly identify images of the preset type.

[0081] Here, the first image database refers to a database used for storing image samples in the field of image processing, and the first image database data set is a certain number of sample sets obtained randomly or according to preset rules from the first image database.

[0082] In some embodiments, the first image database may be an ImageNet database, which is a large visual database used for visual object recognition software research. More than 14 million image URLs (Uniform Resource Loacator) are manually annotated by ImageNet to indicate objects in the pictures; in at least one million images, bounding boxes are also provided.

[0083] In fact, even if the ImageNet database is used, there will still be some types of image samples missing or missing. For example, some vulgar images or some uncommon images in special fields are also rarely available in the ImageNet database. Therefore, the pre-trained model trained using the first image database data set is often not accurate enough in recognizing the semantic information of those missing or missing image types, or even cannot recognize them.

[0084] Of course, in other embodiments, the first image database may also be other image databases, such as image databases provided by some large cloud service providers. In short, the proportion of the number of images of the preset type in the first image database data set is less than the first proportion threshold.

[0085] Here, the second model includes: a pre-trained model trained using the first image database data set. It should be noted that the second model has the ability to obtain the semantic information expression of the image. That is, by inputting the image into the second model, the semantic information of the image can be output.

[0086] Here, transfer learning (TL, Tansfer Learning) is to transfer the parameters of the trained model (pre-trained model) to the new model to help the new model training. Considering that most data or tasks are related, transfer learning can share the learned model parameters with the new model in some way to speed up and optimize the learning efficiency of the model, which is different from learning from scratch like most networks. Here, by performing transfer learning on the second model based on the first image sample set of a preset type, the first model generated by optimizing the second model can have the ability to express the semantic information of the preset type of image.

[0087] The second model here can be a deep learning model with a convolutional neural network CNN, wherein the convolutional neural network includes but is not limited to one of the following: a residual network structure (ResNet, Residual Network), a clustered residual transformation network structure of a deep neural network (ResNeXt-Aggregated Residual Transformations for Deep Neural Networks), or a visualization and understanding network structure of a convolutional network (ZFNet, Visualizing and Understanding Convolutional Networks). In short, the second model is a model trained using the first image database data set.

[0088] In this way, by introducing a first image sample set of a preset type, that is, introducing an image sample set for an image type that is lacking or missing in the first image database, to perform transfer learning training on the second model, the model trained for transfer learning by introducing the first image sample set of a preset type can better identify the semantic information of the image, reduce the lack of semantic representation of images of the preset type, and provide a more accurate classification basis for subsequent classification of the target image based on the semantic information of the target image.

[0089] In some embodiments, determining the semantic information of the target image according to the first model and the target image may include:

[0090] The target image is input into the first model, and semantic information of the target image is output.

[0091] It can be understood that the first model here can be directly used as a processing model for processing the target image, that is, the semantic information of the target image can be directly identified through the first model.

[0092] In this embodiment, the target image is recognized by directly using the first model. Since the first model is a model obtained by transfer learning training of the second model based on a first image sample set of a preset type, the first model can reduce the lack of semantic representation of images of the preset type, improve the accuracy of image recognition, and thus provide a better basis for image classification in the future.

[0093] Of course, in some other embodiments, determining the semantic information of the target image according to the first model and the target image may also include:

[0094] The target image is input into a model related to the first model, and semantic information of the target image is output.

[0095] Here, the first model serves as a model for indirectly outputting semantic information of the target image. Also due to the participation of the first model, the computer device can more accurately recognize images of the preset type.

[0096] Thus, in the above embodiment, since the first model includes: a model obtained by performing transfer learning training on the second model based on a first image sample set of a preset type, a certain correction can be made to the small number of image samples of the preset type in the second model, so that the semantic information of the target image is determined by the model obtained by performing transfer learning training using an image sample set of the preset type, that is, the semantic information of the target image is determined by the first model, which can more effectively identify the semantic information of the target image and reduce the lack of semantic representation of the target image for the preset type of image, so that the recognition of the semantic information of the target image is more accurate, providing a more accurate classification basis for the subsequent classification of the target image based on the semantic information of the target image.

[0097] Based on this, by determining the semantic information of the target image according to the first model and the target image, the semantic representation of the target image can be obtained more accurately, reducing the lack of semantic representation for images of preset types, thereby laying the foundation for subsequent determination of whether the target image is an image of preset types based on the semantic information of the target image.

[0098] It can be understood that, according to the semantic information of the target image, the object contained in the target image can be determined, thereby determining the image type of the target image.

[0099] In some embodiments, the step 103, i.e., determining whether the target image is an image of the preset type according to the semantic information of the target image, may include:

[0100] Determining object information contained in the target image according to the semantic information of the target image;

[0101] Determining the type of the target image according to the object information contained in the target image;

[0102] According to the type of the target image, it is determined whether the target image is an image of the preset type.

[0103] Here, the object information includes but is not limited to: the type of objects contained in the target image, the number of each object, the area occupied by each object in the target image, and the importance of the object in the target image.

[0104] In some embodiments, determining the type of the target image according to the object information contained in the target image may include:

[0105] The type of the target image is determined according to at least one of the type of objects contained in the target image and the number of each object, the area of ​​the target image occupied by each object, and the importance of the object in the target image.

[0106] Exemplarily, determining the type of the target image based on the object information contained in the target image may include: according to the types of objects contained in the target image and the number of each object, taking the type corresponding to the largest number of objects as the type of the target image.

[0107] Exemplarily, determining the type of the target image based on the object information contained in the target image may include: based on the types of objects contained in the target image and the area of ​​the target image occupied by each object, taking the type corresponding to the object with the largest area as the type of the target image.

[0108] Exemplarily, determining the type of the target image based on the objects contained in the target image may include: based on the types of objects contained in the target image and the importance of each object in the target image, taking the type corresponding to the most important object as the type of the target image.

[0109] Here, the importance of each object in the target image can be determined by whether each object is focused or blurred. It can be understood that the importance of the focused object is greater than the importance of the blurred object. Of course, the importance of each object in the target image can also be determined by other methods, which are not limited here.

[0110] In this embodiment, by determining the object contained in the target image based on the semantic information of the target image, the image type of the target image can be determined, and finally determining whether the target image is an image of a preset type can more accurately classify the target image.

[0111] In the above embodiment, since the semantic information of the target image is extracted by the first model, and since the first model is a model obtained by introducing a first image sample set of a preset type to perform transfer learning optimization training on the second model, it is possible to make up for the lack of semantic information of the second model for the preset type of image, so that the semantic information of the target image extracted by the first model can be more accurate, so that it is possible to more accurately determine whether the target image belongs to the preset type of image based on the semantic information of the target image. In this way, when the distribution of sample data of each type is unbalanced and / or the sample data of the preset type is insufficient, the accuracy of image recognition for the preset type of image can be improved, and the failure rate of image recognition for the preset type of image can be reduced to increase the image recognition rate for the preset type of image. This in turn improves the image classification performance of the computer device.

[0112] Based on this, in the case where images of preset types are not allowed on the network or are not allowed by law, the method described in this embodiment can also improve the accuracy of the selection of images of preset types during the review, reduce the flow of images of preset types into the network, and provide favorable guarantees for purifying the network environment.

[0113] As an optional embodiment, the method further includes:

[0114] The first image sample set of the preset type is obtained from a second image database data set, wherein the image sample set of the preset type, wherein a proportion of the number of images of the first image samples of the preset type in the number of images in the second image database data set is greater than a second proportion threshold, and the second proportion threshold is greater than the first proportion threshold.

[0115] Here, the proportion of the number of images of the preset type in the second image database data set is greater than a second proportion threshold, wherein the second proportion threshold is greater than the first proportion threshold, that is, the second proportion threshold is greater than the above-mentioned minimum proportion requirement for training a deep learning model.

[0116] In practical applications, the second ratio threshold is much greater than the above-mentioned minimum ratio required to train a deep learning model. The second image database data set may itself be a first image sample set of a preset type.

[0117] Taking the preset type of image as vulgar image as an example, the second image database can be an open source vulgar image dataset (NSFW, Not Safe For Work). NSFW means that the content is not suitable for browsing during working hours. By extracting the preset type of image samples from the open source NSFW, a first image sample set of the preset type is obtained, which provides a large number of samples for the second model to learn the image semantic information of vulgar images, so that the first model after the image semantic information transfer learning based on the vulgar images has a better expression of the semantic information of the vulgar images, so that the first model can accurately identify the semantic information of the vulgar images.

[0118] In the above embodiment, by obtaining the first image samples of the preset type from the second image database data set in which the proportion of the number of images of the preset type in the second image database data set is greater than the second proportion threshold, the shortage of image samples of the preset type in the first image database data set can be compensated, thereby providing a more favorable guarantee for the image samples of the preset type for training the second model, and further providing a basis for the first model to accurately identify the semantic information of the preset type of image.

[0119] As mentioned above, the first model can be used as a model to indirectly output the semantic information of the target image, so that the computer device can more accurately recognize the preset type of image. Based on this, as another optional embodiment, determining the semantic information of the target image based on the first model and the target image may include:

[0120] Determine a third model based on the first model;

[0121] The target image is processed using the third model to determine the semantic information of the target image.

[0122] In some embodiments, determining the third model based on the first model may include: using the model parameters of the first model to train the recognition model to obtain the third model. Exemplarily, the recognition model may be a deep learning model for image recognition stored in the computer device itself, or it may be a recognition model obtained by the computer device from a server or the cloud based on an image recognition request. The recognition model here is a model trained by using images randomly obtained from big data in the cloud as image samples.

[0123] In this way, the existing recognition model is trained according to the first model to obtain a third model that can better recognize images of a preset type, so that the semantic information of the target image can be accurately determined by processing the target image using the third model.

[0124] In other embodiments, training the recognition model using the model parameters of the first model to obtain the third model may include: performing transfer learning training on the recognition model using the model parameters of the first model to obtain the third model.

[0125] It should be noted that, according to the first model, there is no limitation on the form of determining the third model. In short, the third model determined according to the first model has the recognition function of the first model, that is, the third model determined according to the first model can enable the third model to have the ability to express the semantic information of the preset type of image. Based on this, in the embodiment of the present disclosure, by using the first model to adjust the existing recognition model to obtain the third model, and then using the third model to process the target image to determine the semantic information of the target image, the computer device can accurately identify the semantic information of the image whether online or offline, especially accurately identify the semantic information of the preset image, and then better classify the image, thereby improving the image classification performance of the computer device.

[0126] In some embodiments, determining the semantic information of the target image according to the first model and the target image may include:

[0127] Performing transfer learning training on M fourth models according to the model parameters of the first model to obtain M fifth models, wherein M is a positive integer greater than or equal to 1, and the fourth model includes: a pre-trained model obtained by training with the second image sample set in the cloud database;

[0128] Performing integrated learning on the M fifth models to obtain the third model;

[0129] The target image is processed using the third model to determine the semantic information of the target image.

[0130] It should be noted that the above-mentioned images randomly obtained through big data in the cloud can be used as image samples to train a recognition model, and another recognition model can be obtained by training with images randomly obtained again as image samples. It can be understood that in some embodiments, the fourth model is one of the above-mentioned recognition models. In order to further improve the recognition accuracy of the third model, the embodiment of the present disclosure can integrate information of multiple recognition models to obtain a more stable recognition model, thereby improving the accuracy of image recognition.

[0131] It is understandable that in the supervised learning algorithm of machine learning, our goal is to learn a stable model that performs well in all aspects. But in reality, it is often not so ideal. Sometimes we can only get multiple preferred models, that is, weakly supervised models. Ensemble learning is to combine multiple weakly supervised models here in order to get a better and more comprehensive strong supervised model. It is understandable that in the model after ensemble learning, even if a weak classifier makes an incorrect prediction, other weak classifiers can correct the error.

[0132] In some embodiments, performing integrated learning on the M fifth models to obtain the third model may include: averaging model parameters of the M fifth models to obtain model parameters of the third model.

[0133] Of course, there are many other ways to perform ensemble learning on the M fifth models, not limited to the averaging method. Any method that can optimize the model performance of each fifth model belongs to the ensemble learning method mentioned in this embodiment.

[0134] In the above embodiment, by using the model parameters of the first model to perform transfer learning on multiple fourth models respectively, multiple fifth models are obtained, and then the third model obtained by ensemble learning of the multiple fifth models can make the recognition accuracy of the third model higher and more effective in identifying preset types of images.

[0135] As another optional embodiment, training the fourth image model according to the second image sample set includes:

[0136] Dividing the second image sample set into a first positive image sample set and a first negative image sample set, wherein the images in the positive image sample set are images of the preset type, and the images in the negative image sample set are not images of the preset type;

[0137] Performing data enhancement processing on the first positive image sample set to obtain a second positive image sample set, and performing data distillation processing on the first negative image sample set to obtain a second negative image sample set;

[0138] The fourth model is obtained by training according to the second positive image sample set and the second negative image sample set.

[0139] Here, the second image sample set may be the image sample set randomly extracted from the big data in the cloud as described above. Of course, the second image sample set may be an image sample set obtained from a specific server. In short, the fourth image model is trained based on the second image sample set.

[0140] It can be understood that the positive image sample set is the sample set corresponding to the target image that the model mainly wants to find. The negative image sample set is the sample set corresponding to the image other than the target image. This embodiment aims to identify the image of the preset type, so the images in the positive image sample set here can be images of the preset type, and the images in the negative image sample set are not images of the preset type.

[0141] It should be noted that, since the number of images of the preset types is small, the number requirement for training a good model cannot be met, and the number of images of types other than the preset types is large, which will lead to an imbalance in the sample types in the second image sample set used for training. Based on this, in this embodiment, the first positive image sample set is subjected to data enhancement processing to obtain the second positive image sample set, and the first negative image sample set is subjected to data distillation processing to obtain the second negative image sample set. In this way, the imbalance in the proportion of images of the preset types in the entire image sample set can be reduced.

[0142] In some embodiments, performing data enhancement processing on the first positive image sample set includes: performing data enhancement processing on image features in the first positive image sample set.

[0143] Here, data enhancement processing is performed on the image features in the first positive image sample set, including but not limited to: random offset of the image features in the first positive image sample set, or random angle rotation of the image features and random scaling of the image features. Exemplarily, the image features in the first image sample set are randomly offset by a predetermined number of pixels, such as 15 pixels, or the image features in the first image sample set are randomly rotated at an angle other than a predetermined angle, such as 10°, or the image features in the first image sample set are randomly scaled at a preset range multiple, such as 0.9 to 1.1 times.

[0144] Exemplarily, performing data enhancement processing on image features in the first positive image sample set may include: performing data enhancement processing on all image features in the first positive image sample set. In this way, omission of certain image features in the first positive image sample set can be reduced, thereby improving the accuracy of the model.

[0145] Exemplarily, performing data enhancement processing on the images in the first positive image sample set may also include: performing data enhancement processing on predetermined image features in the first positive image sample set. The predetermined image features here may be image features containing objects of a preset type. In this way, by enhancing only the predetermined image features, interference from other image information can be reduced, thereby improving the learning efficiency of the model.

[0146] For example, see Figure 2 , Figure 2 is a schematic diagram of a scene of an image processing method according to an exemplary embodiment. Figure 2 As shown, taking the image in the first positive image sample set as a vulgar image as an example, this embodiment performs an upward shift of a predetermined number of pixels on all image features in the first positive image sample.

[0147] For example, see Figure 3 , Figure 3 is another schematic diagram of a scene of an image processing method according to an exemplary embodiment. Figure 3 As shown, also taking the case where the images in the first positive image sample set are vulgar images as an example, this embodiment rotates all image features in the first positive image samples by a predetermined angle.

[0148] In some embodiments, performing data distillation processing on the first negative image sample set includes: performing data distillation processing on image features in the first negative image sample set.

[0149] It can be understood that the data distillation process can compress a large number of first negative image sample sets into less data, so that the compressed data can also play the same training effect as the data samples of the original data as data samples.

[0150] It can be understood that most of the second image samples obtained online are first negative image samples. In order to make the phenomenon of extreme imbalance between positive image samples and negative image samples in the second image samples, in the implementation of the present disclosure, the first positive image sample set in the second image samples is subjected to data enhancement processing to obtain the second positive image sample set, and the first negative image sample in the second image samples is subjected to data distillation processing to obtain the second negative image sample set, and then the fourth model is determined based on the second positive image sample set and the second negative image samples, so that the fourth model can more effectively identify positive images, that is, more effectively identify images of preset types.

[0151] It should be added that sometimes, even if the first positive image sample set in the second image sample set is subjected to data enhancement processing, and the first negative image sample set in the second image sample set is subjected to data distillation processing, the ratio of the number of images in the second positive image sample set to the number of images in the second negative image sample set is still very small and cannot meet the training requirements of the second positive image sample set for the model.

[0152] Based on this, in some embodiments, training the fourth model according to the second positive image sample set and the second negative image sample set includes:

[0153] In response to a ratio of the number of images of the second positive image sample set to that of the second negative image sample set being less than a ratio threshold, sampling the second negative image sample set to obtain a third negative image sample set, wherein the ratio of the number of images of the second image sample set to that of the third negative image sample set is greater than or equal to the ratio threshold;

[0154] The fourth model is obtained by training according to the second positive image sample set and the third negative image sample set.

[0155] The ratio threshold here may be a ratio of the number of images in the second positive image sample set that can be used for training to the number of images in the second negative image sample set that is determined by artificial experience.

[0156] In an embodiment of the present disclosure, if the ratio of the number of images in the second positive image sample set to that in the second negative image sample set is less than a ratio threshold, sampling processing is performed on the second negative image sample set, which can further reduce the difference in the number of images between the second positive image sample set and the second negative image sample set, and reduce the uneven distribution of the second positive image sample set and the second negative image sample set, so that the trained fourth model can more effectively identify positive images, that is, more effectively identify images of a preset type.

[0157] As another optional embodiment, the third model is capable of dividing the image samples into a correctly classified sample set and an incorrectly classified sample set;

[0158] The method further comprises:

[0159] In response to the third model not satisfying the convergence condition, the third model is adjusted until the adjusted third model satisfies the convergence condition, wherein the convergence condition is: the proportion of the N-th misclassified image sample set obtained by classifying the N-1-th misclassified image sample set using the third model adjusted for the N-1-th time to the N-1-th misclassified image sample set is less than a proportion threshold, wherein N is a positive integer greater than or equal to 1, and when N is equal to 1, the N-1-th misclassified image sample set is the third image sample set;

[0160] The third image sample set is classified by using the third model adjusted for the N-1th time to obtain an Nth correctly classified image sample set, and the Nth correctly classified image sample set is used as the second image sample set.

[0161] It can be understood that the third model here includes: a classifier, which is a classifier that can classify samples and put the wrongly classified samples into the negative sample set and continue training. Through this classifier, difficult-to-classify samples can be eliminated and put into the next round of classifiers for retraining.

[0162] Exemplarily, in response to the third model not satisfying the convergence condition, adjusting the third model until the adjusted third model satisfies the convergence condition may include:

[0163] In response to the third model not satisfying the convergence condition, using the first classifier to perform a first classification on the third image sample set to obtain a first correctly classified image sample set and a first incorrectly classified image sample set;

[0164] In response to the fact that the proportion of the number of images in the first misclassified image sample set to the number of images in the third image sample set is less than a proportion threshold, the first classifier is used as the adjusted third model.

[0165] Exemplarily, the method further includes:

[0166] In response to the fact that the proportion of the number of images in the first misclassified image sample set to the image data of the third image sample set is greater than or equal to the proportion threshold, using the Sth classifier to perform the Sth classification on the S-1th misclassified image sample set to obtain the Sth correctly classified image sample set and the Sth misclassified image sample set, wherein S is greater than or equal to 2;

[0167] In response to the fact that the proportion of the number of images in the S-th incorrectly classified image sample set to the number of images in the S-1-th incorrectly classified image sample set is less than the proportion threshold, the S-th classifier is used as the adjusted third model.

[0168] In the disclosed embodiment, by iteratively training the third model for difficult sample mining, the recognition accuracy of the third model can be improved, so that the semantic information of the target image recognized by the third model is more accurate, thereby providing a basis for determining whether the target image is a preset type of image based on the semantic information of the target image.

[0169] It should be supplemented that the method further comprises:

[0170] In response to the target image being an image of the preset type, image processing is performed on the image of the preset type.

[0171] Here, performing image processing on the preset type of image includes: performing image processing on all image features of the preset type of image, or performing image processing on preset image features of the preset type of image.

[0172] Here, image processing includes but is not limited to: hiding processing or deleting processing, wherein hiding processing includes but is not limited to coding processing.

[0173] It should be added that class activation mapping (CAM) is a capability of convolutional neural network (CNN) that can locate key parts in an image. Here, the image processing of preset image features on the preset type of image includes: firstly performing CAM processing on the preset type of image to locate the preset image features; and then performing image processing on the preset image features.

[0174] See also Figure 4 , Figure 4 is another scene schematic diagram of an image processing method according to an exemplary embodiment. Figure 4 As shown, still taking the preset type of image as a vulgar image as an example, a specific object in the vulgar image, that is, an object of a type corresponding to the preset type, can be located, so that image processing can be performed on the image features later. Figure 4 The left side of the figure shows an image of a preset type, and the right side shows an image of the preset type with preset image features located.

[0175] In this way, the preset image features of the preset type of image are located through the CAM function of the convolution application network CNN, and the located preset image features are processed accordingly, which can reduce the amount of image processing data when purifying the network environment where the existing preset type of image is located.

[0176] Furthermore, the present disclosure also provides a specific implementation to further understand the image processing method provided by the embodiment of the present disclosure.

[0177] With the development of Internet technology, users can obtain or publish various information on the Internet, for example, search, browse or publish various pictures. However, some categories of pictures are not suitable for circulation on the Internet, such as vulgar pictures. In order to identify whether the pictures obtained or published on the Internet are vulgar pictures, the inventors used neural networks for identification and detection in the relevant technology. However, the recognition accuracy of the existing deep learning model is closely related to the number of samples used in the training of the deep learning model. It is understandable that if the number of samples of a certain type of image used to train the deep learning model is insufficient, it will inevitably affect the recognition of the deep learning model for such images. It is known that for vulgar pictures, the number of pictures of related categories that users can find as samples is very small. Therefore, the existing deep learning model has low recognition accuracy for vulgar pictures, and even cannot recognize them. Identifying vulgar images and purifying the network environment have become an indispensable part of major Internet giants or Internet providers when conducting network content review.

[0178] How to effectively identify vulgar images and classify them from any images has become a technical problem that needs to be solved urgently.

[0179] See also Figure 5 , Figure 5 is a flowchart of an image processing method according to a specific embodiment. Figure 5 As shown, the method includes:

[0180] Step 51: Use the first image database data set to train and obtain a second model;

[0181] The first image database dataset here may be the ImageNet dataset mentioned above.

[0182] Step 52: Obtain a set of vulgar image samples from the second image database data set.

[0183] The second image database data set here may be the open source vulgar picture data set mentioned above, and the vulgar image sample set here may be the first image sample set of the preset type mentioned above.

[0184] Step 53: Perform transfer learning training on the second model based on the vulgar image sample set to obtain the first model.

[0185] Step 54: Perform data enhancement and data distillation processing on the first preset image sample set, and train M first base models based on the processed second preset image sample set to obtain a second base model.

[0186] The first base model here is the recognition model mentioned above, and the second base model here is the fourth model mentioned above; the first preset image sample set here is the second image sample set mentioned above, and the second image sample set here is the second positive image sample set and the second negative image sample set mentioned above.

[0187] Step 55: Perform transfer learning training on the M second base models according to the model parameters of the first model to obtain M third base models.

[0188] The third base model here is the fifth model.

[0189] Step 56: Perform ensemble learning on the M base models to obtain a third model.

[0190] Here, the method of ensemble learning of the M base models is the same as the method of ensemble learning described in the above embodiment.

[0191] Step 57: If the third model does not meet the convergence condition, iterative training of the third model using difficult data mining processing is performed until the third model meets the convergence condition.

[0192] Here, the manner of iterative training of the difficult data mining process for the third model is the same as the manner of iterative training of the difficult data mining process described in the above embodiment.

[0193] The disclosed embodiment, through multiple transfer learning, data distillation, data enhancement, ensemble learning and iterative training of difficult sample mining processing, etc., obtains a deep learning model, which can well solve the problem of insufficient samples of vulgar pictures, and the inability to identify vulgar images or wrong identification under extremely unbalanced samples. Through the disclosed embodiment, vulgar images can be identified, and the semantic information of vulgar images can be better parsed based on CAM technology, and based on this semantic information, image classification can be better performed, thereby improving the classification performance of computer equipment.

[0194] Figure 6 is a functional module diagram of an image processing according to an exemplary embodiment. Figure 6 The device includes: a first acquisition module 61, a first determination module 62 and a second determination module 63; wherein,

[0195] The first acquisition module 61 is used to acquire a target image to be processed;

[0196] The first determination module 62 is used to determine the semantic information of the target image according to the first model and the target image, wherein the first model includes: a model obtained by performing transfer learning training on a second model based on a first image sample set of a preset type, and the second model includes: a pre-trained model obtained by training using a first image database data set, wherein the proportion of image data of the first image sample set of the preset type in the number of images in the first image database data set is less than a first proportion threshold;

[0197] The second determination module 63 is used to determine whether the target image is an image of the preset type according to the semantic information of the target image.

[0198] In some optional embodiments, the device further comprises:

[0199] A second acquisition module is used to acquire the first image sample set of the preset type from a second image database data set, wherein a proportion of the number of images in the first image sample set of the preset type to the number of images in the second image database data set is greater than a second proportion threshold, and the second proportion threshold is greater than the first proportion threshold.

[0200] In some optional embodiments, the first determining module 62 includes:

[0201] A first obtaining submodule is used to perform transfer learning training on M fourth models according to the model parameters of the first model to obtain M fifth models, wherein M is a positive integer greater than or equal to 1, and the fourth model includes: a pre-trained model obtained by training with a second image sample set in a cloud database;

[0202] A second obtaining submodule is used to perform integrated learning on the M fifth models to obtain the third model;

[0203] The first determination submodule is used to process the target image using the third model to determine the semantic information of the target image.

[0204] In some optional embodiments, the apparatus further includes: a third determination module, configured to determine the fourth model according to the second image sample set;

[0205] The third determining module comprises:

[0206] a division submodule, configured to divide the second image sample set into a first positive image sample set and a first negative image sample set;

[0207] A third obtaining submodule is used to perform data enhancement processing on the first positive image sample set to obtain a second positive image sample set, and to perform data distillation processing on the first negative image sample set to obtain a second negative image sample set;

[0208] The fourth obtaining submodule is used to train and obtain the fourth model according to the second positive image sample set and the second negative image sample set.

[0209] In some optional embodiments, the fourth obtaining submodule is further used for:

[0210] In response to a ratio of the number of images in the second positive image sample set to the number of images in the second negative image sample set being less than a ratio threshold, sampling the second negative image sample set to obtain a third negative image sample set, wherein the ratio of the number of images in the second image sample set to the third negative image sample set is greater than or equal to the ratio threshold;

[0211] The fourth model is obtained by training according to the second positive image sample set and the third negative image sample set.

[0212] In some optional embodiments, the third model is capable of dividing the image samples into a correctly classified sample set and an incorrectly classified sample set;

[0213] The device also includes:

[0214] an adjustment module, configured to adjust the third model in response to the third model not satisfying the convergence condition, until the adjusted third model satisfies the convergence condition, wherein the convergence condition is: the proportion of the N-th misclassified image sample set obtained by classifying the N-1-th misclassified image sample set using the third model adjusted for the N-1-th time to the N-1-th misclassified image sample set is less than a proportion threshold, wherein N is a positive integer greater than or equal to 1, and when N is equal to 1, the N-1-th misclassified image sample set is the third image sample set;

[0215] A processing module is used to classify the third image sample set by using the third model adjusted for the N-1th time to obtain an Nth correctly classified image sample set, and use the Nth correctly classified image sample set as the second image sample set.

[0216] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0217] Figure 7700 is a block diagram of a computer device 700 according to an exemplary embodiment. For example, the computer device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0218] Reference Figure 7 , the computer device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0219] The processing component 702 generally controls the overall operation of the computer device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0220] The memory 704 is configured to store various types of data to support operations on the computer device 700. Examples of such data include instructions for any application or method operating on the computer device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0221] Power component 706 provides power to various components of computer device 700. Power component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to computer device 700.

[0222] The multimedia component 708 includes a screen that provides an output interface between the computer device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the computer device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0223] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the computer device 700 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0224] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0225] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the computer device 700. For example, the sensor assembly 714 can detect the open / closed state of the computer device 700, the relative positioning of the components, such as the display and keypad of the computer device 700, and the sensor assembly 714 can also detect the position change of a component of the computer device 700 or the terminal 700, the presence or absence of contact between the user and the terminal 700, the orientation or acceleration / deceleration of the computer device 700, and the temperature change of the computer device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0226] The communication component 716 is configured to facilitate wired or wireless communication between the computer device 700 and other devices. The terminal 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0227] In an exemplary embodiment, the computer device 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0228] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by a processor 720 of a computer device 700 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0229] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute the image processing method described in the above embodiments.

[0230] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0231] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Obtaining a target image to be processed; Determine semantic information of the target image according to the first model and the target image, wherein the first model comprises: a model obtained by performing transfer learning training on a second model based on a first image sample set of a preset type, and the second model comprises: a pre-trained model obtained by training using a first image database data set, wherein a proportion of the number of images of the first image sample set of the preset type to the number of images in the first image database data set is less than a first proportion threshold; It is determined whether the target image is an image of the preset type according to the semantic information of the target image.

2. The method according to claim 1, characterized in that The method further comprises: The first image sample set of the preset type is obtained from a second image database data set, wherein a proportion of the number of images in the first image sample set of the preset type to the number of images in the second image database data set is greater than a second proportion threshold, and the second proportion threshold is greater than the first proportion threshold.

3. The method according to claim 1 or 2, characterized in that: The determining, according to the first model and the target image, the semantic information of the target image includes: Performing transfer learning training on M fourth models according to the model parameters of the first model to obtain M fifth models, wherein M is a positive integer greater than or equal to 1, and the fourth model includes: a pre-trained model obtained by training with the second image sample set in the cloud database; Performing integrated learning on the M fifth models to obtain a third model; The target image is processed using the third model to determine the semantic information of the target image.

4. The method according to claim 3, characterized in that The fourth model is obtained by training according to the second image sample set, comprising: Dividing the second image sample set into a first positive image sample set and a first negative image sample set, wherein the images in the positive image sample set are images of the preset type, and the images in the negative image sample set are not images of the preset type; Performing data enhancement processing on the first positive image sample set to obtain a second positive image sample set, and performing data distillation processing on the first negative image sample set to obtain a second negative image sample set; The fourth model is obtained by training according to the second positive image sample set and the second negative image sample set.

5. The method according to claim 4, characterized in that The step of training the fourth model according to the second positive image sample set and the second negative image sample set comprises: In response to a ratio of the number of images of the second positive image sample set to that of the second negative image sample set being less than a ratio threshold, sampling the second negative image sample set to obtain a third negative image sample set, wherein the ratio of the number of images of the second image sample set to that of the third negative image sample set is greater than or equal to the ratio threshold; The fourth model is obtained by training according to the second positive image sample set and the third negative image sample set.

6. The method according to claim 3, characterized in that The third model can divide the image samples into a correctly classified sample set and an incorrectly classified sample set; The method further comprises: In response to the third model not satisfying the convergence condition, the third model is adjusted until the adjusted third model satisfies the convergence condition, wherein the convergence condition is: the proportion of the N-th misclassified image sample set obtained by classifying the N-1-th misclassified image sample set using the third model adjusted for the N-1-th time to the N-1-th misclassified image sample set is less than a proportion threshold, wherein N is a positive integer greater than or equal to 1, and when N is equal to 1, the N-1-th misclassified image sample set is the third image sample set; The third image sample set is classified by using the third model adjusted for the N-1th time to obtain an Nth correctly classified image sample set, and the Nth correctly classified image sample set is used as the second image sample set.

7. An image processing device, characterized in that: include: A first acquisition module, used for acquiring a target image to be processed; A first determination module is configured to determine semantic information of the target image based on a first model and the target image, wherein the first model comprises: a model obtained by performing transfer learning training on a second model based on a first image sample set of a preset type, and the second model comprises: a pre-trained model obtained by training using a first image database data set, wherein a proportion of the number of images of the first image sample set of the preset type in the number of images in the first image database data set is less than a first proportion threshold; The second determination module is used to determine whether the target image is an image of the preset type according to the semantic information of the target image.

8. The device according to claim 7, characterized in that The device also includes: A second acquisition module is used to acquire the first image sample set of the preset type from a second image database data set, wherein a proportion of the number of images in the first image sample set of the preset type to the number of images in the second image database data set is greater than a second proportion threshold, and the second proportion threshold is greater than the first proportion threshold.

9. The device according to claim 7 or 8, characterized in that The first determining module includes: A first obtaining submodule is used to perform transfer learning training on M fourth models according to the model parameters of the first model to obtain M fifth models, wherein M is a positive integer greater than or equal to 1, and the fourth model includes: a pre-trained model obtained by training with a second image sample set in a cloud database; A second obtaining submodule is used to perform integrated learning on the M fifth models to obtain a third model; The first determination submodule is used to process the target image using the third model to determine the semantic information of the target image.

10. The device according to claim 9, characterized in that The device further includes: a third determination module, configured to determine the fourth model according to the second image sample set; The third determining module comprises: a division submodule, configured to divide the second image sample set into a first positive image sample set and a first negative image sample set; A third obtaining submodule is used to perform data enhancement processing on the first positive image sample set to obtain a second positive image sample set, and to perform data distillation processing on the first negative image sample set to obtain a second negative image sample set; The fourth obtaining submodule is used to train and obtain the fourth model according to the second positive image sample set and the second negative image sample set.

11. The device according to claim 10, characterized in that The fourth submodule is further used for: In response to a ratio of the number of images in the second positive image sample set to the number of images in the second negative image sample set being less than a ratio threshold, sampling the second negative image sample set to obtain a third negative image sample set, wherein the ratio of the number of images in the second image sample set to the third negative image sample set is greater than or equal to the ratio threshold; The fourth model is obtained by training according to the second positive image sample set and the third negative image sample set.

12. The device according to claim 9, characterized in that The third model can divide the image samples into a correctly classified sample set and an incorrectly classified sample set; The device also includes: an adjustment module, configured to adjust the third model in response to the third model not satisfying the convergence condition, until the adjusted third model satisfies the convergence condition, wherein the convergence condition is: the proportion of the N-th misclassified image sample set obtained by classifying the N-1-th misclassified image sample set using the third model adjusted for the N-1-th time to the N-1-th misclassified image sample set is less than a proportion threshold, wherein N is a positive integer greater than or equal to 1, and when N is equal to 1, the N-1-th misclassified image sample set is the third image sample set; A processing module is used to classify the third image sample set by using the third model adjusted for the N-1th time to obtain an Nth correctly classified image sample set, and use the Nth correctly classified image sample set as the second image sample set.

13. A computer device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, implements the method steps described in any one of claims 1 to 6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method steps described in any one of claims 1 to 6.

Images